Knowledge graph construction method and system based on large model technology

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a knowledge graph construction method and system based on a large model technology, and the method comprises the following steps: carrying out large model enhanced data annotation; performing entity extraction, relationship extraction, event extraction and causal relationship extraction by utilizing the large model; performing entity type extraction, relation type extraction, event type extraction and knowledge system extraction by utilizing the large model; the knowledge graph data is complemented by using a large model, so that the knowledge complementing capability is improved; performing relation reasoning and event reasoning by using the large model; the knowledge graph construction method and system based on the large model technology have the beneficial effects that the large model masters strong semantic understanding and context reasoning capabilities through deep learning and wide corpus training. Therefore, the knowledge graph can more accurately analyze the complex semantic relationship in the text and identify important entities and relationships hidden in the text, so that the information is effectively classified and integrated.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and specifically to a knowledge graph construction method and system based on big model technology. Background Art

[0002] Knowledge modeling is to perform business abstraction and business modeling based on the industry's application attributes, knowledge characteristics, and actual needs, in accordance with the knowledge graph model, mainly entity definition, relationship definition, and attribute definition.

[0003] In the existing technology, knowledge graph construction is difficult, inefficient, and difficult to ensure quality. The reasons are as follows:

[0004] (1) Difficulty in ontology modeling: Ontology construction requires high domain expertise and construction experience. There are still technical challenges in the identification and alignment of entities and relationships, ontology expansion and updating, ontology evaluation, and ontology fusion.

[0005] (2) Low efficiency of corpus annotation: Constructing a knowledge graph requires corpus standards, but corpus data annotation still relies heavily on manual labor, which leads to problems such as low annotation efficiency and strong subjectivity.

[0006] (3) Quality assurance is difficult: The construction of knowledge extraction rules still mainly relies on manual model building, which is highly subjective, resulting in poor portability and error propagation, making it difficult to ensure the quality of knowledge extraction. Summary of the invention

[0007] The purpose of the present invention is to provide a knowledge graph construction method and system based on big model technology to solve the problems raised in the above background technology.

[0008] To achieve the above object, the present invention provides the following technical solution: a method for constructing a knowledge graph based on a large model technology, the method comprising the following steps:

[0009] Large model enhanced data annotation;

[0010] Use large models to extract entities, relationships, events, and causal relationships;

[0011] Use big models to extract entity types, relationship types, event types, and knowledge systems; Entity type extraction: select a suitable big model and pre-train it on general corpus, fine-tune it on annotated entity type datasets, so that the model can recognize and classify different types of entities, input text into the model, and use sequence labeling technology to extract entities and their corresponding types; Relation type extraction: input the context information of two entities in the text into the big model, generate a context vector representation, and predict the relationship type between entity pairs by fine-tuning the model. In some methods, entity recognition and relationship extraction tasks are trained at the same time, and shared features are used to improve classification effects; Event type extraction: use big models to identify the trigger words of events and preliminarily determine the event type. Use trigger words and context information to predict the event type. On the basis of identifying the event type, further extract event-related parameter information; Knowledge system extraction: use big models to extract entity, relationship, and event information from text and build a knowledge graph. Use models to predict the categories and hierarchical structures of knowledge items, gradually build a complete knowledge system, and continuously fine-tune the model based on the expansion and feedback of the knowledge base to improve the accuracy of extraction and classification;

[0012] Use large models to complete knowledge graph data and improve knowledge completion capabilities;

[0013] Use large models to complete term definitions, align and standardize terms, standardize entities, and extract and merge synonyms; Complete term definitions: Collect a large amount of text data in related fields, including literature, technical documents, encyclopedias and other resources. Preprocess these texts to clean up noise and redundant information; Term alignment: In cross-language or cross-domain alignment, collect term libraries in different languages ​​or fields, and vectorize these terms through pre-trained multilingual models or cross-domain models; Real-time application and maintenance: Update models and corpora regularly to ensure that models can cope with the emergence of new terms and changes in definitions, and integrate term completion and alignment functions into automated tools to facilitate rapid response and processing of term management tasks in practical applications;

[0014] Use large models for relational reasoning and event reasoning.

[0015] Preferably, the steps of using a large model to enhance data annotation are as follows:

[0016] Use of pre-trained models: Select a large model that has been pre-trained on a large-scale dataset, pre-train these models on massive data, and learn common semantic representations;

[0017] Semi-supervised learning: Use a small amount of labeled data to conduct preliminary training on a large model, use the model to predict a large amount of unlabeled data, and add the prediction results with high confidence as pseudo labels to the training set for secondary training;

[0018] Active learning: Through uncertainty sampling and entropy maximization strategies, the most informative unlabeled samples are selected and given priority for manual labeling;

[0019] Data augmentation and generation: Using generative adversarial networks (GANs) or autoregressive models to generate synthetic data is another way to enhance labeled data. These synthetic data simulate the distribution of real data, thereby expanding the diversity of the training set.

[0020] Label correction and consistency regularization: re-evaluate the labeled data through a large model, correct potential labeling errors, and ensure label consistency:

[0021] Data labeling automation platform: Integrate the above technologies into an automated data labeling platform, combine human-computer interaction, and allow artificial intelligence and human labelers to collaborate to complete labeling tasks.

[0022] Preferably, the steps of using the large model to extract entities, relations, events, and causal relationships are as follows:

[0023] Entity extraction: Use a large-scale corpus to pre-train the model so that it can learn rich semantic representations, and fine-tune it on an entity annotation dataset. Input each sentence into the model, and the model outputs the entity label corresponding to each word. Apply rules or conditional random fields (CRF) to improve the coherence and accuracy of annotations.

[0024] Relation extraction: Input entity pairs and context into the big model, obtain the context vector representation between entities, classify them on predefined relationship categories, and fine-tune the pre-trained big model to enable it to predict the relationship category between entity pairs;

[0025] Event extraction: Use the big model to identify the trigger words of the event, use the classifier to determine whether each word is a trigger word, and after determining the trigger word, identify the parameters related to it, which is completed through sequence labeling technology;

[0026] Causal relationship extraction: Use large models to identify causal pairs in text, determine which sentences or sentence pairs have causal relationships through classification tasks, and combine dependency syntactic analysis to help the model understand the structure of sentences, accurately identify the subject and object of causal relationships, and fine-tune the model on specific causal relationship datasets so that it can accurately identify complex causal relationships.

[0027] Preferably, the specific operations of using the large model to complete the knowledge graph data include:

[0028] Knowledge representation learning: Convert entities and relations in the knowledge graph into vector representations, usually using graph embedding-based techniques, pre-training language models on large-scale corpora to allow the models to learn rich contextual semantic representations. The pre-trained models are fine-tuned to adapt to the knowledge graph completion task.

[0029] Link prediction: Use the trained knowledge graph embedding representation and pre-trained model to train a link prediction model. The model predicts possible missing relationships or entities by inputting existing entities and relationships.

[0030] Generation and completion of entities and relationships: Generate potential entities or relationships using generative models. When there is some missing data in the knowledge graph, the missing parts are predicted through generative models. The language understanding ability of large models is used to infer the missing entities and relationships in the knowledge graph by analyzing the context text.

[0031] Multimodal fusion: Combine text, image, and audio multimodal data, use large models to extract information from multi-source data, complete the multimodal content in the knowledge graph, and use the transfer learning capabilities of large models to apply models trained in one field to complete knowledge graphs in other fields to improve versatility.

[0032] Preferably, the specific operations of using the large model for relational reasoning and event reasoning include:

[0033] Data collection and preprocessing: Collect relationship and event-related data from various text data sources. The data includes entity, event, time, and location information. Clean, deduplicate, annotate, and format the collected data to make it suitable for training and reasoning of large models.

[0034] Model pre-training and fine-tuning: Choose large models such as BERT and GPT, pre-train them on large-scale corpora, master a wide range of language and knowledge representations, and fine-tune the models on specially annotated relationship and event datasets so that the models can perform specific tasks more accurately;

[0035] Relational reasoning: Use a large model to extract feature information between entity pairs from text, including contextual information and entity attributes. Based on the extracted features, the model classifies the relationships between entity pairs. It uses a multi-task learning approach to process multiple relationship types at the same time. For complex relationships, the model performs chain reasoning, that is, deriving complex relationships from simple relationships through a multi-step reasoning process.

[0036] Event reasoning: Use large models to identify event trigger words in text and preliminarily determine possible events. The model analyzes the causal relationship between events to infer the possible results or causes of an event. It usually combines time series information for reasoning, performs sequence modeling on related events, and predicts the development trend of future events or possible subsequent events.

[0037] A knowledge graph construction system based on big model technology, applied to a knowledge graph construction method based on big model technology, the system comprising:

[0038] Data annotation enhancement module, used to enhance the accuracy and efficiency of data annotation through large model technology;

[0039] Information extraction module, used to extract entities, relations, events and causal relations using large models;

[0040] Type extraction and knowledge system construction module, used to extract entity types, relationship types, event types, and construct a knowledge system;

[0041] The knowledge graph completion module is used to complete the data in the knowledge graph and improve the completeness and accuracy of the knowledge graph;

[0042] The term and entity standardization alignment module is used to complete term definitions, align and standardize terms, standardize entity alignment, and extract and merge synonyms.

[0043] Reasoning module, used for relational reasoning and event reasoning.

[0044] Preferably, the data annotation enhancement module further includes:

[0045] Pre-trained model usage unit, used to select and apply large models pre-trained with large-scale datasets;

[0046] Semi-supervised learning unit, which is used to train models using a small amount of labeled data and a large amount of unlabeled data and generate pseudo labels;

[0047] Active learning unit, used to select the most informative unlabeled samples for manual labeling through uncertainty sampling and entropy maximization strategy;

[0048] Data augmentation and generation unit, used to generate synthetic data using generative adversarial networks or autoregressive models;

[0049] Label correction and consistency regularization unit, used to re-evaluate the labeled data and correct potential labeling errors;

[0050] The data labeling automation platform unit is used to integrate the above technologies to realize human-computer interactive data labeling tasks.

[0051] Preferably, the information extraction module further comprises:

[0052] The entity extraction unit is used to pre-train the model using a large-scale corpus and fine-tune it on an entity annotation dataset to extract entities from text;

[0053] The relation extraction unit is used to input entity pairs and context into the large model, obtain the context vector representation between entities, and predict the relationship category between entity pairs;

[0054] An event extraction unit, which is used to identify the trigger words of events using the large model and to identify the parameters related to them;

[0055] The causal relationship extraction unit is used to identify causal pairs in text using a large model and accurately identify the subject and object of the causal relationship.

[0056] Preferably, the knowledge graph completion module further includes:

[0057] Knowledge representation learning unit, used to convert entities and relations in the knowledge graph into vector representations;

[0058] Link prediction unit, which is used to predict possible missing relations or entities using the trained knowledge graph embedding representation and pre-trained models;

[0059] Entity and relationship generation and completion unit, which is used to generate potential entities or relationships using generative models and complete the missing parts in the knowledge graph;

[0060] The multimodal fusion unit is used to combine multimodal data such as text, images, and audio, extract information from multi-source data, and complete the multimodal content in the knowledge graph.

[0061] Preferably, the reasoning module further comprises:

[0062] Data collection and preprocessing unit, which is used to collect relational and event-related data from various text data sources and perform cleaning, deduplication, annotation, and formatting;

[0063] Model pre-training and fine-tuning unit, which is used to select large models and pre-train them on large-scale corpora, and then fine-tune them on specially annotated relationship and event datasets;

[0064] The relational reasoning unit is used to extract feature information between entity pairs from the text and classify the relationship between entity pairs;

[0065] The event reasoning unit is used to identify event trigger words in the text, analyze the causal relationship between events, and infer the possible results or causes of an event.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] The present invention proposes a knowledge graph construction method and system based on big model technology. The big model has mastered powerful semantic understanding and contextual reasoning capabilities through deep learning and extensive corpus training. This enables the knowledge graph to more accurately parse complex semantic relationships in the text, identify important entities and relationships hidden in the text, and then effectively classify and integrate this information.

[0068] The introduction of big models has revolutionized this process to a certain extent. The contextual understanding ability of big models enables them to automatically extract valuable knowledge points from large amounts of unstructured data and perform knowledge modeling and fusion. This means that the degree of manual intervention can be reduced, greatly reducing the cost of building and maintaining knowledge graphs.

[0069] The generation capability of large models enables the knowledge graph to generate answers, summaries or suggestions in natural language, thereby improving the service effect of the knowledge graph system. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 Constructing a flow chart for the knowledge graph of the present invention;

[0071] Figure 2 Construct a full process flow diagram for the knowledge graph of the present invention. DETAILED DESCRIPTION

[0072] In order to make the purpose and technical solution of the present invention clearly and completely described, and the advantages more clearly understood, the embodiments of the present invention are further described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are part of the embodiments of the present invention, rather than all of the embodiments, and are only used to explain the embodiments of the present invention, and are not used to limit the embodiments of the present invention. All other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] For example, see Figure 1 to Figure 2 The present invention provides a technical solution: a method for constructing a knowledge graph based on a large model technology, the method comprising the following steps:

[0074] (1) Large Model Enhanced Data Annotation

[0075] The steps to enhance data annotation using a large model are as follows:

[0076] 1) Use of pre-trained models

[0077] Choose a large model that has been pre-trained on a large dataset (such as BERT, GPT, etc.). Pre-training these models on massive data can learn universal semantic representations.

[0078] 2) Semi-supervised learning

[0079] In semi-supervised learning, a small amount of labeled data is used to perform preliminary training on a large model. Then, the model is used to make predictions on a large amount of unlabeled data, and the prediction results with high confidence are added to the training set as pseudo labels for secondary training. This can greatly expand the scale of labeled data and improve the performance of the model.

[0080] 3) Active learning

[0081] In the process of active learning, the large model can evaluate which samples are most important for improving model performance. Through strategies such as uncertainty sampling and entropy maximization, the most informative unlabeled samples are selected and manually labeled first. This method can maximize model performance with the least amount of labeling work.

[0082] 4) Data enhancement and generation

[0083] Generating synthetic data using generative adversarial networks (GANs) or autoregressive models such as GPT is another way to enhance labeled data. These synthetic data can simulate the distribution of real data, thereby expanding the diversity of the training set. This technique is particularly important when data is scarce.

[0084] 5) Label correction and consistency regularization

[0085] The labeled data can be re-evaluated through the large model to correct potential labeling errors and ensure label consistency. At the same time, the consistency regularization technology can be used to ensure the consistency of the model's predictions for the same data under different conditions, thereby improving the robustness of the model.

[0086] 6) Data annotation automation platform

[0087] Integrating the above technologies into an automated data annotation platform and combining it with human-computer interaction allows artificial intelligence and human annotators to collaborate to complete the annotation task. This platform can significantly reduce the time and cost of manual annotation while improving the quality of annotation.

[0088] (2) Use large models to extract entities, relationships, events, causal relationships, etc.

[0089] The steps for using the big model to extract entities, relationships, events, and causal relationships are as follows:

[0090] 1) Entity extraction

[0091] Pre-train the model using a large-scale corpus to learn rich semantic representations. Fine-tune on entity annotation datasets (such as CoNLL-2003). Input each sentence into the model, and the model outputs the entity label corresponding to each word (such as B-PER, I-PER). Apply rules or conditional random fields (CRF) to improve the coherence and accuracy of annotations.

[0092] 2) Relationship Extraction

[0093] Input entity pairs and context into a large model (such as BERT) to obtain context vector representations between entities. Classify on predefined relationship categories. Fine-tune the pre-trained large model to enable it to predict the relationship category between entity pairs. In some methods, entity and relationship extraction tasks are trained simultaneously to share representations and improve performance.

[0094] 3) Event extraction

[0095] Using large models (such as BERT) to identify the trigger words of events is the core step of event extraction. Usually, a classifier is used to determine whether each word is a trigger word. After determining the trigger word, the parameters related to it (such as participating entities, time, and location) are identified. This can be done through sequence labeling technology. Trigger word identification and event parameter identification can also be trained as a joint task to improve the overall effect of extraction.

[0096] 4) Causal Relationship Extraction

[0097] Use a large model to identify causal pairs in text, and use classification tasks to determine which sentences or sentence pairs have causal relationships. Combined with dependency syntactic analysis, help the model understand the structure of sentences and accurately identify the subject and object of causal relationships. Fine-tune the model on a specific causal relationship dataset so that it can accurately identify complex causal relationships.

[0098] (3) Use large models to extract entity types, relationship types, event types, knowledge systems, etc.

[0099] 1) Entity type extraction

[0100] Select a suitable large model (such as BERT, RoBERTa) and pre-train it on general corpus. Fine-tune it on a dataset of labeled entity types so that the model can recognize and classify different types of entities (such as people, places, organizations, etc.). Input the text into the model and use sequence annotation techniques (such as BIO annotation) to extract entities and their corresponding types.

[0101] 2) Relationship type extraction

[0102] The context information of two entities in the text is input into the large model to generate a context vector representation. By fine-tuning the model, the relationship type between entity pairs (such as "employment", "belonging to", etc.) is predicted. In some methods, entity recognition and relationship extraction tasks are trained simultaneously, and shared features are used to improve classification results.

[0103] 3) Event type extraction

[0104] The trigger words of the event are identified through the big model, and the event type is preliminarily determined. The event type (such as "meeting", "transaction", etc.) is predicted using the trigger words and context information. Based on the identification of the event type, the parameter information related to the event is further extracted.

[0105] 4) Knowledge system extraction

[0106] Use large models to extract entities, relationships, events and other information from texts and build knowledge graphs. Use models to predict the categories and hierarchical structures of knowledge items and gradually build a complete knowledge system. Based on the expansion and feedback of the knowledge base, continue to fine-tune the model to improve the accuracy of extraction and classification.

[0107] (4) Using large models to complete knowledge graph data

[0108] 1) Knowledge Representation Learning

[0109] Convert entities and relations in the knowledge graph into vector representations (embedding), usually using graph embedding-based techniques such as TransE and RotatE. These embedding representations can capture the semantic similarity and structural information between entities and relations. Pre-train language models (such as BERT, GPT, etc.) on large-scale corpora to allow the models to learn rich contextual semantic representations. These pre-trained models can be adapted to the knowledge graph completion task through fine-tuning.

[0110] 2) Link Prediction

[0111] Using the trained knowledge graph embedding representation and pre-trained model, a link prediction model is trained. The model can predict possible missing relationships or entities by inputting existing entities and relationships. For example, given a relationship (A, R, ?), the model can predict entity B so that the relationship (A, R, B) holds. During the training process, negative samples (i.e., non-existent relationships) are introduced to optimize the model's discrimination ability and avoid model overfitting.

[0112] 3) Generation and completion of entities and relations

[0113] Generate potential entities or relationships using generative models (such as GPT). When there is some missing data in the knowledge graph, the missing part can be predicted by the generative model. Leverage the language understanding ability of the large model to infer the missing entities and relationships in the knowledge graph by analyzing the context text. For example, in the corporate relationship graph, by analyzing news reports or corporate documents, relevant business relationships can be automatically extracted and completed.

[0114] 4) Multimodal Fusion

[0115] Combining multimodal data such as text, images, and audio, using large models to extract information from multi-source data, and completing the multimodal content in the knowledge graph. For example, by analyzing the scene in the picture, the scene description and the knowledge of related entities can be completed. Using the transfer learning ability of the large model, the model trained in one field can be applied to the knowledge graph completion in other fields to improve versatility.

[0116] (5) Use large models to complete term definitions, align and standardize terms, standardize entities, and extract and merge synonyms.

[0117] 1) Completion of term definitions

[0118] Collect a large amount of text data in related fields, including literature, technical documents, encyclopedias and other resources. Preprocess these texts to clean up noise and redundant information. Select large models (such as BERT, GPT, etc.) and pre-train them on the above corpus to enable them to master the language characteristics and concepts of specific fields. Use generative models (such as GPT) to complete term definitions. Input some terms and their related contexts, and the model generates complete term definitions. Fine-tune the model on a dataset of term definitions in a specific field to make the definitions it generates more accurate and in line with field specifications. Manually verify the term definitions generated by the model to ensure the accuracy and professionalism of the definitions.

[0119] 2) Term Alignment

[0120] In cross-language or cross-domain alignment, terminology libraries from different languages ​​or domains are collected. These terms are vectorized using a pre-trained multilingual model or cross-domain model. The vector representations generated by the large model are used to calculate the similarity between different terms. Common methods include cosine similarity or Euclidean distance. The large model is trained on a predefined term alignment dataset so that it can accurately predict the alignment relationship between term pairs. Through iterative optimization, the alignment rules of the model are continuously adjusted to improve the accuracy of the alignment, especially in complex or ambiguous terms. Expert feedback is introduced to adjust the alignment results of the model to ensure the consistency of the aligned terms in semantics and application.

[0121] 3) Real-time application and maintenance

[0122] Update models and corpora regularly to ensure that the models can cope with the emergence of new terms and changes in definitions. Integrate term completion and alignment functions into automation tools to facilitate rapid response and processing of terminology management tasks in practical applications.

[0123] (7) Using large models for relational reasoning, event reasoning, etc.

[0124] 1) Data collection and preprocessing

[0125] Collect relational and event-related data from various text data sources (such as documents, social media, databases, etc.). This data can include information about entities, events, time, location, etc. Clean, deduplicate, annotate, and format the collected data to make it suitable for training and reasoning of large models.

[0126] 2) Model pre-training and fine-tuning

[0127] Choose large models such as BERT and GPT, pre-train them on large-scale corpora, and master a wide range of language and knowledge representations. Fine-tune the model on specially annotated relationship and event datasets so that the model can perform specific tasks more accurately (such as identifying complex relationships or reasoning about event causality).

[0128] 3) Relational Reasoning

[0129] The large model is used to extract feature information between entity pairs from the text, including context information, entity attributes, etc. Based on the extracted features, the model classifies the relationship between entity pairs. Multi-task learning can be used to process multiple relationship types at the same time. For complex relationships, chain reasoning can be performed through the model, that is, complex relationships can be derived from simple relationships through a multi-step reasoning process.

[0130] 4) Event Reasoning

[0131] Use the big model to identify event trigger words in the text and preliminarily determine possible events. The model infers the possible results or causes of an event by analyzing the causal relationship between events. Usually, it combines time series information for reasoning. Sequence modeling of related events is performed to predict the development trend of future events or possible subsequent events.

[0132] Embodiment 2, based on embodiment 1, proposes a knowledge graph construction system based on big model technology, which is applied to the knowledge graph construction method based on big model technology. The system includes:

[0133] The data labeling enhancement module is used to enhance the accuracy and efficiency of data labeling through large model technology; it further includes: a pre-trained model usage unit, which is used to select and apply a large model that has been pre-trained with a large-scale data set; a semi-supervised learning unit, which is used to train the model using a small amount of labeled data and a large amount of unlabeled data, and generate pseudo labels; an active learning unit, which is used to select the most informative unlabeled samples for manual labeling through uncertainty sampling and entropy maximization strategies; a data enhancement and generation unit, which is used to generate synthetic data using a generative adversarial network or an autoregressive model; a label correction and consistency regularization unit, which is used to re-evaluate the labeled data and correct potential labeling errors; a data labeling automation platform unit, which is used to integrate the above technologies to realize human-computer interactive data labeling tasks.

[0134] The information extraction module is used to use the big model to perform entity extraction, relationship extraction, event extraction and causal relationship extraction; further includes: an entity extraction unit, which is used to pre-train the model using a large-scale corpus and fine-tune it on an entity annotation dataset to extract entities in the text; a relationship extraction unit, which is used to input entity pairs and context into the big model, obtain context vector representations between entities, and predict the relationship categories between entity pairs; an event extraction unit, which is used to use the big model to identify the trigger words of events and identify the parameters related to them; a causal relationship extraction unit, which is used to use the big model to identify causal pairs in the text and accurately identify the subject and object of the causal relationship.

[0135] Type extraction and knowledge system construction module, used to extract entity types, relationship types, event types, and construct a knowledge system;

[0136] The knowledge graph completion module is used to complete the data in the knowledge graph and improve the completeness and accuracy of the knowledge graph; it further includes: a knowledge representation learning unit, which is used to convert entities and relationships in the knowledge graph into vector representations; a link prediction unit, which is used to use the trained knowledge graph embedding representation and pre-trained model to predict possible missing relationships or entities; an entity and relationship generation and completion unit, which is used to use a generative model to generate potential entities or relationships and complete the missing parts in the knowledge graph; a multimodal fusion unit, which is used to combine multimodal data such as text, images, audio, etc., to extract information from multi-source data and complete the multimodal content in the knowledge graph.

[0137] The term and entity standardization alignment module is used to complete term definitions, align and standardize terms, standardize entity alignment, and extract and merge synonyms.

[0138] The reasoning module is used for relational reasoning and event reasoning; further includes: a data collection and preprocessing unit, which is used to collect relational and event-related data from various text data sources, and clean, deduplicate, annotate and format them; a model pretraining and fine-tuning unit, which is used to select a large model and pre-train it on a large-scale corpus, and then fine-tune it on specially annotated relational and event data sets; a relational reasoning unit, which is used to extract feature information between entity pairs from the text and classify the relationship between entity pairs; an event reasoning unit, which is used to identify event trigger words in the text, analyze the causal relationship between events, and infer the possible results or causes of an event.

[0139] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A knowledge graph construction method based on large model technology, characterized by: The method comprises the following steps: Large model enhanced data annotation; Use large models to extract entities, relationships, events, and causal relationships; Use big models to extract entity types, relationship types, event types, and knowledge systems; Entity type extraction: select a suitable big model and pre-train it on general corpus, fine-tune it on annotated entity type datasets, enable the model to recognize and classify different types of entities, input text into the model, and use sequence labeling technology to extract entities and their corresponding types; Relation type extraction: input the context information of two entities in the text into the big model, generate a context vector representation, and predict the relationship type between entity pairs by fine-tuning the model. In some methods, entity recognition and relationship extraction tasks are trained at the same time, and shared features are used to improve classification effects; Event type extraction: use big models to identify the trigger words of events and preliminarily determine the event type. Use trigger words and context information to predict the event type. On the basis of identifying the event type, further extract event-related parameter information; Knowledge system extraction: use big models to extract entity, relationship, and event information from text and build a knowledge graph; use models to predict the categories and hierarchical structures of knowledge items, gradually build a complete knowledge system, and continuously fine-tune the model based on the expansion and feedback of the knowledge base to improve the accuracy of extraction and classification; Use large models to complete knowledge graph data and improve knowledge completion capabilities; Use large models to complete term definitions, align and standardize terms, standardize entities, and extract and merge synonyms; Complete term definitions: collect a large amount of text data in related fields, including literature, technical documents, encyclopedias and other resources; pre-process these texts to clean up noise and redundant information; Term alignment: In cross-language or cross-domain alignment, collect term libraries in different languages ​​or fields, and vectorize these terms through pre-trained multilingual models or cross-domain models; Real-time application and maintenance: regularly update models and corpora to ensure that models can cope with the emergence of new terms and changes in definitions, and integrate term completion and alignment functions into automated tools to facilitate rapid response and processing of term management tasks in practical applications; Use large models for relational reasoning and event reasoning.

2. According to claim 1, a knowledge graph construction method based on large model technology is characterized in that: The steps to enhance data annotation using a large model are as follows: Use of pre-trained models: Select a large model that has been pre-trained on a large-scale dataset, pre-train these models on massive data, and learn common semantic representations; Semi-supervised learning: Use a small amount of labeled data to conduct preliminary training on a large model, use the model to predict a large amount of unlabeled data, and add the prediction results with high confidence as pseudo labels to the training set for secondary training; Active learning: Through uncertainty sampling and entropy maximization strategies, the most informative unlabeled samples are selected and given priority for manual labeling; Data augmentation and generation: Using generative adversarial networks (GANs) or autoregressive models to generate synthetic data is another way to enhance labeled data. These synthetic data simulate the distribution of real data, thereby expanding the diversity of the training set. Label correction and consistency regularization: re-evaluate the labeled data through a large model, correct potential labeling errors, and ensure label consistency: Data labeling automation platform: Integrate the above technologies into an automated data labeling platform, combine human-computer interaction, and allow artificial intelligence and human labelers to collaborate to complete labeling tasks.

3. The method for constructing a knowledge graph based on a large model technology according to claim 2, characterized in that: The steps for using the big model to extract entities, relationships, events, and causal relationships are as follows: Entity extraction: Use a large-scale corpus to pre-train the model so that it can learn rich semantic representations, and fine-tune it on an entity annotation dataset. Input each sentence into the model, and the model outputs the entity label corresponding to each word. Apply rules or conditional random fields (CRF) to improve the coherence and accuracy of annotations. Relation extraction: Input entity pairs and context into the big model, obtain the context vector representation between entities, classify them on predefined relationship categories, and fine-tune the pre-trained big model to enable it to predict the relationship category between entity pairs; Event extraction: Use the big model to identify the trigger words of the event, use the classifier to determine whether each word is a trigger word, and after determining the trigger word, identify the parameters related to it, which is completed through sequence labeling technology; Causal relationship extraction: Use large models to identify causal pairs in text, determine which sentences or sentence pairs have causal relationships through classification tasks, and combine dependency syntactic analysis to help the model understand the structure of sentences, accurately identify the subject and object of causal relationships, and fine-tune the model on specific causal relationship datasets so that it can accurately identify complex causal relationships.

4. The method for constructing a knowledge graph based on a large model technology according to claim 1, characterized in that: The specific operations of using the big model to complete the knowledge graph data include: Knowledge representation learning: Convert entities and relations in the knowledge graph into vector representations, usually using graph embedding-based techniques, pre-training language models on large-scale corpora to allow the models to learn rich contextual semantic representations. The pre-trained models are fine-tuned to adapt to the knowledge graph completion task. Link prediction: Use the trained knowledge graph embedding representation and pre-trained model to train a link prediction model. The model predicts possible missing relationships or entities by inputting existing entities and relationships. Generation and completion of entities and relationships: Generate potential entities or relationships using generative models. When there is some missing data in the knowledge graph, the missing parts are predicted through generative models. The language understanding ability of large models is used to infer the missing entities and relationships in the knowledge graph by analyzing the context text. Multimodal fusion: Combine text, image, and audio multimodal data, use large models to extract information from multi-source data, complete the multimodal content in the knowledge graph, and use the transfer learning capabilities of large models to apply models trained in one field to complete knowledge graphs in other fields to improve versatility.

5. The method for constructing a knowledge graph based on a large model technology according to claim 1, characterized in that: The specific operations of using large models for relational reasoning and event reasoning include: Data collection and preprocessing: Collect relationship and event-related data from various text data sources. The data includes entity, event, time, and location information. Clean, deduplicate, annotate, and format the collected data to make it suitable for training and reasoning of large models. Model pre-training and fine-tuning: Choose large models such as BERT and GPT, pre-train them on large-scale corpora, master a wide range of language and knowledge representations, and fine-tune the models on specially annotated relationship and event datasets so that the models can perform specific tasks more accurately; Relational reasoning: Use a large model to extract feature information between entity pairs from text, including contextual information and entity attributes. Based on the extracted features, the model classifies the relationships between entity pairs. It uses a multi-task learning approach to process multiple relationship types at the same time. For complex relationships, the model performs chain reasoning, that is, deriving complex relationships from simple relationships through a multi-step reasoning process. Event reasoning: Use large models to identify event trigger words in text and preliminarily determine possible events. The model analyzes the causal relationship between events to infer the possible results or causes of an event. It usually combines time series information for reasoning, performs sequence modeling on related events, and predicts the development trend of future events or possible subsequent events.

6. A knowledge graph construction system based on big model technology, applied to the knowledge graph construction method based on big model technology according to any one of claims 1 to 5, characterized in that: The system comprises: Data annotation enhancement module, used to enhance the accuracy and efficiency of data annotation through large model technology; Information extraction module, used to extract entities, relations, events and causal relations using large models; Type extraction and knowledge system construction module, used to extract entity types, relationship types, event types, and construct a knowledge system; The knowledge graph completion module is used to complete the data in the knowledge graph and improve the completeness and accuracy of the knowledge graph; The term and entity standardization alignment module is used to complete term definitions, align and standardize terms, standardize entity alignment, and extract and merge synonyms. Reasoning module, used for relational reasoning and event reasoning.

7. A knowledge graph construction system based on large model technology according to claim 6, characterized in that: The data annotation enhancement module further includes: Pre-trained model usage unit, used to select and apply large models pre-trained with large-scale datasets; Semi-supervised learning unit, which is used to train models using a small amount of labeled data and a large amount of unlabeled data and generate pseudo labels; Active learning unit, used to select the most informative unlabeled samples for manual labeling through uncertainty sampling and entropy maximization strategy; Data augmentation and generation unit, used to generate synthetic data using generative adversarial networks or autoregressive models; Label correction and consistency regularization unit, used to re-evaluate the labeled data and correct potential labeling errors; The data labeling automation platform unit is used to integrate the above technologies to realize human-computer interactive data labeling tasks.

8. The knowledge graph construction system based on large model technology according to claim 6 is characterized by: The information extraction module further includes: The entity extraction unit is used to pre-train the model using a large-scale corpus and fine-tune it on an entity annotation dataset to extract entities from text; The relation extraction unit is used to input entity pairs and context into the large model, obtain the context vector representation between entities, and predict the relationship category between entity pairs; An event extraction unit, used to identify the trigger words of events using the large model and identify the parameters related to them; The causal relationship extraction unit is used to identify causal pairs in text using a large model and accurately identify the subject and object of the causal relationship.

9. The knowledge graph construction system based on large model technology according to claim 6 is characterized by: The knowledge graph completion module further includes: Knowledge representation learning unit, used to convert entities and relations in the knowledge graph into vector representations; Link prediction unit, which is used to predict possible missing relations or entities using the trained knowledge graph embedding representation and pre-trained models; Entity and relationship generation and completion unit, which is used to generate potential entities or relationships using generative models and complete the missing parts in the knowledge graph; The multimodal fusion unit is used to combine multimodal data such as text, images, and audio, extract information from multi-source data, and complete the multimodal content in the knowledge graph.

10. The knowledge graph construction system based on large model technology according to claim 6 is characterized by: The reasoning module further includes: Data collection and preprocessing unit, which is used to collect relational and event-related data from various text data sources and perform cleaning, deduplication, annotation, and formatting; Model pre-training and fine-tuning unit, which is used to select large models and pre-train them on large-scale corpora, and then fine-tune them on specially annotated relationship and event datasets; The relational reasoning unit is used to extract feature information between entity pairs from the text and classify the relationship between entity pairs; The event reasoning unit is used to identify event trigger words in the text, analyze the causal relationship between events, and infer the possible results or causes of an event.

Images