Method and system for constructing knowledge graph based on unsupervised learning

Abstract

The present invention relates to the field of knowledge graph technology, specifically to a method and system for constructing a knowledge graph based on unsupervised learning, the method comprising: obtaining text data, preprocessing the collected text data, and segmenting the text data to obtain segmentation results of the text data; performing entity recognition on the segmentation results of the text data based on a pre-trained language model, and performing ambiguity elimination on the entity recognition results in the text data; extracting entity relationships using an unsupervised model to generate entity and entity relationship triples; constructing an entity relationship weight calculation formula, and mapping the entity relationship weight to an entity relationship confidence, filtering entity relationships with confidence values ​​below a threshold; and constructing a knowledge graph of the text data based on the entities and entity relationships extracted from the text data. The present invention improves the efficiency and quality of knowledge graph construction, and provides important knowledge support for knowledge-driven applications such as intelligent search, question-answering, and recommendation.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph technology, and specifically to a method and system for constructing a knowledge graph based on unsupervised learning. Background Art

[0002] Knowledge graph is a structured semantic knowledge base used to describe entities and their relationships. It is widely used in intelligent search, question-answering systems, recommendation systems and other fields.

[0003] Traditional knowledge graph construction methods mainly rely on manually defined entities and large amounts of labeled data, which is time-consuming and labor-intensive.

[0004] Existing text knowledge graph construction methods are mainly divided into three categories: rule-based, supervised learning-based, and unsupervised learning-based.

[0005] Rule-based methods require manual definition of complex templates and rules, and have poor generalization capabilities; supervised learning-based methods require a large amount of high-quality labeled data, and have high labor costs; unsupervised learning-based methods do not require manual definition of rules and labeled data, and can automatically mine the knowledge contained in the text, but the quality and richness of the extracted knowledge needs to be improved.

[0006] In terms of entity recognition, traditional methods are mainly based on dictionary matching, conditional random fields and other methods, and have limited generalization capabilities.

[0007] In recent years, named entity recognition methods based on deep learning have made significant progress, especially pre-trained language models such as BERT and RoBERTa, which can make full use of the semantic knowledge in large-scale corpora and greatly improve the entity recognition effect.

[0008] However, simply identifying entities is not enough; entities with the same name need to be disambiguated and linked to standard entities in the knowledge base.

[0009] In terms of relationship extraction, early methods were mainly based on pattern matching, heuristic rules, etc., and were easily restricted by manually defined rules and patterns.

[0010] With the development of semantic parsing technology, researchers have proposed some relationship extraction methods based on supervised learning, which extract entity pairs and relationship trigger words from syntactic analysis trees or semantic dependency graphs, and then use classification models to determine the relationship type.

[0011] This type of method can handle complex sentence structures and extract implicit relationships, but it requires a large amount of labeled data.

[0012] Therefore, there is an urgent need for an end-to-end, efficient and accurate knowledge graph construction method that can fully mine the structured knowledge in text data, reduce labor costs, and improve the quality and efficiency of knowledge extraction.

[0013] The present invention is proposed in this context, and aims to provide a method and system for constructing a knowledge graph based on unsupervised learning. Summary of the Invention

[0014] To solve the above technical problems, the present invention provides the following technical solutions: a method for constructing a knowledge graph based on unsupervised learning, comprising:

[0015] Acquire text data, preprocess the collected text data, and segment the text data to obtain segmentation results of the text data;

[0016] Based on the pre-trained language model, entity recognition is performed on the word segmentation results of the text data, and the entity recognition results in the text data are disambiguated;

[0017] Use unsupervised models to extract entity relationships and generate entity and entity relationship triplets;

[0018] Construct an entity relationship weight calculation formula, map the entity relationship weight to the entity relationship confidence, and filter out entity relationships with confidence values ​​below the threshold;

[0019] Based on the entities and entity relationships extracted from text data, a knowledge graph of text data is constructed.

[0020] As a preferred solution of the method for constructing a knowledge graph based on unsupervised learning described in the present invention, the text data is preprocessed to remove label data and symbol data in the text data; the text in the text data is divided into word sequences based on a dictionary and word segmentation tools;

[0021] Based on predefined dictionaries and rules, the forward maximum matching algorithm is used to segment text: Indicated by A sequence of words: ;

[0022] Definition Dictionary , which contains Words: Scan text from left to right , find the longest word match;

[0023] set up is the current scanning position, For the end position of the word , judge the substring Is it in the dictionary In; if the substring In the dictionary, As a word, and Move to Position; if the substring If it is not in the dictionary, Move to Repeat the scanning process until , get the word segmentation results of text data .

[0024] As a preferred solution of the method for constructing a knowledge graph based on unsupervised learning described in the present invention, the RoBERTa pre-trained model is selected as the entity recognition model, and based on the RoBERTa pre-trained model, the RoBERTa pre-trained model is adjusted to add entity recognition specific network layers, including a linear layer and a CRF layer;

[0025] The word segmentation results are input into the adjusted model to obtain the label probability distribution of each word segmentation, and the Viterbi algorithm is used to decode and obtain the optimal label sequence; the entities and types are extracted to obtain the entity recognition results;

[0026] For each entity recognition result, retrieve candidate entities with the same name from the knowledge base of the RoBERTa pre-trained model and calculate the entity With candidate entities Similarity ; Select the candidate entity with the highest similarity As a link object:

[0027] in, It is a collection of entities in the knowledge base. Link to selected candidate entity , to eliminate ambiguity.

[0028] As a preferred solution of the method for constructing a knowledge graph based on unsupervised learning described in the present invention, the OpenIE model is selected as the unsupervised entity relationship extraction model;

[0029] The preprocessed text data and entity recognition results are input into the OpenIE model, which extracts entity relationships through the steps of syntactic analysis, entity recognition, relationship trigger word recognition, relationship definition, and triple generation.

[0030] The syntactic analysis includes performing syntactic analysis on the input text using a dependency parser to identify the subject, predicate and object components of the sentence;

[0031] The entity recognition includes locating entities in a sentence using pre-recognized entity information;

[0032] The relationship trigger word identification includes identifying trigger words representing entity relationships based on syntactic analysis results and predefined rules;

[0033] The relationship definition includes defining the boundaries of entity relationships and extracting entity relationships based on the trigger words and the locations of entity mentions;

[0034] The triple generation includes combining entity pairs and relationship descriptions into triples , representing an entity and entities There is a relationship between .

[0035] As a preferred solution of the method for constructing a knowledge graph based on unsupervised learning described in the present invention, a formula for calculating entity relationship weights is constructed to calculate the relationship weights between entities:

[0036] in, and Represents an entity; is the release date decay index of text data; The date on which the text data is published; is the current date; is a parameter; For text data Source index; and Entity and entities In the article The word frequency in For text data The maximum word frequency of the entity in .

[0037] As a preferred solution of the method for constructing a knowledge graph based on unsupervised learning described in the present invention, wherein: for the extracted relationship , mapping entity relationship weights to entity and entity relationship confidences: in, It represents the confidence of entities and entity relationships, indicating the degree of trust in the existence of entities and entity relationships.

[0038] As a preferred solution of the method for constructing a knowledge graph based on unsupervised learning described in the present invention, the confidence threshold is set , filter out entities and entity relationships whose confidence does not meet the requirements:

[0039] like , then the entity is considered and entities The relationship between them does not exist;

[0040] like , then the entity is considered and entities There is a relationship between them.

[0041] As a preferred solution of the method for constructing a knowledge graph based on unsupervised learning described in the present invention, the following steps are used: constructing a knowledge graph of text data based on entities and entity relationships extracted from text data; defining an undirected weighted knowledge graph ,in Represents a collection of entity nodes, represents the entity relationship edge set, Represents the entity relationship weight set;

[0042] For each triplet that meets the confidence requirement , perform the following steps:

[0043] Step a: The entity and entities Add to Node Collection middle: ;

[0044] Step b: Entity relationships As an undirected edge Add to edge set middle:

[0045] ;

[0046] Step c: Assign entity relationship confidence As an edge The weight is added to the weight set middle: ;

[0047] Step d: If the entity and entities If there are multiple entity relationships, the relationship with the highest confidence is selected. As an entity and entities The edges between: ;

[0048] Repeat steps a to d until all triples that meet the confidence requirements have been processed.

[0049] As a preferred solution of the method for constructing a knowledge graph based on unsupervised learning according to the present invention, after processing all triples, the entity nodes are obtained. , entity relationship edge and entity relationship weights Undirected weighted knowledge graph composed of , where the node Indicates the Entity; Edge Representing an entity and entities There is a semantic relationship between them; weight Represents an edge The corresponding entity relationship confidence reflects the credibility of the entity relationship.

[0050] A system for constructing a knowledge graph based on unsupervised learning, which is used to implement a method for constructing a knowledge graph based on unsupervised learning, including: a data acquisition module, an entity recognition module, an entity relationship extraction module, an entity relationship weight calculation module, and a knowledge graph module;

[0051] The data acquisition module is used to acquire text data, pre-process the collected text data, and segment the text data to obtain segmentation results of the text data;

[0052] The entity recognition module performs entity recognition on the word segmentation results of the text data based on the pre-trained language model, and eliminates ambiguity on the entity recognition results in the text data;

[0053] The entity relationship extraction module uses an unsupervised model to extract entity relationships and generate entity and entity relationship triples;

[0054] The entity relationship weight calculation module constructs an entity relationship weight calculation formula, maps the entity relationship weight to the entity relationship confidence, and filters entity relationships with confidence values ​​lower than a threshold;

[0055] The knowledge graph module constructs a knowledge graph of text data based on entities and entity relationships extracted from text data.

[0056] Beneficial effects of the present invention: The present invention improves data quality through preprocessing and word segmentation, provides structured word sequence input for subsequent steps, and improves processing efficiency and accuracy.

[0057] The present invention utilizes a pre-trained language model for entity recognition and ambiguity elimination, fully utilizing the semantic knowledge learned by the model to improve the accuracy and consistency of entity recognition.

[0058] The present invention adopts an unsupervised model to automatically extract entity relationships, without the need to manually define complex rules, reducing labor costs and extracting rich and comprehensive entity relationship knowledge.

[0059] The present invention constructs a weight calculation formula and filters low-confidence relationships, calculates weights based on multiple factors, and improves the credibility of entity relationships in the knowledge graph.

[0060] The present invention constructs a knowledge graph to realize the conversion from unstructured text to structured knowledge, provides high-level semantic representation for knowledge-driven tasks, and improves task performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0062] Figure 1 Provide a flow chart of the method for constructing an unsupervised learning knowledge graph for the present invention;

[0063] Figure 2 This is a structural diagram of the unsupervised learning knowledge graph construction system provided by the present invention. DETAILED DESCRIPTION

[0064] To make the above-mentioned objects, features, and advantages of the present invention more easily understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0065] Example 1

[0066] See Figure 1 ,The first embodiment of the present invention provides a method for constructing a ,knowledge graph based on unsupervised learning.

[0067] S1: Acquire text data, preprocess the collected text data, and segment the text data to obtain the segmentation results of the text data.

[0068] Acquire text data, preprocess the text data, remove label data and symbol data in the text data; based on the dictionary and word segmentation tools, divide the text in the text data into word sequences.

[0069] Based on predefined dictionaries and rules, the forward maximum matching algorithm is used to segment text: Indicated by A sequence of words: .

[0070] Definition Dictionary , which contains Words: Scan text from left to right , find the longest word match.

[0071] set up is the current scanning position, For the end position of the word , judge the substring Is it in the dictionary middle.

[0072] If the substring In the dictionary, As a word, and Move to Position; if the substring If it is not in the dictionary, Move to Repeat the scanning process until , get the word segmentation results of text data .

[0073] This step removes noise data by preprocessing and segmenting text data, converting unstructured text into structured word sequence representation, providing high-quality input data for subsequent entity recognition and relationship extraction steps, and improving the availability and processing efficiency of text data.

[0074] S2: Based on the pre-trained language model, perform entity recognition on the word segmentation results of the text data and eliminate ambiguity on the entity recognition results in the text data.

[0075] The RoBERTa pre-trained model is selected as the entity recognition model. Based on the RoBERTa pre-trained model, the RoBERTa pre-trained model is adjusted to add entity recognition specific network layers, including linear layers and CRF layers.

[0076] The word segmentation results are input into the adjusted model to obtain the label probability distribution of each word segmentation, and the Viterbi algorithm is used to decode it to obtain the optimal label sequence; the entities and types are extracted to obtain the entity recognition results.

[0077] For each entity recognition result, retrieve candidate entities with the same name from the knowledge base of the RoBERTa pre-trained model and calculate the entity With candidate entities Similarity ; Select the candidate entity with the highest similarity As a link object: in, It is a collection of entities in the knowledge base. Link to selected candidate entity , to eliminate ambiguity.

[0078] This step uses the pre-trained RoBERTa language model for entity recognition, making full use of the rich semantic knowledge learned by the model on large-scale corpus to improve the accuracy of entity recognition.

[0079] At the same time, through entity ambiguity elimination, the linking problem of entities with the same name is solved, and the accuracy and consistency of entity recognition results are improved.

[0080] S3: Use an unsupervised model to extract entity relationships and generate entity and entity relationship triplets.

[0081] The OpenIE model is selected as the unsupervised entity relationship extraction model; OpenIE is a model that can automatically discover entity relationship patterns from text without manually defining relationship templates or annotating training data.

[0082] The preprocessed text data and entity recognition results are input into the OpenIE model. The OpenIE model extracts entity relationships through the steps of syntactic analysis, entity recognition, relationship trigger word recognition, relationship definition, and triple generation.

[0083] The syntactic analysis includes performing syntactic analysis on the input text using a dependency syntactic analyzer to identify the subject, predicate and object components of the sentence.

[0084] The entity recognition includes locating entities in a sentence using pre-recognized entity information.

[0085] The relationship trigger word identification includes identifying trigger words representing entity relationships based on syntactic analysis results and predefined rules.

[0086] The relationship definition includes defining the boundary of the entity relationship and extracting the entity relationship based on the trigger word and the location of the entity mention.

[0087] The triple generation includes combining entity pairs and relationship descriptions into triples , representing an entity and entities There is a relationship between .

[0088] For each identified entity pair , the OpenIE model may extract multiple relationship descriptions , generate a set of triples output by the OpenIE model Representing an entity and entities There are multiple semantic relationships between them.

[0089] This step uses the OpenIE unsupervised model to extract entity relationships, which eliminates the need to manually define complex relationship templates and label large amounts of training data, reducing labor costs. At the same time, this method can automatically discover multiple entity relationship patterns contained in the text and extract rich and comprehensive entity relationship knowledge.

[0090] S4: Construct an entity relationship weight calculation formula, map the entity relationship weight to the entity relationship confidence, and filter out entity relationships with confidence values ​​lower than the threshold.

[0091] Construct the entity relationship weight calculation formula to calculate the relationship weight between entities:

[0092] in, and Represents an entity; is the release date decay index of text data; The date on which the text data is published; is the current date; is a parameter; For text data Source index; and Entity and entities In the article The word frequency in For text data The maximum word frequency of the entity in .

[0093] For the extracted relationship , mapping entity relationship weights to entity and entity relationship confidences: in, It represents the confidence of entities and entity relationships, indicating the degree of trust in the existence of entities and entity relationships.

[0094] Setting confidence thresholds , filter out entities and entity relationships whose confidence does not meet the requirements:

[0095] like , then the entity is considered and entities The relationship between them does not exist;

[0096] like , then the entity is considered and entities There is a relationship between them.

[0097] This step constructs an entity relationship weight calculation formula, comprehensively considering factors such as entity co-occurrence frequency, text time decay, and data source credibility, to obtain more accurate and comprehensive entity relationship weights; at the same time, the weights are mapped to confidence and low-confidence relationships are filtered out, thereby improving the credibility of entity relationships in the knowledge graph.

[0098] S5: Build a knowledge graph of text data based on the entities and entity relationships extracted from the text data.

[0099] Construct a knowledge graph of text data based on entities and entity relationships extracted from text data; define an undirected weighted knowledge graph ,in Represents a collection of entity nodes, represents the entity relationship edge set, Represents a set of entity relationship weights.

[0100] For each triplet that meets the confidence requirement , perform the following steps:

[0101] Step a: The entity and entities Add to Node Collection middle: ;

[0102] Step b: Entity relationships As an undirected edge Add to edge set middle: ;

[0103] Step c: Assign entity relationship confidence As an edge The weight is added to the weight set middle: ;

[0104] Step d: If the entity and entities If there are multiple entity relationships, the relationship with the highest confidence is selected. As an entity and entities The edges between: ;

[0105] Repeat steps a to d until all triples that meet the confidence requirements have been processed.

[0106] After processing all triples, we get the entity nodes , entity relationship edge and entity relationship weights Undirected weighted knowledge graph composed of , where the node Indicates the Entity; Edge Representing an entity and entities There is a semantic relationship between them; weight Represents an edge The corresponding entity relationship confidence reflects the credibility of the entity relationship.

[0107] Furthermore, in the process of entity recognition and relationship extraction, the diversity of entity relationships and information redundancy are sometimes contradictory. To address this, our researchers have designed a creative algorithm and a balancing factor. By combining this creative algorithm with the introduction of the balancing factor, the credibility or confidence threshold affecting the source of text data is calculated using the balancing factor. Specific supplementary steps S5'-S9:

[0108] S6: Use adjacency matrix to represent knowledge graph and build a dynamic update mechanism for knowledge graph.

[0109] Knowledge Graph Represented as an adjacency matrix In the form of: ;

[0110] in, Represents the number of entity nodes in the knowledge graph, Representing an entity and entities The relationship weight between , if the entity and entities If there is no relationship between .

[0111] Build a dynamic update mechanism for the knowledge graph and update it based on newly added text data:

[0112] Step 1: Preprocess, segment, identify entities, and extract relationships on the newly added text data to obtain a new entity set , relationship set and weight set ;

[0113] Step 2: Update the node set of the knowledge graph ;

[0114] Step 3: Update the edge set of the knowledge graph ;

[0115] Step 4: Update the weight set of the knowledge graph :for Each edge in ,if , then update the edge Weight: ;in, Represents the new weight value, is a smoothing coefficient used to balance the ratio of historical weights and new weights.

[0116] if , then the edge and weights Add to and middle: ;

[0117] Step 5: Update the adjacency matrix of the knowledge graph : in, Indicates the number of newly added entity nodes, Represents the updated entity relationship weight.

[0118] By constructing a knowledge graph representation of text data, we achieve the transformation from unstructured text to structured knowledge. This knowledge graph can serve as a high-level semantic representation of text data and be applied to various knowledge-driven tasks, such as information retrieval, question-answering systems, and recommendation systems. It fully mines and utilizes the rich knowledge contained in text data, improving the performance and intelligence of related tasks.

[0119] Example 2

[0120] See Figure 2 ,The second embodiment of the present invention provides a system for building ,knowledge graph based on unsupervised learning.

[0121] The module includes: data acquisition module, entity recognition module, entity relationship extraction module, entity relationship weight calculation module and knowledge graph module.

[0122] The data acquisition module is used to acquire text data, pre-process the collected text data, and segment the text data to obtain segmentation results of the text data.

[0123] The entity recognition module performs entity recognition on the word segmentation results of the text data based on the pre-trained language model, and eliminates ambiguity on the entity recognition results in the text data.

[0124] The entity relationship extraction module uses an unsupervised model to extract entity relationships and generate entity and entity relationship triplets.

[0125] The entity relationship weight calculation module constructs an entity relationship weight calculation formula, maps the entity relationship weight to the entity relationship confidence, and filters entity relationships with confidence values ​​lower than a threshold.

[0126] The knowledge graph module constructs a knowledge graph of text data based on entities and entity relationships extracted from text data.

[0127] Example 3

[0128] The third embodiment of the present invention is different from the previous embodiment in that:

[0129] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0130] Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0131] The logic and / or steps represented in the flowchart or otherwise described herein may be considered, for example, as a sequenced list of executable instructions for implementing the logical functions, and may be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device).

[0132] For the purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or device.

[0133] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection having one or more wires (electronic device), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM).

[0134] Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0135] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware or a combination thereof.

[0136] In the above embodiments, multiple steps or methods may be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.

[0137] For example, if implemented in hardware, as in another embodiment, it can be implemented using any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing logic functions on data signals, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0138] Additionally, in order to provide a concise description of exemplary embodiments, all features of an actual embodiment may not be described (ie, those features that are not relevant to the best mode presently contemplated for carrying out the invention or those that are not relevant to implementing the invention).

[0139] It will be appreciated that in the development of any actual embodiment, as in any engineering or design project, numerous implementation-specific decisions may be made. Such a development effort may be complex and time-consuming, but will, for those of ordinary skill having the benefit of this disclosure, be a routine undertaking of design, fabrication, and production without undue experimentation.

[0140] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for constructing a knowledge graph based on unsupervised learning, characterized in that: include: Acquire text data, preprocess the collected text data, and segment the text data to obtain segmentation results of the text data; Based on the pre-trained language model, entity recognition is performed on the word segmentation results of the text data, and the entity recognition results in the text data are disambiguated; Use unsupervised models to extract entity relationships and generate entity and entity relationship triplets; Construct an entity relationship weight calculation formula, map the entity relationship weight to the entity relationship confidence, and filter out entity relationships with confidence values ​​below the threshold; Build a knowledge graph of text data based on the entities and entity relationships extracted from the text data; Select the RoBERTa pre-trained model as the entity recognition model. Based on the RoBERTa pre-trained model, adjust the RoBERTa pre-trained model and add entity recognition specific network layers, including linear layers and CRF layers. The word segmentation results are input into the adjusted model to obtain the label probability distribution of each word segmentation, and the Viterbi algorithm is used to decode and obtain the optimal label sequence; the entities and types are extracted to obtain the entity recognition results; For each entity recognition result, retrieve candidate entities with the same name from the knowledge base of the RoBERTa pre-trained model and calculate the entity With candidate entities Similarity ; Select the candidate entity with the highest similarity As a link object: ； in, It is a collection of entities in the knowledge base. Link to selected candidate entity , eliminate ambiguity; The OpenIE model is selected as the unsupervised entity relationship extraction model, including; The preprocessed text data and entity recognition results are input into the OpenIE model, which extracts entity relationships through the steps of syntactic analysis, entity recognition, relationship trigger word recognition, relationship definition, and triple generation. The syntactic analysis includes performing syntactic analysis on the input text using a dependency parser to identify the subject, predicate and object components of the sentence; The entity recognition includes locating entities in a sentence using pre-recognized entity information; The relationship trigger word identification includes identifying trigger words representing entity relationships based on syntactic analysis results and predefined rules; The relationship definition includes defining the boundaries of entity relationships and extracting entity relationships based on the trigger words and the locations of entity mentions; The triple generation includes combining entity pairs and relationship descriptions into triples , representing an entity and entities There is a relationship between ; Construct the entity relationship weight calculation formula to calculate the relationship weight between entities: ; in, and Represents an entity; is the release date decay index of text data; The date on which the text data is published; is the current date; is a parameter; is the source index of text data k; and Entity and entities The frequency of words in article k, is the maximum word frequency of entity in text data k.

2. The method for constructing a knowledge graph based on unsupervised learning according to claim 1, characterized in that: Obtain text data, preprocess the text data, remove label data and symbol data in the text data; divide the text in the text data into word sequences based on dictionaries and word segmentation tools; Based on predefined dictionaries and rules, the forward maximum matching algorithm is used to segment text: Indicated by A sequence of words: ; Definition Dictionary , which contains Words: Scan text from left to right , find the longest word match; set up is the current scanning position, For the end position of the word , judge the substring Is it in the dictionary middle; If the substring In the dictionary, As a word, and Move to Position; if the substring If it is not in the dictionary, Move to Repeat the scanning process until , get the word segmentation results of text data .

3. The method for constructing a knowledge graph based on unsupervised learning according to claim 2, characterized in that: For the extracted relationship , mapping entity relationship weights to entity and entity relationship confidences: ； in, It represents the confidence of entities and entity relationships, indicating the degree of trust in the existence of entities and entity relationships.

4. The method for constructing a knowledge graph based on unsupervised learning according to claim 3, characterized in that: Setting confidence thresholds , filter out entities and entity relationships whose confidence does not meet the requirements: like , then the entity is considered and entities The relationship between them does not exist; like , then the entity is considered and entities There is a relationship between them.

5. The method for constructing a knowledge graph based on unsupervised learning according to claim 4, characterized in that: Construct a knowledge graph of text data based on entities and entity relationships extracted from text data; define an undirected weighted knowledge graph ,in Represents a collection of entity nodes, represents the entity relationship edge set, Represents the entity relationship weight set; For each triplet that meets the confidence requirement , perform the following steps: Step a: The entity and entities Add to Node Collection middle: ； Step b: Entity relationships As an undirected edge Add to edge set middle: ； Step c: Assign entity relationship confidence As an edge The weight is added to the weight set middle: ； Step d: If the entity and entities If there are multiple entity relationships, the relationship with the highest confidence is selected. As an entity and entities The edges between: ； ； ； Repeat steps a to d until all triples that meet the confidence requirements have been processed.

6. The method for constructing a knowledge graph based on unsupervised learning according to claim 5, characterized in that: After processing all triples, we get the entity nodes , entity relationship edge and entity relationship weights Undirected weighted knowledge graph composed of , where the node Indicates the Entity; Edge Representing an entity and entities There is a semantic relationship between them; weight Represents an edge The corresponding entity relationship confidence reflects the credibility of the entity relationship.

Images