A named entity recognition method fused with knowledge graph embedding

Abstract

This invention relates to the field of natural language processing technology and discloses a named entity recognition method that integrates knowledge graph embedding. This method uses a pre-trained language model BERT to perform contextual semantic encoding on text and fuses the obtained word vectors with the vectorized representations of entities in a knowledge graph. Addressing the problem of a large amount of redundant information in text that is irrelevant to the entity recognition task, this invention introduces a selective attention mechanism guided by the knowledge graph structure. By using semantic query vectors constructed from triples to guide the model to focus on contextual regions highly relevant to the target structure, it effectively suppresses noise interference, enhances semantic understanding and entity boundary perception capabilities, and finally constructs a cascaded recognition structure, modeling the start and end positions of entities respectively to achieve accurate recognition of nested and overlapping entities. This method integrates contextual representation and structured knowledge, has strong guidance, high recognition accuracy, and good robustness and generalization ability.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a named entity recognition method that integrates knowledge graph embedding. Background Technology

[0002] Knowledge graphs (KGs) play a crucial role in effectively organizing and managing structured information across various applications. Knowledge is presented as a set of linked "facts." These facts are represented as triples (subject, verb, object), where the subject and object are entities, and the verb represents the relationship between them. Named entity recognition, a fundamental task in natural language processing, aims to identify entities with specific semantic categories from unstructured text and has wide applications in fields such as healthcare, finance, and information extraction.

[0003] Text contains a wealth of complex information, and the various fine-grained semantic relationships and boundaries between words make it very difficult for existing language models to understand the grammatical structure of sentences, thus failing to provide a clear semantic understanding. For example, important information is often embedded in complex phrase structures or sentences with cross-phrase modification forms, and its semantic boundaries are unclear, making it difficult to infer entity categories from context alone. This information usually relies on prior knowledge for accurate understanding, such as entity type, hierarchical affiliation, and subordinate relationships.

[0004] Traditional models, such as those using bidirectional Long Short-Term Memory (LSTM) networks combined with Conditional Random Fields (CRFs), consider handling long-distance dependencies. LSTMs model contextual semantic relationships, while CRFs can consider dependency constraints between preceding and following labels during the prediction phase. However, with the increasing complexity of text structure and the diversity of semantic expressions, these models struggle to capture deep semantic information within sentences.

[0005] However, without the support of external knowledge, existing language models often struggle to recognize entities with complex semantic layers and ambiguous structures, leading to problems such as boundary division errors and category judgment biases in named entity recognition, which in turn affects the performance and accuracy of downstream tasks. Summary of the Invention

[0006] (a) Technical problems to be solved

[0007] To address the shortcomings of existing technologies, this invention provides a named entity recognition method that integrates knowledge graph embedding, which has advantages such as enhanced perception of entity boundaries in complex contexts, and solves the aforementioned technical problems.

[0008] (II) Technical Solution

[0009] To achieve the above objectives, the present invention provides the following technical solution: a named entity recognition method integrating knowledge graph embedding, comprising the following steps:

[0010] S1: Collect the original text and perform desensitization and deduplication processing to form a humanities text corpus;

[0011] S2: Select an external knowledge graph to align with the humanities text corpus, map the identified entities to unique entity IDs, and establish a mapping table between words and knowledge graph entity IDs;

[0012] S3: Input the text from S2 into the BERT model for contextual semantic encoding;

[0013] S4: Introduce the structured entity information from the knowledge graph from step S2 to enhance the text representation and obtain the fused semantic representation vector. ;

[0014] S5: Based on the fused semantic representation vector in S4 Build a prediction model and train it to obtain a trained prediction model;

[0015] S6: Based on the prediction model trained in S5, predict the start and end positions of various entities in the text to be identified in sequence, and output the structured named entity recognition results.

[0016] As a preferred embodiment of the present invention, the specific expression for context semantic encoding using the BERT model in step S3 is as follows:

[0017]

[0018] in, express Sentence level representation, These represent the input text. One word, They represent The contextual semantic vector of each word, express Special marker vectors, This represents the BERT embedding model.

[0019] As a preferred embodiment of the present invention, step S4 includes the following steps:

[0020] S4.1: The first Context semantic vector of each word Its corresponding embedding vector The fusion is performed using the following expression:

[0021]

[0022] in, This represents a vector concatenation operation. This represents the first value obtained by encoding the BERT model. The contextual semantic vector of each word; Indicates the first The embedding vector of the corresponding entity in the external knowledge graph for each word; Indicates the first The semantically enhanced representation after word fusion is constructed, and the fused representation sequence is generated. , They represent the 1st to the 2nd. Semantic enhancement representation after word fusion;

[0023] S4.2: Based on the same external knowledge graph as in step S4.1, obtain the triples related to the entity pairs in the current input sentence from the external knowledge graph. ;

[0024] in, Indicates the head entity. Indicates a relationship. Indicates the tail entity;

[0025] The triples are mapped to a vector space using a knowledge graph embedding algorithm, resulting in embedding vectors. , , Embedded vectors are concatenated to form triplet representations :

[0026]

[0027] in, The embedding vector representing the head entity of the triplet. An embedding vector representing a triple relation. The embedding vector represents the tail entity of the triple. The vector concatenation operation performs average pooling to aggregate all triple representations of the current input sentence, resulting in a sentence-level semantic guidance vector. Specifically:

[0028]

[0029] in, This indicates average pooling polymerization;

[0030] Use semantically enhanced representations as key vectors Specifically:

[0031]

[0032] The attention weights are constructed using the following expression:

[0033]

[0034]

[0035] in, This represents an exponential function with the natural constant as its base. Represents semantically guided vectors With the Key vectors of words Through learnable projection matrix The calculated matching score, The attention weights after Softmax normalization of the score. express transpose;

[0036] Globally fused semantic representation vector based on attention weights The specific expression is as follows:

[0037]

[0038] in, Indicates the first stage after initial fusion The contextual semantics of each word and its knowledge graph embedding representation. Its corresponding attention weight; This represents the globally fused semantic representation vector. This indicates a summation.

[0039] As a preferred embodiment of the present invention, the specific steps of S5 are as follows:

[0040] S5.1: For fused semantic representation vectors The first in The fusion representation vector of each word Construct two parameter-independent FFN feedforward neural networks. If the output value is greater than a set probability threshold, output the start or end position respectively, and complete the identification of all possible main entities in the sentence. The specific expression is as follows:

[0041]

[0042]

[0043] in, and These represent the learnable weight matrices used to predict the starting and ending positions of the subject, respectively. and This is the corresponding bias vector; Indicates the first The fusion representation vector of each word; and They represent the first The probability output of each word as the main start word and end word is as follows: Represents the sigmoid function;

[0044] S5.2: Construct a new context vector representation based on each identified subject in S5.1 and determine the start and end positions again. The specific expression is as follows:

[0045]

[0046] in, This represents the global fused semantic representation vector obtained by weighted summation of the fused representations of all words in the current sentence; This represents a vector concatenation operation; Indicating the identification of the first A new contextual fusion representation constructed after each subject, Indicates the first The vector representation of each identified subject is expressed as follows:

[0047]

[0048] in, and These represent the fused representation vectors corresponding to the start and end positions of the subject in the sentence, respectively.

[0049] After obtaining the number Vector representation of each identified subject Then, the first The fusion representation vector of each word Subject conditionalization is performed to construct a new context fusion representation, as shown in the following expression:

[0050]

[0051] in, Indicates the first The first entity is composed of the first Subject-conditional contextual fusion representation constructed from individual words Indicates splicing;

[0052] Then, the start and end positions are determined again based on the FFN feedforward neural network, as shown in the following expression:

[0053]

[0054]

[0055] in, and These represent the learnable weight matrices used to predict the starting and ending positions of the object, respectively. and This is the corresponding bias vector; Indicates the first The first entity is composed of the first Subject-conditional contextualized representation constructed from individual words; and They represent the first Each word is output as the probability of the object's start and end words. This represents the sigmoid function.

[0056] As a preferred embodiment of the present invention, the loss function in the training process of S5 as follows:

[0057]

[0058] in, , These are subject recognition loss and object recognition loss, respectively. This represents the binary cross-entropy loss function. and Other means the first The probability of a word being the starting and ending word of an object; and They represent the first Each word serves as a true label for the object's starting and ending words. This indicates a summation.

[0059] Compared with existing technologies, this invention provides a named entity recognition method that integrates knowledge graph embedding, which has the following beneficial effects:

[0060] 1. This invention introduces external knowledge graphs such as CN-DBpedia to fuse entity and relation triple embedding vectors with contextual semantic representations. This allows each word representation to include not only the contextual information provided by the language model, but also its corresponding structured entity attributes and semantic relationships. This fills the gap in entity understanding caused by insufficient context in traditional semantic encoding. This fusion method provides the model with entity category attribution, semantic hierarchy and association relationships, effectively enhancing the ability to perceive entity boundaries in complex contexts.

[0061] 2. This invention introduces semantic guidance vectors constructed from graph triples as query terms for a selective attention mechanism. By using learnable projections to filter key information regions consistent with the graph structure in a unified semantic space, it suppresses redundant contextual interference, thereby improving the model's ability to focus on key entity words and the quality of representation in task-specific contexts.

[0062] 3. This invention constructs a cascaded named entity recognition structure, which breaks through the limitations of traditional sequence labeling models by independently predicting the start and end positions of entities. It effectively identifies overlapping entities and nested structures in text, achieves accurate modeling of complex entity boundaries, and improves recognition robustness and generalization ability. Attached Figure Description

[0063] Figure 1 This is the overall flowchart of the invention.

[0064] Figure 2 This is a structural diagram of the cascaded model of the present invention. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] Please see Figures 1-2 A named entity recognition method integrating knowledge graph embedding includes the following steps:

[0067] S1. Data preprocessing and corpus construction;

[0068] In this embodiment, the dataset used is the DUIE Chinese Relation Entity Extraction Dataset, which was released by the Baidu Natural Language Processing Team and is widely used in named entity recognition and relation extraction tasks. The DUIE dataset comes from publicly available encyclopedic texts, covering multiple categories such as people, works, organizations, and geography, and is labeled with rich entity types and their corresponding relationships.

[0069] In this embodiment, approximately 1000 sample texts were selected, with 800 used as the training set and 200 as the test set. The entity types labeled included: Person, Work, Organization, Location, Profession, and Achievement. In the data preprocessing stage, the original texts were anonymized and deduplicated to form a corpus of literary texts by famous figures.

[0070] S2: Knowledge Graph Alignment and Entity Labeling;

[0071] CN-DBpedia was selected as the external knowledge graph. Built upon Chinese Wikipedia, it covers various entity types including people, organizations, place names, and works. Named entity words potentially involved in the corpus were extracted to construct a candidate entity vocabulary. This vocabulary was then loaded into the user dictionary of the Jieba word segmentation tool to ensure that entities are recognized in their complete form during segmentation, avoiding incorrect segmentation. Subsequently, the text was segmented using the Jieba tool, and accurate segmentation was achieved by combining the user dictionary with the selected words. Based on this, the identified candidate entity set is... With the standard entity set in the knowledge graph Perform fuzzy matching and linking to establish a mapping relationship between words and graph entities: It represents the mapping relationship between entity words and knowledge graph entities, assigns a unique entity ID to successfully aligned entities, and establishes a mapping table between words and knowledge graph entity IDs;

[0072] In the named entity annotation process, the SPO triple structure annotation format is adopted, which uses (subject, predicate, object) triples to represent entities and their relationships in a structured way for subsequent training and evaluation of entity recognition models.

[0073] S3, Context semantic representation encoding;

[0074] After the text that has completed entity alignment and annotation in step S2 is segmented, the resulting word sequence is added with special tags and then input into the BERT model for contextual semantic encoding.

[0075]

[0076] in, express Sentence level representation, express The first text entered in the input text One word, These represent the input text. One word, They represent The contextual semantic vector of each word, express The Middle The context semantic vector of each word, and , express Special marker vectors, This represents the BERT embedding model; simultaneously, it loads the entity IDs obtained in step S2, constructs an index mapping between words and entity vectors, and prepares for subsequent knowledge fusion.

[0077] While the BERT model can effectively provide contextual semantic representations of text, relying solely on BERT's contextual encoding is insufficient to cover all key information. This is especially true when dealing with complex phrase structures or sentences involving cross-phrase modifications, where important information is often embedded in structures with unclear semantic boundaries, making it impossible to infer entity categories from context alone. For example, in multi-layered or ambiguous sentences, the contextual information obtained by BERT may be insufficient to clearly distinguish the type of an entity and its contextual relationship, affecting the model's recognition accuracy. Therefore, this semantic information often relies on external prior knowledge for further understanding, such as entity type, hierarchical affiliation, and subordinate relationships.

[0078] To overcome this limitation, in the subsequent S4 step, external knowledge graph information is introduced and combined with embedded vectors such as entity type, hierarchical structure, and relation affiliation to construct a fused semantic representation; the semantic expressiveness of word representation is enhanced by structured prior knowledge, further improving the model's ability to understand complex entity structures and enhancing the accuracy and robustness of named entity recognition.

[0079] S4. Integrating external knowledge graph representations;

[0080] Based on the contextual semantic vector of each word, structured entity information from an external knowledge graph is further introduced to enhance the text representation. This includes the following steps:

[0081] S4.1: Based on obtaining the contextual semantic vector of each word, to further introduce structured entity information from an external knowledge graph, the TransE knowledge graph embedding algorithm is applied using triple information from the knowledge graph to obtain entities. and relationships The vectorized representation of ; concatenating the context vector of each word with its corresponding entity vector to form a preliminary fusion; let the . Context semantic vector of each word Its corresponding embedding vector After initial fusion, the semantic enhancement is represented as follows:

[0082]

[0083] in This represents a vector concatenation operation. This represents the first value obtained by encoding the BERT model. The contextual semantic vector of each word; This represents the embedding vector of the corresponding entity in an external knowledge graph (such as CN-DBpedia); This represents the fused semantically enhanced representation, used to construct the subsequent fused representation sequence. If a word is not aligned with an entity, it is padded with a zero vector of the same dimension to ensure that the overall input dimension is consistent. Finally, the fused representation sequence is:

[0084]

[0085] This sequence takes into account both linguistic context and structured knowledge, enhancing the model's ability to distinguish ambiguous entities. They represent the 1st to the 2nd. Semantic enhancement representation after word fusion;

[0086] S4.2: Although the initial fusion of context vectors and entity embeddings was completed in step S4.1, there is still a lot of redundant information in the text instances that is not related to the named entity recognition target. This irrelevant content may introduce noise in subsequent modeling and interfere with the model’s judgment of key entities. In particular, in nested entities or ambiguous contexts, it is easier to affect the accurate recognition of entity boundaries.

[0087] To further filter out task-relevant contextual information from the initially fused representation, this invention proposes a selective attention mechanism guided by knowledge graph structural information. The aim is to further filter out contextual word vectors highly relevant to the target task from the fused representation. Using the embedding representation of triples from the knowledge graph in step S4.1 as a guide to construct a query vector, and using the initially fused contextual representation as a key, the model effectively filters out keywords relevant to the target task by calculating the similarity between the contextual word vectors and the embedded relation vectors. This further strengthens the expression of important information, thereby achieving more efficient semantic understanding and entity relation extraction in task-specific contexts.

[0088] Specifically, considering that for each aligned entity pair Naturally carrying structured semantic information, the concatenated vector representation can serve as a semantic query target, guiding the model to find matching expression patterns in the text. This method constructs a semantic guidance vector (Query) by concatenating the corresponding embedding vectors in the triples. The specific expression is as follows:

[0089] Let each triple in the knowledge graph be denoted as:

[0090]

[0091] in, Represents the header entity; Representing a relationship. Represents the tail entity, the first Related triples.

[0092] Knowledge graph embedding algorithms (such as TransE) are used to map triples into a vector space, resulting in embedding vectors. , , Embedded vectors are concatenated to form triplet representations :

[0093]

[0094] in, The embedding vector representing the head entity of the triplet. An embedding vector representing a triple relation. The embedding vector represents the tail entity of the triple. The vector concatenation operation performs average pooling to aggregate all triple representations of the current input sentence, resulting in a sentence-level semantic guidance vector. Specifically:

[0095]

[0096] in, The average pooling aggregation is used, with the initially fused text sequence representation Z serving as the key in the attention mechanism. A learnable transformation matrix W is introduced to map the guidance vector (Query) constructed from the knowledge graph to the fused text representation (Key) into a unified semantic space. By calculating the similarity between the knowledge graph structural information Query and each word representation, different attention weights are assigned to different words in the context. The model can automatically identify and strengthen important word expressions related to graph relationships and suppress interference information irrelevant to the target task. The context semantic vector is used as the key vector Key, specifically:

[0097]

[0098]

[0099] To construct attention weights, normalize the scores of all tokens using a Softmax operation to obtain the attention weights. The specific expression is as follows:

[0100]

[0101] Here, exp represents an exponential function with the natural constant as the base.

[0102]

[0103] Where A is the learnable projection matrix. This represents an exponential function with the natural constant as its base. Represents semantically guided vectors With the Key vectors of words Through learnable projection matrix The calculated matching score, The attention weights after Softmax normalization of the score. express The transpose of the vector is then applied, and attention weights are applied to the initial fused vector sequence Z to enhance key content related to the embedded relation vector:

[0104]

[0105] in Indicates the first stage after initial fusion The contextual semantics of each word and its knowledge graph embedding representation. Its corresponding attention weight; This represents the global fused semantic representation vector obtained by weighted summation of the representations at each position in the entire sequence.

[0106] By calculating the similarity between context word vectors and embedded relation vectors, the model can effectively filter out keywords related to the target task, further strengthening the expression of important information, thereby achieving more efficient semantic understanding and entity relation extraction in task-specific contexts.

[0107] Unlike traditional self-attention, the attention mechanism in this invention is driven by structured knowledge in a knowledge graph, which is used as a query vector to guide the model to identify key information regions in the text that are semantically consistent with the structure. This mechanism has stronger semantic selectivity and prior guidance. In named entity recognition tasks, especially when facing texts with complex sentence structures and implicit entity ambiguity, this mechanism can significantly enhance the model's ability to focus on key entity words, thereby improving the accuracy and robustness of entity recognition.

[0108] S5, Model Training (Cascaded Recognition Structure);

[0109] Traditional named entity recognition methods often employ sequence labeling frameworks, such as BIO / BILOU. Because each token can only be assigned a fixed entity label, it is difficult to recognize multiple entities simultaneously. There are significant modeling bottlenecks in the problems of nested entities and entity overlap. To solve the above problems, this invention adopts a cascaded recognition structure based on entity boundary modeling. The entity recognition task is decomposed into a prediction method for candidate entity intervals based on the spatial combination of the start and end positions, thereby breaking through the single label limitation.

[0110] Model structure as follows Figure 2 The diagram shows a cascaded two-stage structure. The first stage of the model's task is to identify all possible subject entities in the sentence. The input at the bottom of the diagram is a preprocessed word sequence example: "Ma Yun founded Alibaba, graduated from Hangzhou Normal University, is a Chinese entrepreneur." This example uses the fused semantic representation vector V obtained after steps S1-S4 as input. For each word... To accommodate the varying requirements for determining start and end positions in named entity recognition tasks, this invention designs structurally symmetrical but parameter-independent FFN feedforward neural networks for Start and End, respectively, to determine whether each token represents the start (Start) or end (End) position of the main entity:

[0111]

[0112]

[0113] in, and These represent the learnable weight matrices used to predict the starting and ending positions of the subject, respectively. and This is the corresponding bias vector; Indicates the first The fusion representation vector of each word; and They represent the first The probability output of each word as the main start word and end word is as follows: Let represent the sigmoid function, where each output vector has a dimension of n. The output at each position indicates whether that position is the start or end position of a certain entity. By applying a set threshold (such as 0.5) to the output probabilities of all positions, all start and end position combinations (i, j) that meet the conditions can be obtained, thus forming the candidate interval of potential entities.

[0114] In this way, all possible entity ranges are predicted; for example, the colored boxes in the figure indicate that "Alibaba" and "Hangzhou Normal University" are identified as possible subject entities. The start position prediction output is shown in the left matrix "Subject-Start", and the end position is shown in "Subject-End".

[0115] The second stage, after obtaining all possible subjects in the subject identification stage, first constructs a context-aware representation for each identified subject. Given a subject, it is represented as the average of the vectors corresponding to its starting and ending positions:

[0116]

[0117] Indicates the first The vector representation of the identified subject captures the subject's global semantics within the context; then, the object associated with that subject is predicted and used as a condition to guide the model in recognizing object entities semantically related to that subject. The global fusion semantic representation vector V is then used in the context vector representation under the conditional subject.

[0118]

[0119] in, This represents the global fused semantic representation vector obtained by weighted summation of the fused representations of all words in the current sentence; This represents a vector concatenation operation; Indicating the identification of the first The new context fusion representation constructed after the first subject is obtained Vector representation of each identified subject Then, the first The fusion representation vector of each word Subject conditionalization is performed to construct a new context fusion representation, as shown in the following expression:

[0120]

[0121] in, Indicates the first The first entity is composed of the first Subject-conditional contextual fusion representation constructed from individual words This indicates concatenation; subsequently, the model, analogous to the predictor, constructs two identical feedforward neural network (FFN) modules, used to determine whether each token is the start or end position of the object entity:

[0122]

[0123] in, and These represent the learnable weight matrices used to predict the starting and ending positions of the object, respectively. and This is the corresponding bias vector; Indicates the first The first entity is composed of the first Subject-conditional contextualized representation constructed from individual words; and They represent the first Each word is output as the probability of the object's start and end words. Represents the sigmoid function;

[0124] Through the above process, for each subject, we can predict multiple object entities that it may be associated with. During the training phase, the model uses a joint loss function to optimize both subject identification and object prediction. Specifically, for both phases, cross-entropy loss is used to calculate the loss function for the start and end positions position by position. :

[0125]

[0126] in, , These are subject recognition loss and object recognition loss, respectively. This represents the binary cross-entropy loss function. and Other means the first The probability of a word being the starting and ending word of an object; and They represent the first Each word serves as a true label for the object's starting and ending words. To express summation;

[0127] During the model evaluation phase, to comprehensively measure the performance of named entity recognition, the following three standard metrics are used: Precision (P), Recall (R), and F1 Score. Their calculation formulas are shown below:

[0128]

[0129]

[0130]

[0131] TP represents the number of correctly predicted entities; FP represents the number of incorrectly predicted entities; FN represents the number of unidentified entities missed. The above evaluation metrics are used to quantify the overall performance of the model in entity recognition boundary detection and classification. The F1 score, as a comprehensive performance measure, reflects the model's ability to balance accuracy and recall.

[0132] S6, Entity Recognition Reasoning;

[0133] The text to be recognized is processed in the same way and then input into the trained model to predict the start and end positions of various entities in the text in turn, thus obtaining the structured named entity recognition result.

[0134] Comparative experiment:

[0135] To verify the effectiveness of the method proposed in this embodiment, three sets of comparative experiments were set up:

[0136] (1) BiLSTM-CRF: This model combines a bidirectional LSTM network with a conditional random field (CRF) to encode the context and predict entity labels. LSTM can capture the contextual dependencies of a sequence, while CRF further models the dependencies between labels, thereby improving the continuity and accuracy of entity boundary recognition.

[0137] (2) BERT-Softmax: The pre-trained language model BERT is used as the basic encoder to model the context vector of each word, and a fully connected layer and a Softmax classifier are added at the top to predict entity labels.

[0138] (3) The method described in this invention (BERT-KG-Embedding-CasRel): This method integrates entity embedding representations from external knowledge graphs, introduces structured knowledge into BERT representations, and realizes the fusion of language information and external knowledge; and adopts a cascaded binary annotation structure (CasRel) to model the start and end position pairs of entities, thereby improving the model's ability to recognize complex structural entities.

[0139] Table 1 shows the comparative experimental results on the text dataset.

[0140] Model Name P R F1 BiLSTM-CRF 61.3% 67.4% 64.2% BERT-Softmax 62.5% 63.2% 62.8% BERT-KG-Embedding-CasRel 67.1% 70.1 68.6%

[0141] As shown in Table 1, the proposed method outperforms the comparison models in all three evaluation metrics: Precision (P), Recall (R), and F1 score. Specifically, the F1 score reaches 68.6%, a 4.4% improvement over the traditional BiLSTM-CRF model and also better than methods using only BERT-Softmax. Experimental results validate the significant improvement effect of external knowledge graph embedding and cascaded annotation structures on named entity recognition tasks.

[0142] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A named entity recognition method integrating knowledge graph embedding, characterized in that: Includes the following steps: S1: Collect the original text and perform desensitization and deduplication processing to form a humanities text corpus; S2: Select an external knowledge graph to align with the humanities text corpus, map the identified entities to unique entity IDs, and establish a mapping table between words and knowledge graph entity IDs; S3: Input the text from S2 into the BERT model for contextual semantic encoding; S4: Introduce the structured entity information from the knowledge graph from step S2 to enhance the text representation and obtain the fused semantic representation vector. ; S5: Based on the fused semantic representation vector in S4 Build a prediction model and train it to obtain a trained prediction model; S6: Based on the prediction model trained in S5, predict the start and end positions of various entities in the text to be identified in sequence, and output the structured named entity recognition results.

2. The named entity recognition method integrating knowledge graph embedding according to claim 1, characterized in that: The specific expression for context semantic encoding using the BERT model in S3 is as follows: in, express Sentence level representation, These represent the input text. One word, They represent The contextual semantic vector of each word, express Special marker vectors, This represents the BERT embedding model.

3. The named entity recognition method integrating knowledge graph embedding according to claim 2, characterized in that: S4 includes the following steps: S4.1: The first Context semantic vector of each word Its corresponding embedding vector The fusion is performed using the following expression: in, This represents a vector concatenation operation. This represents the first value obtained by encoding the BERT model. The contextual semantic vector of each word; Indicates the first The embedding vector of the corresponding entity in the external knowledge graph for each word; Indicates the first The semantically enhanced representation after word fusion is constructed, and the fused representation sequence is generated. , They represent the 1st to the 2nd. Semantic enhancement representation after word fusion; S4.2: Based on the same external knowledge graph as in step S4.1, obtain the triples related to the entity pairs in the current input sentence from the external knowledge graph. ; in, Indicates the head entity. Indicates a relationship. Indicates the tail entity; The triples are mapped to a vector space using a knowledge graph embedding algorithm, resulting in embedding vectors. , , Embedded vectors are concatenated to form triplet representations : in, The embedding vector representing the head entity of the triplet. An embedding vector representing a triple relation. The embedding vector represents the tail entity of the triple. The vector concatenation operation performs average pooling to aggregate all triple representations of the current input sentence, resulting in a sentence-level semantic guidance vector. Specifically: in, This indicates average pooling polymerization; Use semantically enhanced representations as key vectors Specifically: The attention weights are constructed using the following expression: in, This represents an exponential function with the natural constant as its base. Represents semantically guided vectors With the Key vectors of words Through learnable projection matrix The calculated matching score, The attention weights after Softmax normalization of the score. express transpose; Globally fused semantic representation vector based on attention weights The specific expression is as follows: in, Indicates the first stage after initial fusion The contextual semantics of each word and its knowledge graph embedding representation Its corresponding attention weight; This represents the globally fused semantic representation vector. This indicates a summation.

4. The named entity recognition method integrating knowledge graph embedding according to claim 1, characterized in that: The specific steps of S5 are as follows: S5.1: For fused semantic representation vectors The first in The fusion representation vector of each word Construct two parameter-independent FFN feedforward neural networks. If the output value is greater than a set probability threshold, output the start or end position respectively, and complete the identification of all possible main entities in the sentence. The specific expression is as follows: in, and These represent the learnable weight matrices used to predict the starting and ending positions of the subject, respectively. and This is the corresponding bias vector; Indicates the first The fusion representation vector of each word; and They represent the first The probability output of each word as the main start word and end word is as follows: Represents the sigmoid function; S5.2: Construct a new context vector representation based on each identified subject in S5.1 and determine the start and end positions again. The specific expression is as follows: in, Represents the global fusion semantic representation vector; This represents a vector concatenation operation; Indicating the identification of the first A new contextual fusion representation constructed after each subject, Indicates the first The vector representation of each identified subject is expressed as follows: in, and These represent the fused representation vectors corresponding to the start and end positions of the subject in the sentence, respectively. After obtaining the number Vector representation of each identified subject Then, the first The fusion representation vector of each word Subject conditionalization is performed to construct a new context fusion representation, as shown in the following expression: in, Indicates the first The first entity is composed of the first Subject-conditional contextual fusion representation constructed from individual words Indicates splicing; Then, the start and end positions are determined again based on the FFN feedforward neural network, as shown in the following expression: in, and These represent the learnable weight matrices used to predict the starting and ending positions of the object, respectively. and This is the corresponding bias vector; Indicates the first The first entity is composed of the first Subject-conditional contextualized representation constructed from individual words; and They represent the first Each word is output as the probability of the object's start and end words. This represents the sigmoid function.

5. The named entity recognition method integrating knowledge graph embedding according to claim 4, characterized in that: The loss function in the training process of S5 as follows: in, , These are subject recognition loss and object recognition loss, respectively. This represents the binary cross-entropy loss function. and Other means the first The probability of a word being the starting and ending word of an object; and They represent the first Each word serves as a true label for the object's starting and ending words. This indicates a summation.

Images