Chinese named entity recognition method fused with knowledge graph embedding and related device

Abstract

The application discloses a kind of Chinese named entity recognition methods and related equipment of fusion knowledge graph embedding, the method includes: constructing knowledge graph on named entity recognition dataset, knowledge graph is used to indicate the relationship between word and entity in named entity recognition dataset;Named entity recognition task and knowledge graph embedding task are combined into multi-task training model and are jointly trained, information between named entity recognition task and knowledge graph embedding task is extracted based on information fusion module;Knowledge sharing is carried out in the embedding layer of named entity recognition task and knowledge graph embedding task, input text is input to the information fusion module, and the vector representation of each character in input text is obtained by processing input text based on information fusion module.The application can better extract word-level information by representing the corresponding relationship between word and entity in text through knowledge graph, obtain better representation effect, and improve the accuracy of named entity recognition.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a Chinese named entity recognition method, terminal, and computer-readable storage medium that integrates knowledge graph embedding. Background Technology

[0002] Named Entity Recognition (NER) is one of the fundamental tasks in Natural Language Processing (NLP). Its main task is to separate various types of entities from unstructured text. Entity types typically include Person, Time, and Location. NER can assist many downstream NLP tasks, such as information retrieval, question answering systems, text understanding, and knowledge base construction. Due to low word segmentation accuracy and the semantic diversity of Chinese, previous Chinese NER work has largely remained at the stage of character-based representation. Following the introduction of Lattice-LSTM, Chinese NER began to focus on the positive impact of word-based and other information embedding methods. Subsequent works, such as SoftLexicon and FLAT, are improvements based on Lattice-LSTM. MECT (Multi-Metadata Embedding based Cross-Transformer) enhances information representation from the perspective of Chinese character shapes and radicals, also achieving leading results.

[0003] However, the improvements used in these works are primarily aimed at better utilizing word vectors and constructing better model structures for information fusion. These word vectors are obtained through word2vec training on large datasets. In Chinese NER tasks, these word vectors struggle to represent the relationships between the entities to which words belong.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] The main objective of this invention is to provide a Chinese named entity recognition method, terminal, and computer-readable storage medium that integrates knowledge graph embedding. This invention aims to solve the problem that the word vectors used in the prior art for information fusion are difficult to represent the relationship between the entities to which the words belong, resulting in low accuracy of named entity recognition.

[0006] To achieve the above objectives, the present invention provides a Chinese named entity recognition method that integrates knowledge graph embedding. The method includes the following steps:

[0007] A knowledge graph is constructed on a named entity recognition dataset, the knowledge graph being used to represent the relationships between words and entities in the named entity recognition dataset;

[0008] The named entity recognition task and the knowledge graph embedding task are combined into a multi-task training model for joint training. The information fusion module in the multi-task training model is used to extract the information between the named entity recognition task and the knowledge graph embedding task.

[0009] Knowledge sharing is performed in the embedding layer of the named entity recognition task and the knowledge graph embedding task. The input text is input to the information fusion module, and the information fusion module processes the input text to obtain a vector representation of each character in the input text.

[0010] Optionally, the Chinese named entity recognition method integrating knowledge graph embedding, wherein constructing a knowledge graph on the named entity recognition dataset, the knowledge graph being used to represent the relationships between words and entities in the named entity recognition dataset, specifically includes:

[0011] All text in the named entity recognition dataset is segmented into words to obtain a set of words called WordSet;

[0012] Add all entity categories appearing in the named entity recognition dataset to the RelationshipSet;

[0013] Add all entities tagged in the named entity recognition dataset to the entity set EntitySet;

[0014] The knowledge graph is constructed from a set of WordSet, a set of RelationshipSet, and a set of EntitySet to complete the construction of a knowledge graph based on words and entities. The construction rules are as follows:

[0015] {w, r, e}, (w∈WordSet, r∈RelationshipSet, e∈EntitySet);

[0016] Where w refers to a single word after word segmentation; r refers to a certain entity category in the named entity recognition dataset, defining the entity category as a relation; and e refers to an entity tagged in a certain named entity recognition dataset.

[0017] Optionally, in the Chinese named entity recognition method that integrates knowledge graph embedding, the loss function of the multi-task training model is composed of the loss function of the knowledge graph embedding task and the loss function of the named entity recognition task.

[0018] The loss function of the multi-task training model is expressed as:

[0019] Loss = Loss1 + Loss2;

[0020] Wherein, Loss represents the loss function of the multi-task training model, Loss1 represents the loss function of the knowledge graph embedding task, and Loss2 represents the loss function of the named entity recognition task.

[0021] Optionally, in the Chinese named entity recognition method that integrates knowledge graph embedding, the multi-task training model uses a dual-path convolutional neural network model to perform the knowledge embedding task, and the dual paths include a first path Path1 and a second path Path2.

[0022] Among them, the first path Path1 uses one-dimensional convolution to extract explicit relationships between entities and relations, while the second path Path2 uses two-dimensional convolution to extract intrinsic and implicit information between entities and relations;

[0023] The knowledge graph triples constructed based on the named entity recognition dataset are defined as h, r, and t, where h, r, and t represent the head entity, relation, and tail entity, respectively.

[0024] Optionally, in the Chinese named entity recognition method that integrates knowledge graph embedding, for the first path Path1, the inputs h and r are concatenated to obtain the vector M. inp Using convolution kernels and convolution kernel In vector M inp We obtain v1, where c represents the size of the first dimension of the convolution kernel;

[0025]

[0026]

[0027]

[0028] Where k represents the size of the first dimension; M mid v1 represents the intermediate result obtained by performing a convolution operation between the vector concatenated with h and r and the convolution kernel; v1 represents the output result obtained by the first path of the knowledge graph embedding task, and v2 represents the result of extracting the explicit relationship between entities and relations.

[0029] Optionally, in the Chinese named entity recognition method that integrates knowledge graph embedding, for the second path Path2, h and r are transformed into entity vector v. h and relation vector v r , Where, k1*k2=k, k1 and k2 represent the dimensional size of the one-dimensional h and r after they are transformed into two-dimensional shapes;

[0030] Construct a solid convolution filter ω respectively h And a relational convolutional filter ω r For the relation vector v r Use solid convolution filter ω h For entity vector v h Use a relational convolution filter ω r ;

[0031]

[0032]

[0033] Where M1 represents the relationship between v h The output obtained after applying a convolution kernel; M2 represents the output obtained after applying a convolution kernel to v. r The output obtained after applying convolution kernels; b, p, and q represent the number, height, and width of the convolution kernels, respectively, and b*p*q=k1*k2;

[0034] M1 and M2 are concatenated, and the output v2 is calculated using a weight matrix W; where the dimension of the weight matrix W is... Used to ensure that the dimensions of v1 and v2 are consistent;

[0035]

[0036] Where v2 represents the output result obtained from the second path of the knowledge graph embedding task, and represents the extraction result of the implicit relationship between entities and relations; o represents the parameter, calculated as follows:

[0037] o=b*2*(k1-p+1)*(k2-q+1).

[0038] Optionally, in the Chinese named entity recognition method that integrates knowledge graph embedding, the output result v1 obtained from the first path Path1 and the output result v2 obtained from the second path Path2 are added together to obtain the final output v, and the current score is calculated:

[0039] v = v1 + v2;

[0040] Score = vv T ;

[0041] Here, Score represents the score, and the model parameters for the current task are optimized based on the score.

[0042] Optionally, in the Chinese named entity recognition method that integrates knowledge graph embedding, knowledge sharing is performed at the embedding layer of the named entity recognition task and the knowledge graph embedding task. The input text is input to the information fusion module, and the information fusion module processes the input text to obtain a vector representation of each character in the input text. Specifically, this includes:

[0043] For the input text T = {a1, a2, ..., a...} n}, where a represents a character in the text, n represents the length of the input text, and vector representations of all characters and words in the text are obtained from the shared embedding space to obtain the output T. emb :

[0044] T emb ={e a1 e a2 , ..., e an e w1 , ..., e ww};

[0045] Where m represents the number of words obtained after T is segmented; e w A vector representation of a word, e wm This represents the vector representation of the m-th word; e an This represents the vector of the nth character.

[0046] Obtain the vector R of all relations emb ={e r1 e r2 , ..., e rz}, where e rz The vector represents the relation, z represents the number of relation vectors; and the start and end positions of each word in the text T are obtained;

[0047] T emb Words embedded in W emb ={e w1 , ..., e wm} and relational embeddings in R emb ={e r1 e r2 , ..., e rz Connecting these together yields a new vector F. emb ={e f1 e f2 , ..., e fu}, where u = m * z; represents all possible combinations of m word vectors and z relation vectors, and u represents the number of all combinations; e fu Represents the vector after the u-th combination;

[0048] Based on the position information of each word, retrieve the characters related to the word, and use the corresponding characters and words as the input to the Transformers module:

[0049] input = [e a e f1 e f2 , ...e fu ];

[0050] Define the input to contain only the character vector e. a and the word vector e that has an opinion on it. f ;

[0051] The Transformers module performs self-attention on the sequence through multi-head attention, concatenating the results from multiple heads. The computation process of the self-attention mechanism is as follows:

[0052] Q = input * W1;

[0053] K = input * W2;

[0054] V = input * W3;

[0055]

[0056] Where d represents the dimensions of Q and K; W1, W2, and W3 are three trainable weight matrices; Q represents the query matrix; K represents the key matrix; and V represents the value matrix.

[0057] final output mid A matrix sequence of the same length as the input;

[0058] token = concat(output) mid )*W;

[0059] output={token1, token2,..., token n};

[0060] Where, token represents the character representation vector generated after the initial input text characters pass through the information fusion module; output represents the set sequence of all tokens;

[0061] output midThe sequences are concatenated into a two-dimensional vector and multiplied by the weight matrix W to convert it into a one-dimensional token representation.

[0062] The token output is fed into an encoding layer to extract the inherent relationships between characters.

[0063] Furthermore, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a Chinese named entity recognition program with fusion knowledge graph embedding stored in the memory and executable on the processor, wherein when the Chinese named entity recognition program with fusion knowledge graph embedding is executed by the processor, it implements the steps of the Chinese named entity recognition method with fusion knowledge graph embedding as described above.

[0064] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a Chinese named entity recognition program that integrates knowledge graph embedding, and when the Chinese named entity recognition program that integrates knowledge graph embedding is executed by a processor, it implements the steps of the Chinese named entity recognition method that integrates knowledge graph embedding as described above.

[0065] In this invention, a knowledge graph is constructed on a named entity recognition dataset. This knowledge graph represents the relationships between words and entities in the dataset. A multi-task training model is jointly trained by combining the named entity recognition task and the knowledge graph embedding task. An information fusion module within this model extracts information between the named entity recognition task and the knowledge graph embedding task. Knowledge sharing is performed at the embedding layers of both tasks. Input text is fed into the information fusion module, which processes the input text to obtain a vector representation of each character. This invention uses a knowledge graph to represent the correspondence between words and entities in text, enabling better extraction of word-level information, achieving better representation results, and improving the accuracy of named entity recognition. Attached Figure Description

[0066] Figure 1 This is a flowchart of a preferred embodiment of the Chinese named entity recognition method integrating knowledge graph embedding of the present invention;

[0067] Figure 2 This is a diagram of the overall network structure of the multi-task training model in a preferred embodiment of the Chinese named entity recognition method that integrates knowledge graph embedding, as described in this invention.

[0068] Figure 3 This is a network structure diagram of the named entity recognition task in a preferred embodiment of the Chinese named entity recognition method that integrates knowledge graph embedding according to the present invention.

[0069] Figure 4 This is a schematic diagram of the internal structure of the information fusion module in the named entity recognition task in a preferred embodiment of the Chinese named entity recognition method that integrates knowledge graph embedding according to the present invention.

[0070] Figure 5 This is a schematic diagram of f1 on datasets with different proportions in a preferred embodiment of the Chinese named entity recognition method that integrates knowledge graph embedding according to the present invention.

[0071] Figure 6 This is a schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0072] The field of Chinese named entity recognition has gradually shifted from character-based research to information augmentation research. Information augmentation mainly employs two methods: enhancing word information and enhancing glyph information. Lattice-LSTM, SoftLexicon, and FLAT are specific implementations of word augmentation. Lattice-LSTM, based on the common LSTM model, introduces word information related to the current character by constructing a dictionary, supplementing the model's input. SoftLexicon improves upon Lattice-LSTM by simplifying its overly complex model structure and fusing word information through connection vectors. FLAT improves upon the dynamically adjusted and represented dot matrix structure, using only positional encoding and introducing Transformers to simulate grid input. The self-attention mechanism of Transformers allows characters to interact with any potential word, including self-matching words. MECT is a representative work on glyph-based information augmentation, inspired by the influence of prefixes and suffixes on English words. In recent work, multimodal information augmentation is also a viable method. MPM-CNER13 is a model that learns multimodal representations of Chinese semantics, glyphs, and speech. A novel cross-modal attention mechanism is proposed to fuse these multimodal features to further improve NER performance.

[0073] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0074] The preferred embodiment of the Chinese named entity recognition method integrating knowledge graph embedding described in this invention, such as... Figure 1 As shown, the Chinese named entity recognition method integrating knowledge graph embedding includes the following steps:

[0075] Step S10: Construct a knowledge graph on the named entity recognition dataset, wherein the knowledge graph is used to represent the relationship between words and entities in the named entity recognition dataset.

[0076] Specifically, a corresponding knowledge graph was constructed based on the named entity recognition dataset. The knowledge graph's triples consist of Word, Entity Type, and Entity. Word refers to the set of all words appearing in the named entity recognition dataset after word segmentation; EntityType refers to all entity types in the named entity recognition dataset, such as PER, ORG, etc.; and Entity refers to the entity labeled in the named entity recognition dataset. Representing the correspondence between words and entities in text using a knowledge graph allows for better extraction of word-level information.

[0077] A knowledge graph is a directed graph with real-world entities as nodes and relationships between entities as edges. While knowledge graphs are very efficient at representing data, their triple-based notation makes them difficult to manipulate. Knowledge graph embedding aims to solve this problem. Its main purpose is to embed the components of a knowledge graph into a continuous vector space.

[0078] Early knowledge graph embedding generally used methods based on transformation distance and semantic matching. TransE, TransH, SME, and DistMult are some related research works. RotatE is a knowledge embedding method inspired by Euler's identity. Specifically, the RotatE model maps entities and relations to a complex vector space and defines each relation as a rotation from the source entity to the target entity.

[0079] Two-dimensional convolutional operations are the best way to facilitate the integration of entities and relations. ConvE was the first to use convolutional neural networks for knowledge graph embedding. However, not all entities and relations can interact at any given time during convolutional operations. ConvE internally constructs 2D convolutional kernels, which reduces some honorific parameters. ConvE uses entity embeddings to construct 2D entity-specific convolutional kernels and performs convolution operations on relation embeddings; it then uses relation convolutional kernels to operate on entity embeddings in a similar way. This mechanism greatly facilitates interaction between entities and relations.

[0080] In order to integrate entity information corresponding to words during word enhancement, this invention uses a novel construction rule to construct a knowledge graph based on a dataset.

[0081] All text in the named entity recognition dataset is segmented into a word set (WordSet); all entity categories appearing in the named entity recognition dataset are added to a relation set (RelationshipSet); all tagged entities in the named entity recognition dataset are added to an entity set (EntitySet); the knowledge graph is constructed from the WordSet, RelationshipSet, and EntitySet to complete the construction of a knowledge graph based on words and entities, according to the following rules:

[0082] {w, r, e}, (W∈WordSet, r∈RelationShipSet, e∈EntitySet);

[0083] Where w refers to a single word after word segmentation; r refers to a certain entity category in the named entity recognition dataset, defining the entity category as a relation; and e refers to an entity tagged in a certain named entity recognition dataset.

[0084] For example, here are a few examples of successfully constructed triples:

[0085] {Beijing, Belong to LOC, Beijing};

[0086] {Beijing, Belong to ORG, Beijing Zoo};

[0087] {Animals, Belong to ORG, Beijing Zoo};

[0088] Complete the construction of a knowledge graph based on words and entities by following the steps and rules above.

[0089] Step S20: Combine the named entity recognition task and the knowledge graph embedding task into a multi-task training model for joint training, and extract the information between the named entity recognition task and the knowledge graph embedding task based on the information fusion module in the multi-task training model.

[0090] Specifically, such as Figure 2As shown, the overall framework of CEKE (Chinese NER with Knowledge Graph Embedding, a multi-task training model integrating knowledge graph embedding) is illustrated. CEKE is a multi-task training model (framework) that includes knowledge graph embedding and named entity recognition tasks. CEKE applies a training method that integrates knowledge embedding to the named entity recognition task. Information transfer between two entities is achieved by placing the entity representations of the KGE (Knowledge Graph Embedding) task and the character and word representations of the NER (Named Entity Recognition) task in the same continuous vector space, i.e., sharing the embedding layer representations.

[0091] The loss function of the multi-task training model CEKE consists of the loss function of the knowledge graph embedding task and the loss function of the named entity recognition task.

[0092] The loss function of the multi-task training model is expressed as:

[0093] Loss = Loss1 + Loss2;

[0094] Wherein, Loss represents the loss function of the multi-task training model, Loss1 represents the loss function of the knowledge graph embedding task, and Loss2 represents the loss function of the named entity recognition task.

[0095] For the Knowledge Graph Embedding (KGE) task, the multi-task training model CEKE uses a dual-path convolutional neural network model similar to JointE to perform the knowledge embedding task. The dual paths include a first path Path1 and a second path Path2. The first path Path1 uses one-dimensional convolution to extract explicit relationships between entities and relations, while the second path Path2 uses two-dimensional convolution to extract intrinsic and implicit information between entities and relations.

[0096] First, the knowledge graph triples constructed based on the named entity recognition dataset are defined as h, r, and t, where h, r, and t represent the head entity, relation, and tail entity, respectively.

[0097] For the first path Path1, first concatenate the inputs h and r to obtain the vector M. inp Then use convolution kernel and convolution kernel In vector M inp We obtain v1, where c represents the size of the first dimension of the convolution kernel.

[0098]

[0099]

[0100]

[0101] Where k represents the size of the first dimension; M mid v1 represents the intermediate result obtained by performing a convolution operation between the vector concatenated with h and r and the convolution kernel; v1 represents the output result obtained by the first path of the knowledge graph embedding task, and v2 represents the result of extracting the explicit relationship between entities and relations.

[0102] For the second path Path2, h and r are first transformed into entity vector v. h and relation vector Where k1*k2=k, k1 and k2 represent the dimensionality of the one-dimensional h and r after transforming them into two-dimensional shapes; then, a solid convolution filter ω is constructed for each. h And a relational convolutional filter ω r Then, for the relation vector v r Use solid convolution filter ω h For entity vector v h Use a relational convolution filter ω r Using entity convolution kernels to process relation vectors and relation convolution kernels to process entity vectors is equivalent to cross-implementing information fusion.

[0103]

[0104]

[0105] Where M1 represents the relationship between v h The output obtained after applying a convolution kernel; M2 represents the output obtained after applying a convolution kernel to v. r The output obtained after applying convolution kernels; b, p, and q represent the number, height, and width of the convolution kernels, respectively, and b*p*q = k1*k2.

[0106] Finally, M1 and M2 are concatenated, and the output v2 is calculated using a weight matrix W; where the dimension of the weight matrix W is... This is used to ensure that the dimensions of v1 and v2 are consistent.

[0107]

[0108] Where v2 represents the output result obtained from the second path of the knowledge graph embedding task, and represents the extraction result of the implicit relationship between entities and relations; o represents the parameter, calculated as follows:

[0109] o=b*2*(k1-p+1)*(k2-q+1).

[0110] Add the output v1 obtained from the first path Path1 and the output v2 obtained from the second path Path2 to obtain the final output v, and calculate the current score:

[0111] v = v1 + v2;

[0112] Score = vv T ;

[0113] Here, "Score" represents the result, and the model parameters for the current task are optimized based on the score, making the model (specifically, the Knowledge Graph Embedding task KGE in CEKE) more accurate in deriving t from h and r. For CEKE, the main role of the Knowledge Graph Embedding task is to obtain the optimal entity representation h and the optimal relation representation r. The KGE and NER tasks in CEKE share the vector space of the input layer. Therefore, the knowledge about entities and relations learned by the KGE task can be shared with the NER task. Conversely, the NER task can also pass training information to the KGE task by sharing the embedding layer.

[0114] For Named Entity Recognition (NER) tasks, such as Figure 3 The diagram illustrates the basic structural design of CEKE's Named Entity Recognition (NER) task. The most crucial component is the Information Fusion Module. CEKE comprises two tasks: Knowledge Graph Embedding (KGE) and Named Entity Recognition (NER). The Information Fusion Module is a part of the Named Entity Recognition task. Its main function is to add implicit information from the KGE task within the shared embedding space to the input of the NER task, focusing more on highly relevant information and reducing the impact of irrelevant information on the input representation.

[0115] Step S30: Knowledge sharing is performed in the embedding layer of the named entity recognition task and the knowledge graph embedding task. The input text is input to the information fusion module, and the information fusion module processes the input text to obtain the vector representation of each character in the input text.

[0116] Specifically, the implementation process of the information fusion module is as follows: Figure 4 As shown, for the input text T = {a1, a2, ..., a...} n}, where 'a' represents a character in the text and 'n' represents the length of the input text. First, vector representations of all characters and words in the text are obtained from the shared embedding space, resulting in the output T. emb :

[0117] T emb ={e a1 e a2 , ..., ean e w1 , ..., e ww};

[0118] Where m represents the number of words obtained after T is segmented; e w A vector representation of a word, e wm This represents the vector representation of the m-th word; e an This represents the vector of the nth character.

[0119] Then we obtain the vector R of all relations. emb ={e r1 e r2 , ..., e rz}, where e rz The vectors representing relations are given, where z represents the number of relation vectors; and the start and end positions of each word in the text T are obtained; thus, we have the vector representation of the text, the vector representation of the relations, and the position information of the words.

[0120] Many current research methods combine characters and words to construct input, enhancing its information content. FLAT and ALFLAT both employ self-attention mechanisms, extracting relevant information from different words through horizontally concatenated character and word vectors. The initial intention behind this network structure design was to simplify the previous Lattice structure used to introduce word information. FLAT allows each character to extract useful information from other characters and word vectors through Transformers. However, this simple structure of applying all characters and words together to Transformers is not specific enough for information extraction. For example, if we want to extract information from words related to a character to enhance its representation, we don't need to pass other unrelated words to the Transformers. Therefore, this invention modifies the FLAT model to address this issue.

[0121] First, put T emb Words embedded in W emb ={e w1 , ..., e wm} and relational embeddings in R emb ={e r1 e r2 , ..., e rz Connecting these together yields a new vector F. emb ={e f1 e f2 , ..., e fu}, where u = m * z; represents all possible combinations of m word vectors and z relation vectors, and u represents the number of all combinations; e fuLet represent the vector after the u-th combination.

[0122] Then, based on the position information of each word, the characters related to the word are obtained, and the corresponding characters and words are used as the Transformers module (Transformers module refers to...). Figure 4 The inputs for Feed Forward, LayerNorm, and Self-Attention:

[0123] input = [e a e f1 e f2 , ..., e fu ];

[0124] Define the input to contain only the character vector e. a and the word vector e that has an opinion on it. f This avoids the interference of irrelevant words with the self-attention mechanism, thus preventing it from affecting the accuracy of the final representation. Using such input allows characters to allocate different attention to the probabilities of different entity types in the representation. This can have a significant impact on text with obvious entity type labels.

[0125] The Transformers module performs self-attention on the sequence through multi-head attention, and then concatenates the results of the multi-head attention. The computation process of the self-attention mechanism is as follows:

[0126] Q = input * W1;

[0127] K = input * W2;

[0128] V = input * W3;

[0129]

[0130] Where d represents the dimensions of Q and K; W1, W2, and W3 are three trainable weight matrices; Q represents the query matrix; K represents the key matrix; and V represents the value matrix (which is the information to be extracted from the current input).

[0131] final output mid It is a matrix sequence of the same length as input.

[0132] token = concat(output) mid )*W;

[0133] output={token1, token2,..., token n};

[0134] Among them, token represents the character representation vector generated after the input text characters pass through the information fusion module; output represents the set sequence of all tokens.

[0135] Finally, the output mid sequence is concatenated into a two-dimensional vector and multiplied by the weight matrix W to be converted into a one-dimensional token representation. At this point, the work of converting a character in the input text sequence T into a vector token is completed; the purpose is to obtain a vector representation of each character in the input text, token is the finally output character vector, and output is the sequence after all characters in the input text become vectors. For example, for the text input "Who are you", after passing through this module, "你" becomes token1, "是" becomes token2, "谁" becomes token3, and output is {token1, token2, token3}.

[0136] The obtained token output is put into the BiLSTM encoding layer to extract the internal relationship between characters. This step generates classification scores for all possible entity types of each input. Finally, the deviation between the predicted score and the actual value needs to be optimized using a conditional random field (CRF). CRF can calculate the dependencies between the classification result labels and reduce the errors caused by unreasonable outputs.

[0137] Furthermore, in order to verify the performance of CEKE on the Chinese named entity recognition dataset, the present invention used a total of five Chinese NER datasets in the experimental part. Four of these datasets belong to the general domain, namely: MSRA, Resume, Weibo, and OntoNote. And a legal dataset of CAIL was also added. The characteristic of this dataset is that the data volume is small and the entity categories are complex, so as to verify the performance of CEKE in a specific domain with a small data volume. The experimental results are analyzed as shown in Table 1 below:

[0138] OntoNotes MSRA Resume Weibo BiLSTM 71.81％ 91.87％ 94.41％ 56.75％ Tenner 72.82％ 93.01％ 95.25％ 58.39％ Lattice-LSTM 73.88％ 93.18％ 94.46％ 58.79％ SoftLexicon 75.64％ 93.66％ 95.77％ 63.02％ FLAT 76.45％ 94.12％ 95.45％ 60.32％ MECT 76.92％ 94.32％ 95.89％ 62.9％ CEKE(ours) 76.9％ 94.56％ 95.53％ 63.2％

[0139] Table 1: f1 scores of CEKE and other models on the general dataset

[0140] Among them, the f1 value = (2 * precision * recall) / (precision + recall), precision = the number of correctly recognized entities / the number of recognized entities, recall = the number of correctly recognized entities / the number of entities in the sample, and the role of f1 is the weighted harmonic mean of precision and recall.

[0141] Table 1 shows the performance of CEKE and other compared models on four general datasets. Compared to the traditional NER serialization model BiLSTM, CEKE achieves a performance improvement of 1%-7% on the four datasets. The improvement is most significant on the Weibo dataset. This is because the Weibo dataset has a relatively small amount of data and relatively complex entity types. This also demonstrates that CEKE can achieve greater advantages on datasets with limited training text. Furthermore, compared to models that incorporate external dictionaries proposed in recent years, CEKE also achieves better results on most models.

[0142]

[0143]

[0144] Table 2: Performance of CEKE and the comparison model on the CAIL dataset

[0145] Table 2 shows the performance of CEKE on the CAIL dataset. Compared to BiLSTM-CRF, CEKE achieves a significant lead in F1 scores for both the overall performance and individual entity types, even improving by 10% for NATS entity types. In the experiments, this invention also introduced a BERT pre-trained model, which also uses the Transformers mechanism, for comparison with CEKE. The experimental results show that CEKE is not inferior to BERT in entity recognition accuracy, and achieves better results in some entity types.

[0146] To validate CEKE's performance using limited data, this invention employs experiments with different proportions of the CAIL dataset. Figure 5 It is clear that as the amount of data decreases, CEKE ( Figure 5 The gray line in the model is compared to the baseline model. Figure 5 The advantage of the black line in the middle is greater.

[0147] To obtain more comprehensive knowledge information, this invention proposes CEKE, a multi-task training model that integrates knowledge graph embedding. To better learn the relationships between words and entities, a knowledge graph is constructed based on the dataset. Furthermore, CEKE innovatively introduces an information fusion module to enhance information sharing between the two tasks. In the experimental phase, CEKE was applied to four general Chinese NER (Named Entity Recognition) datasets, achieving at least a 0.5%-1% improvement compared to other models. To verify the model's performance on smaller datasets, this invention applied CEKE to the CAIL (Challenge of AI in Law) legal dataset, achieving a significant 10% improvement.

[0148] This invention proposes a multi-task training model for the Chinese named entity recognition problem: CEKE. It uses a knowledge graph embedding task and a traditional named entity recognition task as the main framework for joint training. It uses a newly proposed rule to construct a knowledge graph based on the dataset to represent the relationship between words and entities in the dataset. It uses the knowledge graph embedding task to provide additional knowledge information for the named entity recognition task. It uses an information fusion module to share and select information between the two tasks to guide the training direction of the two tasks.

[0149] Through subsequent experiments, this invention has also verified that CEKE outperforms the baseline comparison model on various Chinese NER datasets, especially on datasets with smaller data volume and more complex entity types.

[0150] The beneficial effects of this invention are as follows:

[0151] (1) A new method is proposed to construct a knowledge graph on the named entity recognition dataset, which can better represent the relationship between words and entities in the dataset.

[0152] (2) The named entity recognition task and the knowledge graph embedding task are combined into a multi-task training framework for training, and an information fusion module is introduced to extract information between the two tasks.

[0153] (3) Under the framework of multi-task training, knowledge sharing is carried out in the embedding layer of the two tasks. Each task can select appropriate information to supplement its own input in order to obtain better representation results.

[0154] Furthermore, such as Figure 6 As shown, based on the above-mentioned Chinese named entity recognition method and system with knowledge graph embedding, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 6 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0155] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, SmartMediaCard (SMC), SecureDigital (SD) card, or FlashCard. Furthermore, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a Chinese named entity recognition program 40 with knowledge graph embedding, which can be executed by the processor 10 to implement the Chinese named entity recognition method with knowledge graph embedding in this application.

[0156] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the Chinese named entity recognition method with fused knowledge graph embedding.

[0157] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0158] In one embodiment, when the processor 10 executes the Chinese named entity recognition program 40 with knowledge graph embedding in the memory 20, the steps of the Chinese named entity recognition method with knowledge graph embedding are implemented.

[0159] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a Chinese named entity recognition program that integrates knowledge graph embedding, and the Chinese named entity recognition program that integrates knowledge graph embedding implements the steps of the Chinese named entity recognition method that integrates knowledge graph embedding as described above when executed by a processor.

[0160] In summary, this invention provides a Chinese named entity recognition method and related equipment that integrates knowledge graph embedding. The method includes: constructing a knowledge graph on a named entity recognition dataset, wherein the knowledge graph represents the relationships between words and entities in the named entity recognition dataset; jointly training a multi-task training model consisting of a named entity recognition task and a knowledge graph embedding task; extracting information between the named entity recognition task and the knowledge graph embedding task based on an information fusion module in the multi-task training model; sharing knowledge at the embedding layers of the named entity recognition task and the knowledge graph embedding task; inputting input text into the information fusion module; and processing the input text to obtain a vector representation of each character in the input text. This invention uses a knowledge graph to represent the correspondence between words and entities in text, which can better extract word-level information, obtain better representation results, and improve the accuracy of named entity recognition.

[0161] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0162] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0163] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A Chinese named entity recognition method integrating knowledge graph embedding, characterized in that, The Chinese named entity recognition method that integrates knowledge graph embedding includes: A knowledge graph is constructed on a named entity recognition dataset, the knowledge graph being used to represent the relationships between words and entities in the named entity recognition dataset; The named entity recognition task and the knowledge graph embedding task are combined into a multi-task training model for joint training. The information fusion module in the multi-task training model is used to extract the information between the named entity recognition task and the knowledge graph embedding task. Knowledge sharing is performed at the embedding layer of the named entity recognition task and the knowledge graph embedding task. The input text is input to the information fusion module, and the information fusion module processes the input text to obtain a vector representation of each character in the input text. The multi-task training model uses a dual-path convolutional neural network model to perform knowledge embedding tasks. The dual paths include a first path Path1 and a second path Path2. Among them, the first path Path1 uses one-dimensional convolution to extract explicit relationships between entities and relations, while the second path Path2 uses two-dimensional convolution to extract intrinsic and implicit information between entities and relations; The knowledge graph triples constructed based on the named entity recognition dataset are defined as follows: , and ,in, , and These represent the head entity, relation, and tail entity, respectively. For the first path Path1, connect the input. and To obtain a vector Using convolution kernels and convolution kernel In vector Get it ,in, This indicates the size of the first dimension of the convolution kernel; ； ； ； in, Indicates the size of the first dimension; Indicates use and The intermediate result obtained by convolving the concatenated vector with a convolution kernel; This represents the output result obtained from the first path of the knowledge graph embedding task, and represents the extraction result of explicit relationships between entities and relations; For the second path Path2, and Transform into entity vectors and relation vector , ,in, , and Representing one-dimensional and The dimensional dimensions after being transformed into a two-dimensional shape; Construct a solid convolution filter respectively and a relational convolutional filter For relation vectors Use solid convolution filters For entity vectors Use relational convolution filters ; ； ； in, Indicates to The output obtained after applying a convolution kernel; Indicates to The output obtained after applying a convolution kernel; , and These represent the number of convolution kernels, their height, and their width, respectively. ; Will and Concatenate the components using a weight matrix. Calculate the output Among them, the weight matrix The dimension is Used to guarantee and The dimensions are consistent; ； in, The output of the second path of the knowledge graph embedding task represents the result of extracting the implicit relationship between entities and relations. The parameter is represented and calculated as follows: ； The output obtained from the first path Path1 Output results obtained from the second path Path2 Add them together to get the final output. And calculate the current score: ； ； in, The score represents the value, and the model parameters for the current task are optimized based on the score.

2. The Chinese named entity recognition method integrating knowledge graph embedding according to claim 1, characterized in that, The construction of a knowledge graph on the named entity recognition dataset, wherein the knowledge graph is used to represent the relationships between words and entities in the named entity recognition dataset, specifically includes: All text in the named entity recognition dataset is segmented into words to obtain a set of words. ; Add all entity categories appearing in the named entity recognition dataset to the relation set. ; Add all entities tagged in the named entity recognition dataset to the entity set. ; The knowledge graph consists of a set Relationship set and entity set The following steps are taken to construct a knowledge graph based on words and entities, following the established rules: ； in, Refers to a single word after word segmentation; This refers to a specific entity category in a named entity recognition dataset, defining the entity category as a relation. It refers to the entities tagged in a named entity recognition dataset.

3. The Chinese named entity recognition method integrating knowledge graph embedding according to claim 1, characterized in that, The loss function of the multi-task training model consists of the loss function of the knowledge graph embedding task and the loss function of the named entity recognition task; The loss function of the multi-task training model is expressed as: ； in, This represents the loss function of the multi-task training model. This represents the loss function for the knowledge graph embedding task. This represents the loss function for the named entity recognition task.

4. The Chinese named entity recognition method integrating knowledge graph embedding according to claim 1, characterized in that, The process involves knowledge sharing at the embedding layer of the named entity recognition task and the knowledge graph embedding task. The input text is fed into the information fusion module, which processes the input text to obtain a vector representation of each character in the input text. Specifically, this includes: For input text ,in, Represents characters in the text. The length of the input text is represented by a vector representation of all characters and words in the text obtained from the shared embedding space, and the output is obtained. : ； in, express The number of words obtained after word segmentation; A vector representation of a word. Indicates the first Each word vector is represented; Indicates the first A character vector; Obtain the vector of all relations ,in, Vectors representing relations This indicates the number of relation vectors; and retrieves the text for each word. The beginning and end positions in the text; Will Word embedding in } and relational embedding Connect them to get a new set of vectors. ,in, ;right Word vectors and Represent all possible combinations of relation vectors. Indicates the number of all combinations; Indicates the first The combined vector; Based on the position information of each word, retrieve the characters related to the word, and use the corresponding characters and word as the Transformers module. : ； definition It contains only character vectors and word vectors with viewpoints ; The Transformers module performs self-attention on the sequence through multi-head attention, concatenating the results from multiple heads. The computation process of the self-attention mechanism is as follows: ； ； ； ； in, express and The dimension; , , There are three trainable weight matrices; Represents the query matrix; Represents the key matrix; Represents the value matrix; Final output To and A sequence of matrices of equal length; ； ； in, This represents the character representation vector generated after the initial input text characters pass through the information fusion module; Indicates all A sequence of sets; Will The sequences are concatenated into a two-dimensional vector and then combined with the weight matrix. Multiply to convert to one dimension express; Will The output is fed into an encoding layer to extract the inherent relationships between characters.

5. A terminal, characterized in that, The terminal includes: a memory, a processor, and a Chinese named entity recognition program with fusion knowledge graph embedding stored in the memory and executable on the processor. When the Chinese named entity recognition program with fusion knowledge graph embedding is executed by the processor, it implements the steps of the Chinese named entity recognition method with fusion knowledge graph embedding as described in any one of claims 1-4.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a Chinese named entity recognition program that integrates knowledge graph embedding. When the Chinese named entity recognition program that integrates knowledge graph embedding is executed by a processor, it implements the steps of the Chinese named entity recognition method that integrates knowledge graph embedding as described in any one of claims 1-4.

Images