Chinese entity recognition model and method based on vocabulary enhancement and character external information

Abstract

The invention discloses a Chinese entity recognition model and method based on vocabulary enhancement and character external information, and solves the problem that static weights of matched words cannot be determined by fully utilizing the Chinese character external information and only depending on word frequencies. The Chinese entity recognition model is sequentially connected with a Chinese character information extraction part, a Chinese character feature extraction part containing dynamic matching word features, a two-way encoder part for enhancing character embedding and component embedding and a decoding output part. The implementation method comprises the steps of data cleaning, Chinese character feature extraction containing dynamic matching word features, and two-way coding and decoding output of enhanced character embedding and component embedding. The method comprises the following steps of: fusing vocabularies, pinyin and component information of Chinese characters into a model by using a two-way coding structure; when the matching word features are extracted, dynamic weights are used for embedding and weighting the matching words, the method is more balanced and reasonable, weight distribution is prevented from being excessively biased to high-frequency words, and generalization ability and recognition precision are improved. The method is used for extracting Chinese entities from unstructured input texts.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and natural language processing, and mainly relates to Chinese named entity recognition. Specifically, it is a Chinese entity recognition model and method based on vocabulary enhancement and character external information, which is used to extract Chinese entities such as names of people, places, and organization names from unstructured input text. It has wide applications in multiple fields such as intelligent customer service, intelligent recommendation, and intelligent question and answer. Background Art

[0002] The purpose of named entity recognition (NER) is to detect entities from sentences and identify pre-defined types, such as names of people, places, and institutions. Named entity recognition is the basis of many downstream natural language processing (NLP) problems, such as knowledge graphs, information retrieval, question-answering systems, and machine translation. As one of the most widely spoken languages ​​in the world, Chinese plays an increasingly important role in international communication, business dealings, and cultural dissemination. The popularity of the Internet has brought a large amount of Chinese text data, including social media, news reports, and academic articles. Chinese named entity recognition technology can extract key information from it, helping users quickly find what they need, thereby improving the efficiency of information retrieval and processing.

[0003] Early named entity recognition methods were mainly based on manually written rules and dictionaries to match named entities in text. However, these rules are often strongly related to the domain and language style of the text. The establishment of rules and dictionaries relies on the knowledge of experts and is extremely time-consuming. In addition, such systems have poor portability and are not easy to use across scenarios.

[0004] The statistical method converts the NER task into a sequence labeling task and uses a manually labeled corpus for training. The cost of data labeling is much lower than the cost of designing rules, and the statistical method is universal and does not require too many manually designed rules. It gradually became the mainstream method before the outbreak of deep learning. Commonly used statistical models include conditional random fields, hidden Markov models, maximum entropy models, etc. However, the statistical method relies on manually designed features, such as part of speech, glyph features, etc., which requires the knowledge and experience of domain experts. Incorrect feature design will seriously affect model performance.

[0005] In recent years, deep learning has made significant breakthroughs in the field of NLP. When deep learning is applied to named entity recognition, it can learn complex hidden representations without complex feature engineering and rich domain knowledge. In addition, named entity recognition methods based on deep learning are more generalizable and more universal. Therefore, named entity recognition methods based on deep learning are more widely used and have higher recognition accuracy than traditional rule-based and statistical methods. There have been many studies on named entity recognition methods based on deep learning, but they mainly focus on English named entity recognition. Chinese named entity recognition is more difficult than English named entity recognition, which is mainly reflected in two aspects. First, the significant difference between Chinese and English is that the basic unit of Chinese is Chinese characters, while English is words, and Chinese does not have obvious separators like spaces in English. Second, many entities in English have obvious characteristics, that is, the first letter is capitalized. This makes it more difficult to identify the boundaries of Chinese entities, while the main task of English entity recognition is to identify the type of entity.

[0006] Since there are no clear separators in Chinese, most early studies have performed word segmentation before performing Chinese NER. However, errors in Chinese word segmentation may lead to incorrect recognition of entity boundaries, thus affecting model performance. Recently proposed NER methods are all character-based. Character-based methods can eliminate word segmentation errors, but at the same time lose word information. In Chinese NER, word information and lexical boundary information are crucial. Some recent NER methods based on vocabulary enhancement use external dictionaries to supplement the lost lexical information. By integrating the lexical information in the dictionary into the character-level representation layer, the NER model can more accurately capture the entity information in the language. Zhang Y et al. proposed Lattice LSTM in their paper “Chinese NER using lattice LSTM” (Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018: 1554-1564.). Lattice LSTM is the first method to obtain latent word information from an external dictionary and incorporate it into a character-based Chinese named entity model. It introduces word information without relying on the word segmentation results, avoiding error propagation caused by word segmentation. R Ma et al. proposed SoftLexicon in their paper “Simplify the Usage of Lexicon in Chinese NER” (Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 5951-5960.). SoftLexicon integrates lexical information into character representation, thereby avoiding complex modeling structures. These methods of injecting word information into the model through external dictionaries have achieved promising results on several Chinese NER datasets. However, after integrating dictionary information into character representation, most existing methods ignore additional information of Chinese characters (such as strokes, pinyin, etc.), which has been shown to be beneficial to improving the performance of Chinese NER models. After introducing a variety of external information, how to effectively integrate them into the model is also a key issue. In addition, existing vocabulary enhancement-based methods are directly based on the frequency of matching words as their weights, which does not capture the different associations between characters and vocabulary in different contexts.

[0007] Existing technologies integrate external dictionary information to make up for the lost vocabulary information and word boundary information in character-based Chinese named entity methods. These methods often ignore the additional information of Chinese characters (such as strokes, pinyin, etc.), which can further enrich the feature representation and is very helpful in improving model performance. In addition, existing methods are mostly based on word frequency as weight, which fails to fully capture the correlation between characters and words in different contexts. Summary of the invention

[0008] The purpose of the present invention is to overcome the shortcomings of the existing technology and propose a Chinese named entity recognition model and method based on vocabulary enhancement and character external information with higher recognition accuracy and stronger generalization ability.

[0009] The present invention is a Chinese entity recognition model based on vocabulary enhancement and character external information, characterized in that a Chinese character information extraction part, a Chinese character feature extraction part containing dynamic matching word features, a dual-path encoder part for enhancing character embedding and component embedding, and a decoding output part are sequentially connected; wherein the Chinese character information extraction part is provided with a character extraction module, and based on the character extraction module, a matching word extraction module, a pinyin extraction module and a component extraction module are also provided; the Chinese character feature extraction part containing dynamic matching word features includes a character feature extraction module, a dynamic matching word feature extraction module, a pinyin feature extraction module and a component feature extraction module, wherein the character feature extraction module uses the output of the character extraction module in the Chinese character information extraction part as input to obtain character embedding; the dynamic matching word feature extraction module first ... The output of the matching word extraction module and the output of the character extraction module in the Chinese character information extraction part are taken as input, and then the weighted sum of the matching word embeddings is performed based on the attention mechanism to obtain the dynamic matching word feature embedding; the pinyin feature extraction module extracts the pinyin feature embedding with the output of the pinyin extraction module in the Chinese character information extraction part as input; the component feature extraction module extracts the component feature embedding with the output of the component extraction module in the Chinese character information extraction part as input; the dual-path encoder part of the enhanced character embedding and component embedding includes two branches, which are respectively used to encode the contextual features of the component embedding and the contextual features of the enhanced character embedding that integrates the dynamic matching word features and the pinyin features, and then the component contextual features and the enhanced character contextual features are spliced ​​as the dual-path encoder output; the decoding output part is used to decode the encoder output to obtain the optimal label sequence.

[0010] The present invention is also a Chinese entity recognition method based on vocabulary enhancement and character external information, which is implemented on any of the Chinese entity recognition models based on vocabulary enhancement and character external information described in claims 1-2, and is characterized in that it includes the following steps:

[0011] (1) Data cleaning: Receive the original Chinese input text, split the long text into several short sentences and remove illegal characters and Unicode characters, complete data cleaning, and obtain the cleaned sentences;

[0012] (2) Extraction of Chinese character features containing dynamic matching word features: Extract multiple features of the input sentence after data cleaning, including character features, dynamic matching word features, pinyin features, and component features; extracting character features is to input the cleaned sentence into the BERT model to obtain the feature embedding of each character; extracting dynamic matching word features is to first perform string matching between each character and the external dictionary to obtain the matching word, and then calculate the dynamic weight of each matching word based on the matching word embedding and the character feature embedding, and then weighted sum the embeddings of each matching word to obtain the final dynamic matching word feature embedding; extracting pinyin features is to obtain the pinyin of each Chinese character in the input sentence, and use a convolutional neural network to extract and obtain the pinyin feature embedding; the character feature embedding, dynamic matching word feature embedding, and pinyin feature embedding are sequentially spliced ​​to obtain the enhanced character embedding; when extracting component features, first decompose each Chinese character in the input sentence to obtain the components that make up the Chinese character, and then use a convolutional neural network to extract and obtain the component feature embedding to complete the extraction of Chinese character features containing dynamic matching word features;

[0013] (3) Dual-path encoding of enhanced character embedding and component embedding: Use two bidirectional gated recurrent units (Bi-GRUs) to extract context features of enhanced character embedding and context features of Chinese character component embedding; Concatenate character embedding, dynamic matching word feature embedding, and pinyin embedding to obtain enhanced character embedding and input it into a Bi-GRU to extract context representation of enhanced character embedding; Input the Chinese character component embedding representation into another Bi-GRU to extract context representation of component embedding, concatenate the component context representation and enhanced character context representation, and complete dual-path encoding of enhanced character embedding and component embedding;

[0014] (4) Decoding output: Select the linear chain conditional random field (CRF) as the decoder, and input the encoded context representation that integrates Chinese character information, matching word information, pinyin information, and component information into the CRF for decoding. The CRF will output the label sequence with the highest score, extract the entities in the sentence based on the label sequence, and complete the decoding output.

[0015] The present invention solves the problem that the existing methods fail to fully utilize the external information of Chinese characters and the problem that the existing methods only rely on word frequency to determine the static weight of matching words. The present invention fully utilizes the external information of Chinese characters and improves vocabulary enhancement through dynamic weights, thereby improving the accuracy of Chinese named entity recognition.

[0016] The present invention proposes a two-way encoding structure that effectively integrates the vocabulary information, pinyin information and component information of Chinese characters into the model, greatly enriching the semantic expression of character-based Chinese named entity recognition and solving the problem that existing methods fail to fully utilize the external information of Chinese characters. In addition, when introducing vocabulary information, the present invention adopts an attention mechanism to dynamically adjust the weight of matching words by calculating the correlation between characters and matching words. Compared with the existing method that only relies on word frequency to determine the static weight of matching words, the method of the present invention is more flexible and reasonable in the allocation of matching word weights, and can treat each matching word more evenly, avoiding the situation where the weight allocation is too biased towards high-frequency words, thereby improving the generalization ability and recognition accuracy of the model.

[0017] Compared with the prior art, the present invention has the following advantages:

[0018] Higher accuracy: The present invention integrates pinyin, components and vocabulary information into the character feature extraction process; and adopts a new dual-path encoding fusion strategy, using two parallel modules to process enhanced character representation and Chinese character component embedding respectively during the context encoding process, thereby optimizing the boundary recognition effect of Chinese named entity recognition and effectively improving the recognition accuracy of the model in complex text environments.

[0019] Improve the generalization ability of the model: The present invention uses an attention mechanism to extract dynamic matching word features, and dynamically adjusts the weight of the matching word by calculating the correlation between the characters and the matching words. The dynamic weight flexibly adjusts the matching word weight according to the specific context. The Chinese entity recognition model of the present invention can better capture the different correlations between characters and words in different contexts, and at the same time can treat each matching word more balanced and accurately, avoiding the situation where the weight distribution is too biased towards high-frequency words, and further improving the generalization ability and recognition accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a model block diagram of the present invention;

[0021] Figure 2 It is a model structure diagram of the present invention;

[0022] Figure 3 It is a flowchart of the present invention;

[0023] Figure 4 is a flow chart of the present invention;

[0024] Figure 5 Diagram of the convolutional neural network architecture used to extract pinyin / component features. DETAILED DESCRIPTION

[0025] Example 1

[0026] Most of the existing methods based on vocabulary enhancement in Chinese named entity recognition ignore more external information of Chinese characters after integrating dictionary information into character representation, resulting in insufficient accuracy and robustness of named entity recognition. This is because the entity recognition model fails to fully utilize the rich information of Chinese characters such as structure and pronunciation, and lacks understanding of the structure of Chinese characters themselves. Existing methods based on vocabulary enhancement are directly based on the frequency of matching words as their weights, which do not capture the different associations between characters and vocabulary in different contexts well, and may lose important semantic information, affecting the overall understanding of the text. In response to the above problems, the present invention conducts analysis and research, and proposes a Chinese entity recognition model and method based on vocabulary enhancement and character external information.

[0027] The present invention is firstly a Chinese entity recognition model based on vocabulary enhancement and character external information, see Figure 1 , Figure 1It is a model block diagram of the present invention. The Chinese entity recognition model of the present invention is sequentially connected with a Chinese character information extraction part, a Chinese character feature extraction part containing dynamic matching word features, a dual-path encoder part for enhancing character embedding and component embedding, and a decoding output part. The Chinese character information extraction part of the present invention is provided with a character extraction module, and a matching word extraction module, a pinyin extraction module and a component extraction module are also provided based on the character extraction module. The Chinese character feature extraction part containing dynamic matching word features of the present invention includes a character feature extraction module, a dynamic matching word feature extraction module, a pinyin feature extraction module and a component feature extraction module, wherein the character feature extraction module uses the output of the character extraction module in the Chinese character information extraction part as the input of the character feature extraction module to obtain character embedding. The dynamic matching word feature extraction module first uses the output of the matching word extraction module in the Chinese character information extraction part and the output of the character extraction module as input, and then performs weighted summation of each matching word embedding based on the attention mechanism to obtain the final dynamic matching word feature embedding. The dynamic weights used in the present invention can better capture the different relevance of characters and vocabulary in different contexts, and can treat each matching word more evenly and accurately. Compared with the static weights adopted by most current Chinese entity recognition methods, dynamic weights can avoid the situation where the weight distribution is too biased towards high-frequency words, and improve the generalization ability and recognition accuracy of the model. The pinyin feature extraction module uses the output of the pinyin extraction module in the Chinese character information extraction part as input, and uses a convolutional neural network to extract pinyin feature embedding. The phenomenon of polyphonic characters is particularly common in Chinese. It is easy to distinguish the different meanings of homophones based on pinyin feature embedding. The component feature extraction module uses the output of the component extraction module in the Chinese character information extraction part as input, and uses a convolutional neural network to extract component feature embedding. Chinese characters are pictographic characters, and the structural information of Chinese characters such as strokes and components contains rich semantics. Embedding component features into the Chinese entity recognition model of the present invention can improve the semantic representation ability of the model and enhance the understanding of the structure of Chinese characters themselves, thereby achieving performance improvement. The dual-path encoder part of the enhanced character embedding and component embedding in the present invention includes two branches, which are respectively used to encode the context features of component embedding and the context features of enhanced character embedding that integrate dynamic matching word features and pinyin features, which helps the model to understand different types of inputs more deeply, and then splices the component context features and enhanced character context features as the dual-path encoder output. In addition, the dual-path encoder can adjust the hyperparameters of the encoder of each branch separately, so as to more accurately control the learning process of the model and improve the model performance. The decoding output part is used to decode the encoder output to obtain the optimal label sequence.

[0028] The model proposed in the present invention introduces pinyin and component information, and uses a two-way encoding structure to effectively integrate character information and vocabulary information, pinyin information, and component information to improve the accuracy of the model. In addition, when introducing vocabulary information, an attention mechanism is used to calculate the correlation between characters and matching words, and each matching word can be treated more evenly at the same time, avoiding the situation where the weight distribution is too biased towards high-frequency words, thereby improving the generalization ability of the model.

[0029] The present invention proposes a Chinese entity recognition technical solution based on vocabulary enhancement and character external information in response to the prior art, which is used to solve the problem that the prior methods fail to fully utilize the external information of Chinese characters, and lack dynamic weight allocation when integrating external dictionary information, resulting in insufficient generalization ability. The present invention is an overall technical solution, including a Chinese character information extraction part, a Chinese character feature extraction part containing dynamic matching word features, a dual-path encoder part for enhanced character embedding and component embedding, and a decoding output part, which improves the semantic representation ability and recognition accuracy of the Chinese entity recognition model.

[0030] Example 2

[0031] The overall structure of the Chinese entity recognition model based on vocabulary enhancement and character external information is the same as in Example 1. In the dynamic matching word feature extraction module, the weighted sum of the embeddings of each matching word is obtained to obtain the final dynamic matching word feature embedding, wherein the dynamic attention weight α of each matching word is obtained by calculating the correlation between the character embedding and each matching word embedding through the attention mechanism, and the calculation formula is as follows:

[0032]

[0033] Where x and y are character embeddings and matching word embeddings, respectively, Q and W K is a learnable parameter, is the scaling factor;

[0034] In order to retain the segmentation information, the present invention divides the matching words into four groups, B, M, E, and S, according to the relationship between the characters and the matching words, wherein in group B, the relationship between the characters and the matching words is that the characters are the first characters of the matching words, in group M, the relationship between the characters and the matching words is that the characters appear in the middle of the matching words, in group E, the relationship between the characters and the matching words is that the characters are the last characters of the matching words, and in group S, the relationship between the characters and the matching words is that the characters and the matching words are equal; the weighted summation of the word embeddings of each group of matching words in the four groups of matching words is performed to obtain the weighted word embeddings of each group of matching words.

[0035]

[0036] in α ω is the attention weight of the matching word ω, eω (ω) is the embedding representation of the matching word ω.

[0037] The final dynamic matching word embedding representation v can be obtained by concatenating the four sets of weighted word embeddings i , and its calculation formula is as follows:

[0038] v i = {v i (B);v i (M); v i (E); v i (S)}

[0039] where v i (B) v i (M), v i (E) v i (S) are the dynamic weighted word embeddings of the four groups of matching words B, M, E, and S respectively.

[0040] The dynamic weights used in the present invention are calculated through an attention mechanism, combining the embedding of characters in a sentence with the correlation of matching words, and can flexibly adjust the weights according to the specific context. Therefore, the Chinese entity recognition model of the present invention can better capture the different correlations between characters and words in different contexts and improve recognition accuracy. In addition, the dynamic weights of the present invention avoid the situation where the weight distribution is too biased towards high-frequency words, thereby improving the generalization ability of the model.

[0041] Example 3

[0042] The present invention is also a Chinese entity recognition method based on vocabulary enhancement and character external information, which is implemented on any of the above-mentioned Chinese entity recognition models based on vocabulary enhancement and character external information. The Chinese entity recognition model based on vocabulary enhancement and character external information is the same as that in Embodiment 1-2. Figure 3 , Figure 3 The flowchart of the present invention includes the following steps:

[0043] (1) Data cleaning: Receive the original Chinese input text, split the long text into several short sentences and remove illegal characters, Unicode characters, etc. to complete data cleaning and obtain the cleaned sentences.

[0044] The present invention divides long text into short sentences during data cleaning, which helps the model focus more on local context and improves processing efficiency and accuracy. Illegal characters and Unicode characters interfere with the learning process of the model, resulting in reduced recognition accuracy, so they also need to be removed.

[0045] (2) Extraction of Chinese character features containing dynamic matching word features: Extract multiple features of the input sentence after data cleaning, including character features, dynamic matching word features, pinyin features, and component features. Extracting character features is to input the cleaned sentence into the BERT model to obtain the feature embedding of each character. To extract dynamic matching word features, each character is first matched with an external dictionary to obtain a matching word. After calculating the dynamic weight of each matching word based on the matching word embedding and the character feature embedding, the weighted sum of each matching word embedding is obtained to obtain the final dynamic matching word feature embedding. The dynamic weight mechanism used in the present invention can more effectively capture the relevance of characters and vocabulary in different contexts. Compared with the static weights used in most current Chinese entity recognition methods, the dynamic weights avoid excessive bias towards high-frequency words and improve the generalization ability and recognition accuracy of the model. The phenomenon of polyphones is particularly common in Chinese. It is easy to distinguish the different meanings of homophones based on pinyin feature embedding. Integrating pinyin feature embedding into the Chinese entity recognition model of the present invention can improve the model's entity recognition ability on texts with polyphones. Extracting pinyin features is to obtain the pinyin of each Chinese character in the input sentence, extract it with a convolutional neural network and obtain pinyin feature embedding. Character feature embedding, dynamic matching word feature embedding, and pinyin feature embedding are sequentially spliced ​​to obtain enhanced character embedding. Chinese characters are pictographic characters, and the structural information such as strokes and components of Chinese characters contains rich semantics. Embedding component features into the Chinese entity recognition model of the present invention can enhance the semantic representation ability of the model and achieve performance improvement. When extracting component features, the present invention first decomposes each Chinese character in the input sentence to obtain the components that make up the Chinese characters, and then uses a convolutional neural network to extract and obtain component feature embedding to complete the extraction of Chinese character features containing dynamic matching word features.

[0046] (3) Dual-path encoding of enhanced character embedding and component embedding: In the present invention, two bidirectional gated recurrent units Bi-GRU are used to extract context features of enhanced character embedding and context features of Chinese character component embedding; character embedding, dynamic matching word feature embedding, and pinyin embedding are spliced ​​to obtain enhanced character embedding and input into a Bi-GRU to extract context representation of enhanced character embedding. The present invention uses two Bi-GRUs to extract context features of enhanced character embedding and context features of Chinese character component embedding, respectively, which helps the model to understand different types of inputs more deeply. In addition, the dual-path encoder can adjust the hyperparameters of each Bi-GRU encoder separately, so as to more accurately control the learning process of the model and improve model performance. The Chinese character component embedding representation is input into another Bi-GRU to extract the context representation of component embedding, and the component context representation and the enhanced character context representation are spliced ​​to complete the dual-path encoding of enhanced character embedding and component embedding.

[0047] See also Figure 2 , Figure 2The structural diagram of the Chinese entity recognition model of the present invention is that the present invention first obtains characters from the input sentence, and obtains matching words, pinyin and components based on the characters. Among them, the components of Chinese characters are extracted from the component extraction module, and the Chinese character component feature embedding is obtained through the convolutional neural network as the input of the first Bi-GRU in the two-way encoding; the matching word of the character is obtained in the matching word extraction module, and the matching word feature extraction module uses the character embedding and the matching word embedding as inputs, and the correlation between the character embedding and the matching word embedding is calculated based on the attention mechanism as the dynamic weight, and the weighted sum of each matching word embedding is obtained to obtain the final dynamic matching word feature embedding; the pinyin of the character is obtained in the pinyin extraction module, and the Chinese character pinyin feature embedding is obtained through the convolutional neural network, and the character embedding, the dynamic matching word embedding, and the pinyin embedding are sequentially spliced ​​to obtain the enhanced character embedding as the input of another Bi-GRU in the two-way encoding; in the Chinese entity recognition model of the present invention, the character embedding is directly used as a part of the enhanced character embedding on the one hand, and on the other hand, it participates in the calculation of the dynamic matching word feature as the input of the matching word extraction module.

[0048] (4) Decoding output: The present invention selects the linear chain conditional random field CRF as the decoder. CRF can effectively model the dependency between labels, thereby improving recognition accuracy. The encoded context representation that integrates Chinese character information, matching word information, pinyin information and component information is input into CRF for decoding. CRF will output the label sequence with the highest score, extract the entities in the sentence based on the label sequence, and complete the decoding output.

[0049] The present invention is directed to the Chinese entity recognition method based on vocabulary enhancement and character external information proposed in the prior art, and is used to solve the problem that the existing methods fail to make full use of the external information of Chinese characters, such as stroke information, pinyin information, component information, etc., and lack dynamic weight allocation when integrating external dictionary information, resulting in insufficient generalization ability. The present invention is an overall technical solution, and the process includes data cleaning, Chinese character feature extraction containing dynamic matching word features, dual-path encoding of enhanced character embedding and component embedding, and decoding output. The present invention uses dynamic weights to weight matching word embedding when extracting matching word features. The dynamic weights better capture the different associations between characters and vocabulary in different contexts, and at the same time treat each matching word more balanced and accurately. Compared with the static weights adopted by most current Chinese entity recognition methods, the dynamic weights used in the present invention avoid the situation where the weight allocation is too biased towards high-frequency words, thereby improving the generalization ability and recognition accuracy of the model. When extracting Chinese character features, the present invention extracts pinyin and component feature embedding. The phenomenon of polyphones is particularly common in Chinese. It is easy to distinguish the different meanings of homophones based on pinyin feature embedding. Integrating pinyin feature embedding into the Chinese entity recognition model of the present invention improves the model's entity recognition ability on texts with polyphones. Chinese characters are pictographic characters. The structural information of Chinese characters such as strokes and components contains rich semantics. Integrating component feature embedding into the Chinese entity recognition model of the present invention improves the model's semantic representation ability and achieves performance improvement. The present invention uses a dual-path encoding structure to extract contextual features for enhanced character embedding and contextual features for Chinese character component embedding. The dual-path encoder can adjust hyperparameters for each Bi-GRU encoder separately, more accurately control the learning process of the model, and improve model performance.

[0050] Example 4

[0051] The Chinese entity recognition model and method based on vocabulary enhancement and character external information are the same as those in Embodiments 1-3. The Chinese character feature extraction in step (2) of the present invention specifically includes:

[0052] 2.1) Character feature extraction: Input the cleaned sentences into the Chinese-BERT-wwm model to obtain the feature embedding of each character. The Chinese-BERT-wwm model has been pre-trained on a large-scale Chinese corpus, has rich language knowledge, and can effectively capture contextual information.

[0053] 2.2) Dynamic matching word feature extraction: see Figure 2, the present invention first matches each character with an external dictionary to obtain the matching words of each character, and then divides the matching words into four groups of B, M, E, and S according to the relationship between the characters and the matching words. The correlation between the character embedding and the embedding of each matching word is calculated based on the attention mechanism as the dynamic weight of the matching word, and then each group of matching words is weighted and summed to obtain the weighted word embedding of each group of matching words, and the four groups of weighted word embeddings are spliced ​​to obtain the final dynamic matching word feature embedding. The dynamic weight flexibly adjusts the matching word weight according to the specific context. The Chinese entity recognition model of the present invention can better capture the different associations between characters and vocabulary in different contexts, and at the same time can treat each matching word more evenly and accurately, avoiding the situation where the weight distribution is too biased towards high-frequency words, and further improving the generalization ability and recognition accuracy of the model.

[0054] 2.3) Pinyin feature extraction: Polyphones are particularly common in Chinese. Based on pinyin feature embedding, it is easy to distinguish the different meanings of homophones. Integrating pinyin feature embedding into the Chinese entity recognition model of the present invention can improve the entity recognition ability of the model on texts with polyphones. The present invention first uses the PyPinyin tool to obtain the pinyin of each Chinese character in the input sentence, and then uses a convolutional neural network to extract and obtain pinyin feature embedding.

[0055] 2.4) Component feature extraction: Chinese characters are pictographic characters. The structural information of Chinese characters such as strokes and components contains rich semantics. Embedding component features into the Chinese entity recognition model of the present invention can enhance the semantic representation ability of the model and achieve performance improvement. The present invention first decomposes each Chinese character in the input sentence to obtain the components that make up the Chinese character, and then uses a convolutional neural network to extract and obtain component feature embedding.

[0056] 2.5) See Figure 5 , Figure 5 This is the architecture diagram of the convolutional neural network used to extract pinyin / component features. The convolutional neural network structures in pinyin feature extraction and component feature extraction are consistent, and both are connected in sequence with a one-dimensional convolution layer, a maximum pooling layer, and a fully connected layer.

[0057] The present invention extracts dynamic matching word features, and the dynamic weights better capture the different associations between characters and words in different contexts, while treating each matching word more evenly and accurately, avoiding the situation where the weight distribution is too biased towards high-frequency words, and improving the generalization ability and recognition accuracy of the model. The present invention also extracts the phonetic features and component features of Chinese characters. The phonetic and component information of Chinese characters contain rich semantics. Integrating the phonetic features and component features into the Chinese entity recognition model of the present invention improves the semantic representation ability of the model and achieves performance improvement.

[0058] Example 5

[0059] The Chinese entity recognition model and method based on vocabulary enhancement and character external information are the same as those in Embodiment 1-4, and the matching word feature extraction described in step 2.2) includes the following steps:

[0060] 2.2.1) Dynamic matching word weight calculation: In step (2), the weighted sum of the matching word embeddings is used to obtain the final matching word feature embedding. The dynamic attention weight α of each matching word is obtained by calculating the correlation between the character embedding and each matching word embedding through the attention mechanism. The calculation formula is as follows:

[0061]

[0062] Where x and y are character embeddings and matching word embeddings, respectively, Q and W K is a learnable parameter, is the scaling factor.

[0063] 2.2.2) Dynamic matching word embedding calculation: In order to retain the segmentation information, the matching words are divided into four groups: B, M, E, and S according to the relationship between the characters and the matching words. In group B, the relationship between the characters and the matching words is that the characters are the first characters of the matching words, in group M, the relationship between the characters and the matching words is that the characters appear in the middle of the matching words, in group E, the relationship between the characters and the matching words is that the characters are the last characters of the matching words, and in group S, the relationship between the characters and the matching words is that the characters and the matching words are equal. The weighted sum of the word embeddings of each group of matching words in the four groups of matching words is obtained to obtain the weighted word embedding of each group of matching words.

[0064]

[0065] in α ω is the attention weight of the matching word ω, e ω (ω) is the embedding representation of the matching word ω.

[0066] The final dynamic matching word embedding representation v can be obtained by concatenating the four sets of weighted word embeddings i , and its calculation formula is as follows:

[0067] v i = {v i (B);v i (M); v i (E); v i (S)}

[0068] where v i (B) v i (M), v i (E) v i (S) are the dynamic weighted word embeddings of the four groups of matching words B, M, E, and S respectively.

[0069] The dynamic weights used in the present invention are calculated through the attention mechanism, combining the embedding of characters in the sentence with the relevance of matching words, and can flexibly adjust the weights according to the specific context. This means that the model can better capture the different relevance of characters and words in different contexts, thereby improving recognition accuracy. In addition, dynamic weights can avoid the situation where weight distribution is too biased towards high-frequency words, thereby improving the generalization ability of the model.

[0070] Example 6

[0071] The Chinese entity recognition model and method based on vocabulary enhancement and character external information are the same as in Examples 1-5. In step 2.4), the components constituting the Chinese characters are obtained based on the Chinese character component dictionary. For most common characters, the dictionary is directly queried to obtain the components constituting the character, which is relatively fast and convenient. If the character cannot be found in the dictionary, it is usually an uncommon character with a low probability of occurrence. In the present invention, a special character ["None"] is used to represent its composition.

[0072] Example 7

[0073] The Chinese entity recognition model and method based on vocabulary enhancement and character external information are the same as in Example 1-6. The two bidirectional gated recurrent units Bi-GRU used in step (3) dual-path encoding are parameter-independent, independently optimized, and do not interfere with each other. The bidirectional gated recurrent unit is a combination of the forward and reverse GRUs. The two are consistent in network structure, except that the input sequence directions are opposite. The implementation of the forward network is:

[0074] z t =σ(W z ·[h t-1 ,x t ])

[0075] r t =σ(W r ·[h t-1 ,x t ])

[0076]

[0077] Where σ represents the sigmoid function, tanh is the hyperbolic tangent function, r and z represent the reset gate and update gate respectively, and h t represents the hidden state at time step t, is the candidate hidden state, W r , W z , W is a trainable weight parameter; the bidirectional hidden state of Bi-GRU at character i is the concatenation of the forward hidden state and the backward hidden state: is the forward hidden state, Backward hidden state.

[0078] The present invention adopts a new dual-path encoding fusion strategy, using two parallel modules to process enhanced character representation and Chinese character component embedding respectively during context encoding, optimizing the boundary recognition effect of Chinese named entity recognition and effectively improving the recognition accuracy of the model in complex text environments. The dual-path encoder in the present invention can adjust the hyperparameters for each Bi-GRU encoder separately, and more accurately control the learning process of the model.

[0079] The present invention integrates the pinyin, components and vocabulary information into the character feature extraction process. It also adopts a new fusion strategy, using two parallel modules to process enhanced character representation and Chinese character component embedding respectively, which can well combine the Chinese character pinyin information and Chinese character component information with vocabulary information. It further enriches the feature representation of characters and achieves performance improvement.

[0080] Example 8

[0081] The Chinese entity recognition model and method based on vocabulary enhancement and character external information are the same as Examples 1-7. The present invention designs a dual-stream model, adopts a new dual-path encoding fusion strategy, and uses two parallel modules to respectively process enhanced character representation and Chinese character component embedding, so as to effectively integrate the rich external information of Chinese characters (the external information of Chinese characters usually refers to additional information that is not inside a single Chinese character but is crucial to understanding and processing Chinese characters) to further improve the Chinese named entity model based on vocabulary enhancement and improve the accuracy of named entity recognition.

[0082] To achieve the above object, the technical solution of the present invention comprises the following steps:

[0083] Step 1) Preprocess the input text data set, including removing redundant characters and special symbols. Then, the overly long text data is segmented into reasonable sentence units through the intelligent sentence segmentation algorithm to ensure the efficient processing capability of the model;

[0084] Step 2) This step aims to extract various features of the input sentence, including character features, vocabulary features, pinyin features, component features, etc.;

[0085] Step 2.1) The cleaned sentence s = {c1, c2, c3, ..., c n} Input to the BERT model to obtain each character c i Embedded representation of ;

[0086] Step 2.2) Perform string matching on each character and the external dictionary to obtain the matching words of each character. Each character may have multiple matching words, and a weight needs to be assigned to each matching word. The previous method directly uses the frequency of the matching word as its weight. In order to capture the relationship between the character and its matching word and more reasonably assign the weight of each matching word, the present invention calculates the attention weight of the character and its matching words based on the attention mechanism:

[0087]

[0088] Where x and y are character embeddings and matching word embeddings, respectively, Q and W K is a learnable parameter, is the scaling factor.

[0089] In order to preserve the segmentation information, the matching words are divided into four groups according to the relationship between the character and the matching word: B (the character is the first character of the matching word), M (the character appears in the middle of the matching word), E (the character is the last character of the matching word), and S (the character is equal to the matching word). The weighted word embedding of each group of matching words in the four groups is summed up to obtain the weighted word embedding of each group of matching words.

[0090]

[0091] in α ω is the attention weight of the matching word ω, e ω (ω) is the embedding representation of the matching word ω.

[0092] The final dynamic matching word embedding representation v can be obtained by concatenating the four sets of weighted word embeddings i , and its calculation formula is as follows:

[0093] v i = {v i (B);v i (M); v i (E); v i (S)}

[0094] where v i (B) v i (M), v i (E) v i (S) are the dynamic weighted word embeddings of the four groups of matching words B, M, E, and S respectively.

[0095] Step 2.3) Obtain the pinyin of each Chinese character in the input sentence, and use a convolutional neural network to extract pinyin features to obtain pinyin feature embedding;

[0096] Step 2.4) Decompose each Chinese character in the input sentence to obtain the components that make up the Chinese character, and use a convolutional neural network to extract the features of the components to obtain component embedding;

[0097] Step 3) Concatenate character embedding, vocabulary embedding, and pinyin embedding to obtain enhanced character representation:

[0098] x i ←[x i ;v i ;e p ]

[0099] where x i is the character embedding representation, e p is the pinyin embedding representation;

[0100] Step 4) using a dual bidirectional gated recurrent unit (Bi-GRU) to extract contextual features for enhanced character representation and contextual features for embedding Chinese character components; the two Bi-GRU parameters are independent and do not interfere with each other;

[0101] Step 4.1) Concatenate the character embedding, vocabulary embedding, and pinyin embedding to obtain the enhanced character representation and input it into a Bi-GRU to further extract the contextual representation of the character embedding; the bidirectional gated recurrent unit can be understood as a combination of the forward and reverse GRUs, which have the same network structure, except that the input sequence directions are opposite. The implementation of the forward network is:

[0102] z t =σ(W z ·[h t-1 ,x t ])

[0103] r t =σ(W r ·[h t-1 ,x t ])

[0104]

[0105] Where σ represents the sigmoid function, tanh is the hyperbolic tangent function, r and z represent the reset gate and update gate respectively, and h t represents the hidden state at time step t, is the candidate hidden state, W r , W z , W is a trainable weight parameter; the bidirectional hidden state of Bi-GRU at character i is the concatenation of the forward hidden state and the backward hidden state: is the forward hidden state, Backward hidden state; the Bi-GRU output of this step can be represented as a set of hidden vectors

[0106] Step 4.2) Input the Chinese character component embedding representation into another Bi-GRU to further extract the contextual representation of the component embedding; the Bi-GRU output of this step can be represented as a hidden vector set

[0107] Step 5) The present invention selects a linear chain conditional random field (CRF) as a decoder, and concatenates the outputs of the Bi-GRU in step 4.1) and step 4.2) to obtain S l Input to CRF, CRF will output entity label sequence y = [l1,l2,...,l n ]'s score:

[0108]

[0109] Where y′ represents all possible label sequences, Is specific to l i Learnable parameters, From the label l i-1 To i The offset of

[0110] The present invention uses the Viterbi algorithm to search for the tag sequence with the highest score, and entities in the sentence can be extracted based on the tag sequence.

[0111] The model and the method are combined to give an example to further illustrate the present invention.

[0112] Example 9

[0113] The Chinese entity recognition model and method based on vocabulary enhancement and character external information are the same as those in Examples 1-3. The specific embodiments and effects of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0114] Reference Figure 1 , the implementation steps of this example are as follows:

[0115] Step 1: Data cleaning.

[0116] The input text may be irregular and may contain irregular characters, Unicode characters, etc., which need to be cleared. In addition, some input text may be too long. BERT has a limit on the length of input text, and too long text is not conducive to Bi-GRU to extract context information. Therefore, the too long input text needs to be split into several short sentences.

[0117] Step 2: Extract the features of the input sentence.

[0118] This step aims to extract multiple features of the input sentence, including character features, vocabulary features, pinyin features, component features, etc. The Chinese named entity recognition method proposed in the present invention is a character-based named entity recognition method. For the sentence cleaned in step 1, its character embedding, matching word embedding, pinyin embedding, and Chinese character component embedding can be obtained based on each character in the sentence.

[0119] The specific implementation of this step is as follows:

[0120] 2.1) Extracting character embeddings: There are many ways to extract character embeddings for the cleaned sentences. The present invention uses the BERT model to extract character embeddings. BERT is based on the Transformer architecture, uses the masked language model and next sentence prediction as pre-training tasks, and is pre-trained with a large amount of data to generate high-quality language representations. n} Input into the BERT model to obtain each character c i Embedded representation of ;

[0121] 2.2) Extract vocabulary embedding: Perform string matching on each character in the sentence and the external dictionary to obtain the matching words of each character. Each character may have multiple matching words, and a weight needs to be assigned to each matching word. Most previous methods directly use the frequency of matching words as their weights. In order to capture the relationship between characters and their matching words and more reasonably assign weights to each matching word, this paper calculates the attention weights of characters and their matching words based on the attention mechanism:

[0122]

[0123] Where x and y are character embeddings and matching word embeddings, respectively, Q and W K is a learnable parameter, is the scaling factor.

[0124] In order to preserve the segmentation information, the matching words are divided into four groups according to the relationship between the character and its matching word: B (the character is the first character of the matching word), M (the character appears in the middle of the matching word), E (the character is the last character of the matching word), and S (the matching word is a single character). The weighted word embedding of each group of matching words in the four groups is summed up to obtain the weighted word embedding of each group of matching words.

[0125]

[0126] in α ω is the attention weight of the matching word ω, e ω (ω) is the embedding representation of the matching word ω.

[0127] Concatenating the four groups of weighted word embeddings can obtain the final dynamic matching word embedding representation v i , and its calculation formula is as follows:

[0128] v i = {v i (B); v i (M); v i (E); v i}

[0129] where v i (B), v i (M), v i (E), v i (S) are the dynamic weighted word embeddings of the four groups of matching words B, M, E, and S respectively.

[0130] 2.3) Extracting pinyin embeddings: First, use the PyPinyin tool to extract the pinyin of each character in the sentence. For example, for the character "中", the PyPinyin output is ["z", "h", "o", "n", "g", "1"], where "1" represents the first tone. Then, input it into a convolutional neural network to extract pinyin embeddings. Refer to Figure 3 , this convolutional neural network includes a one-dimensional convolutional layer, a max-pooling layer, and a fully connected layer.

[0131] 2.4) Extracting Chinese character component embeddings: The present invention constructs a Chinese character component dictionary. For the vast majority of commonly used characters, the components that make up the character can be directly queried from the dictionary. For example, the components of "亮" are ["亠", "口", "冖", "儿"]. Similarly, input it into a convolutional neural network to extract Chinese character component embeddings, and the structure of this convolutional neural network is the same as the convolutional neural network mentioned in 2.3).

[0132] Step 3, obtaining enhanced character embeddings.

[0133] Concatenating the character embeddings, lexical embeddings, and pinyin embeddings to obtain an enhanced character representation:

[0134] x i ← [x i ; v i ; e p

[0135] where x i is the character embedding representation, and e p is the pinyin embedding representation.

[0136] Step 4, extracting text context features.

[0137] ​The present invention proposes a two-stream model to fuse multiple external information of Chinese characters, and selects two bidirectional gated recurrent units (Bi-GRU) as encoders to extract contextual features for enhanced character representation and component embedding respectively. Bi-GRU can capture both forward and backward information in the sequence at the same time, which means that the output at each time point is based on the context of the entire sequence, which is crucial for named entity recognition. Bi-GRU can be flexibly applied to different sequence lengths and effectively handle long-term dependencies in long sequences.

[0138] 4.1) Concatenate character embedding, vocabulary embedding, and pinyin embedding to obtain enhanced character representation and input it into a Bi-GRU to further extract the contextual representation of the character embedding; the bidirectional gated recurrent unit can be understood as a combination of forward and reverse GRUs. The two have the same network structure, but the input sequence direction is opposite. The implementation of the forward network is:

[0139] z t =σ(W z ·[h t-1 ,x t ])

[0140] r t =σ(W r ·[h t-1 ,x t ])

[0141]

[0142] Where σ represents the sigmoid function, z and r represent the reset gate and update gate respectively, and W is a trainable parameter; the bidirectional hidden state of Bi-GRU at character i can be expressed as the concatenation of the forward hidden state and the backward hidden state: The Bi-GRU output of this step can be expressed as a set of hidden vectors

[0143] 4.2) Input the Chinese character component embedding representation into another Bi-GRU to further extract the contextual representation of the component embedding; the Bi-GRU output of this step can be represented as a hidden vector set

[0144] Step 5: CRF decoding to get output.

[0145] The linear chain conditional random field (CRF) can capture the global information of the entire sequence and ensure that the annotation of the entire sequence is optimal. This means that CRF not only considers the optimal annotation of a single position, but also considers the optimal annotation combination of the entire sequence, which is crucial for the NER task; after splicing the outputs of the two Bi-GRU modules in step 4, we can get S lInput to the CRF model, CRF will output the entity label sequence y = [l1,l2,...,l n ]'s score:

[0146]

[0147] Where y′ represents all possible label sequences, is specific to l i Learnable parameters, From the label l i-1 To i The present invention uses the Viterbi algorithm to search for the tag sequence with the highest score, and the entities in the sentence can be extracted based on the tag sequence.

[0148] The present invention integrates pinyin, components and vocabulary information into the character feature extraction process. A new fusion strategy is adopted, and two parallel modules are used to process enhanced character representation and Chinese character component embedding respectively, which can well combine Chinese character pinyin information and Chinese character component information with vocabulary information, further enrich the feature representation of characters and achieve performance improvement. Its implementation scheme is: cleaning the input text, removing invalid characters and segmenting overlong text. Extracting multiple features of the input sentence, including character features, vocabulary features, pinyin features, and component features. First, the BERT model is used to extract the embedding of each character in the input sentence. Each character in the input sentence is matched with an external dictionary to obtain the matching words of each character, and the matching words are weighted and summed based on the attention mechanism to obtain the vocabulary embedding. After using the PyPinyin tool to obtain the pinyin of each character in the input sentence, it is sent to the convolutional neural network to extract the pinyin embedding. After obtaining the components of each character based on the constructed Chinese character component dictionary, it is sent to the convolutional neural network to extract the Chinese character component embedding. The above character features, pinyin features, and vocabulary features are concatenated to obtain enhanced character embedding, which is input into a Bi-GRU model to extract character context features; the component embedding is input into another Bi-GRU model to extract component context features; the outputs of the two Bi-GRUs are concatenated and input into the linear chain conditional random field model for decoding to obtain the output entity label sequence.

[0149] The following is a verification explanation of the technical effect of the present invention through simulation.

[0150] Example 10

[0151] The Chinese entity recognition model and method based on vocabulary enhancement and character external information are the same as those in Examples 1-9.

[0152] Simulation conditions

[0153] The experimental data used are Weibo dataset, OntoNotes dataset, and Resume dataset. The simulation platform is: IntelCore i5-12400 CPU, 16.0GB memory, NVIDIA GeForce RTX 3090Ti graphics card, Windows 11 operating system.

[0154] Simulation content

[0155] The above three data sets are tested respectively by using the present invention and the existing SoftLexicon algorithm, TENER algorithm, FLAT algorithm and MECT algorithm, and the F1 score index is used to evaluate the performance of each. The results are shown in Table 1, which is a comparison table of simulation results of the present invention and the prior art.

[0156] Simulation Results and Analysis

[0157] See Table 1, where the present invention improves the F1 index by 2.58% compared with the baseline model SoftLexicon on the Weibo dataset; on the OntoNotes dataset, the present invention improves the F1 score to 83.32%; and on the Resume dataset, the best result is also achieved, with the F1 score improved by 0.15% compared with the baseline model SoftLexicon. The simulation results show that on the more difficult Weibo dataset, the F1 score performance of the method proposed by the present invention is significantly higher than that of other methods, proving the adaptability of the present invention in complex text environments. The simulation results show that the results of the present invention on the three datasets are better than the existing methods.

[0158] Table 1 Comparison of simulation results between the present invention and the prior art

[0159] Weibo OntoNotes Resume SoftLexicon 70.50 82.81 96.11 TENER 58.39 72.82 95.25 FLAT 68.55 81.82 95.86 MECT 70.43 82.57 95.98 The present invention 73.08 83.32 96.26

[0160] Embodiment 11

[0161] The Chinese entity recognition model and method based on vocabulary enhancement and character external information are the same as those in Examples 1-9, and the simulation conditions are the same as those in Example 10. In this example, only the Weibo dataset is used as the experimental dataset.

[0162] Simulation content

[0163] In order to verify the contribution of each module in the model of the present invention, the present invention conducted an ablation experiment on the Weibo dataset, and used precision, recall, and F1 score indicators to evaluate their respective performances. The results are shown in Table 2, which is the ablation experiment results of the present invention.

[0164] Simulation Results and Analysis

[0165] In order to verify the contribution of each module in the model of the present invention, the present invention performs an ablation experiment on the Weibo dataset. The ablation experiment results are shown in Table 2.

[0166] Table 2 Ablation experimental results of the present invention

[0167] Model Accuracy Recall F1 score Complete Model 73.73 72.46 73.08 Remove pinyin information 71.06 72.95 71.99 Remove component information 73.42 70.04 71.69 Remove components and pinyin information at the same time 72.82 68.60 70.65 Remove dynamic weights 73.27 71.50 72.37

[0168] When the pinyin embedding is removed, its F1 score drops to 71.99, which is 1.09% lower than the complete model. The present invention removes component information to evaluate the contribution of component information. After removing the component information, the F1 score of the model on the Weibo dataset dropped by 1.39%. In addition, when the pinyin and component information are deleted, the F1 score further drops to 70.65, which is 2.43% lower than the complete model. The above results also prove that the various external information of the present invention complement each other to further improve the model performance. In this example, the dynamic weight used in the present invention is cancelled, and the word frequency is used as the weight of the matching word like SoftLexicon to evaluate the importance of the dynamic weight. The experimental results show that this causes the F1 score to drop by 0.71%, verifying that the dynamic weight used in the present invention can effectively improve the model accuracy and generalization ability.

[0169] In summary, the Chinese entity recognition model and method based on vocabulary enhancement and character external information proposed in the present invention integrates pinyin, components and vocabulary information into the character feature extraction process. A new fusion strategy is adopted, and two parallel modules are used to process the enhanced character representation and Chinese character component embedding respectively, which can well combine the Chinese character pinyin information and Chinese character component information with the vocabulary information. The feature representation of the character is further enriched and the performance improvement is achieved. The problem that the existing method fails to make full use of the external information of Chinese characters and the problem that the static weight of the matching word is determined only by the word frequency in the existing method is solved. The Chinese entity recognition model is sequentially connected with a Chinese character information extraction part, a Chinese character feature extraction part containing dynamic matching word features, a dual-path encoder part for enhancing character embedding and component embedding, and a decoding output part. The implementation method includes data cleaning, Chinese character feature extraction containing dynamic matching word features, and dual-path encoding and decoding output for enhancing character embedding and component embedding. The present invention proposes a dual-path encoding structure, which effectively integrates the vocabulary information, pinyin information and component information of Chinese characters into the model, greatly enriches the semantic expression of character-based Chinese named entity recognition, and solves the problem that the existing method fails to make full use of the external information of Chinese characters. In addition, the present invention adopts an attention mechanism when introducing vocabulary information, and dynamically adjusts the weight of matching words by calculating the correlation between characters and matching words. Compared with the existing method that only relies on word frequency to determine the static weight of matching words, the method of the present invention is more flexible and reasonable in the allocation of matching word weights, and can treat each matching word more evenly, avoiding the situation where the weight allocation is too biased towards high-frequency words, thereby improving the generalization ability and recognition accuracy of the model. It is used to extract Chinese entities from unstructured input text.

Claims

1. A Chinese entity recognition model and method based on vocabulary enhancement and character external information, characterized in that: A Chinese character information extraction part, a Chinese character feature extraction part containing dynamic matching word features, a dual-path encoder part for enhancing character embedding and component embedding, and a decoding output part are sequentially connected; wherein the Chinese character information extraction part is provided with a character extraction module, and based on the character extraction module, a matching word extraction module, a pinyin extraction module, and a component extraction module are also provided; the Chinese character feature extraction part containing dynamic matching word features includes a character feature extraction module, a dynamic matching word feature extraction module, a pinyin feature extraction module, and a component feature extraction module, wherein the character feature extraction module uses the output of the character extraction module in the Chinese character information extraction part as input to obtain character embedding; the dynamic matching word feature extraction module first uses the output of the matching word extraction module in the Chinese character information extraction part and the output of the character extraction module as input, and then performs weighted summation of each matching word embedding based on the attention mechanism to obtain the dynamic matching word feature embedding; Pinyin The feature extraction module uses the output of the pinyin extraction module in the Chinese character information extraction part as input to extract pinyin feature embedding; The component feature extraction module uses the output of the component extraction module in the Chinese character information extraction part as input to extract component feature embedding; the dual-path encoder part of the enhanced character embedding and component embedding includes two branches, which are respectively used to encode the context features of component embedding and the context features of enhanced character embedding that integrates dynamic matching word features and pinyin features, and then the component context features and the enhanced character context features are spliced ​​as the dual-path encoder output; the decoding output part is used to decode the encoder output to obtain the optimal label sequence.

2. The Chinese entity recognition model based on vocabulary enhancement and character external information according to claim 1, characterized in that: When the dynamic matching word feature extraction module performs weighted summation of the embeddings of each matching word to obtain the dynamic matching word feature embedding, the dynamic attention weight α of each matching word is obtained by calculating the correlation between the character embedding and the embedding of each matching word through the attention mechanism, and the calculation formula is as follows: Where x and y are character embeddings and matching word embeddings, respectively, Q and W K is a learnable parameter, is the scaling factor; In order to retain the segmentation information, the matching words are divided into four groups, B, M, E, and S, according to the relationship between the characters and the matching words. In group B, the relationship between the characters and the matching words is that the characters are the first characters of the matching words, in group M, the relationship between the characters and the matching words is that the characters appear in the middle of the matching words, in group E, the relationship between the characters and the matching words is that the characters are the last characters of the matching words, and in group S, the relationship between the characters and the matching words is that the characters and the matching words are equal. The weighted word embeddings of each group of matching words in the four groups of matching words are summed up to obtain the weighted word embeddings of each group of matching words in α ω is the attention weight of the matching word ω, e ω (ω) is the embedding representation of the matching word ω; The final dynamic matching word embedding representation v can be obtained by concatenating the four sets of weighted word embeddings i , and its calculation formula is as follows: v i ={v i (B);v i (M);v i (E);v i (S)}; where v i (B) v i (M), v i (E) v i (S) are the dynamic weighted word embeddings of the four groups of matching words B, M, E, and S respectively.

3. A Chinese entity recognition method based on vocabulary enhancement and character external information is implemented on any of the Chinese named entity recognition models based on vocabulary enhancement and character external information described in claims 1-2, characterized in that: The steps include: (1) Data cleaning: Receive the original Chinese input text, split the long text into several short sentences and remove illegal characters and Unicode characters, complete data cleaning, and obtain the cleaned sentences; (2) Extraction of Chinese character features containing dynamic matching word features: Extract multiple features of the input sentence after data cleaning, including character features, dynamic matching word features, pinyin features, and component features; To extract character features, the cleaned sentences are input into the BERT model to obtain the feature embedding of each character. To extract dynamic matching word features, each character is first matched with an external dictionary to obtain matching words. After calculating the dynamic weight of each matching word based on the matching word embedding and character feature embedding, the weighted sum of each matching word embedding is performed to obtain the dynamic matching word feature embedding. extract Pinyin feature is to obtain the pinyin of each Chinese character in the input sentence, extract it with convolutional neural network and get pinyin feature embedding; character feature embedding, dynamic matching word feature embedding and pinyin feature embedding are sequentially spliced ​​to get enhanced character embedding; when extracting component features, first decompose each Chinese character in the input sentence to obtain the components that make up the Chinese character, then use convolutional neural network to extract and get component feature embedding, and complete the extraction of Chinese character features containing dynamic matching word features; (3) Dual-path encoding of enhanced character embedding and component embedding: Two bidirectional gated recurrent units (Bi-GRUs) are used to extract contextual features of enhanced character embedding and contextual features of Chinese character component embedding; Concatenate character embedding, dynamic matching word feature embedding, and pinyin embedding to obtain enhanced character embedding and input it into a Bi-GRU to extract the context representation of the enhanced character embedding; The Chinese character component embedding representation is input into another Bi-GRU to extract the context representation of the component embedding, and the component context representation and the enhanced character context representation are concatenated to complete the dual-path encoding of enhanced character embedding and component embedding; (4) Decoding output: Select the linear chain conditional random field (CRF) as the decoder, and input the encoded context representation that integrates Chinese character information, matching word information, pinyin information, and component information into the CRF for decoding. The CRF will output the label sequence with the highest score, extract the entities in the sentence based on the label sequence, and complete the decoding output.

4. The Chinese entity recognition method based on vocabulary enhancement and character external information according to claim 3 is characterized in that: The Chinese character feature extraction in step (2) specifically includes: 2.1) Character feature extraction: Input the cleaned sentences into the Chinese-BERT-wwm model to obtain the feature embedding of each character; 2.2) Dynamic matching word feature extraction: First, string match each character with the external dictionary to obtain the matching word of each character. Then, according to the relationship between the character and the matching word, the matching words are divided into four groups: B, M, E, and S. Based on the attention mechanism, the correlation between the character embedding and the embedding of each matching word is calculated as the dynamic weight of the matching word. Then, each group of matching words is weighted and summed to obtain the weighted word embedding of each group of matching words. The four groups of weighted word embeddings are spliced ​​to obtain the final dynamic matching word feature embedding. 2.3) Pinyin feature extraction: Use the PyPinyin tool to obtain the pinyin of each Chinese character in the input sentence, and use a convolutional neural network to extract and obtain the pinyin feature embedding; 2.4) Component feature extraction: Decompose each Chinese character in the input sentence to obtain the components that make up the Chinese character, and use a convolutional neural network to extract and embed the component features; 2.5) The convolutional neural network structures in pinyin feature extraction and component feature extraction are consistent, both of which are connected in sequence with a one-dimensional convolution layer, a maximum pooling layer and a fully connected layer.

5. The Chinese entity recognition method based on vocabulary enhancement and character external information according to claim 4 is characterized in that: The dynamic matching word feature extraction described in step 2.2) includes the following steps: 2.2.1) Dynamic matching word weight calculation: The dynamic attention weight α of each matching word is obtained by calculating the correlation between the character embedding and each matching word embedding through the attention mechanism. The calculation formula is as follows: Where x and y are character embeddings and matching word embeddings, respectively, Q and W K is a learnable parameter, is the scaling factor; 2.2.2) Dynamic matching word embedding calculation: In order to retain the segmentation information, the matching words are divided into four groups: B, M, E, and S according to the relationship between the characters and the matching words. In group B, the relationship between the characters and the matching words is that the characters are the first characters of the matching words, in group M, the relationship between the characters and the matching words is that the characters appear in the middle of the matching words, in group E, the relationship between the characters and the matching words is that the characters are the last characters of the matching words, and in group S, the relationship between the characters and the matching words is that the characters and the matching words are equal; The weighted word embeddings of each group of matching words in the four groups of matching words are summed up to obtain the weighted word embeddings of each group of matching words in α ω is the attention weight of the matching word ω, e ω (ω) is the embedding representation of the matching word ω; The final dynamic matching word embedding representation v can be obtained by concatenating the four sets of weighted word embeddings i , and its calculation formula is as follows: v i ={v i (B);v i (M);v i (E);v i (S)}; where v i (B) v i (M), v i (E) v i (S) are the dynamic weighted word embeddings of the four groups of matching words B, M, E, and S respectively.

6. The Chinese entity recognition method based on vocabulary enhancement and character external information according to claim 4 is characterized in that: The components constituting the Chinese characters in step 2.4) are obtained based on a Chinese character component dictionary. For most common characters, the components constituting the character are directly queried in the dictionary. If the character cannot be found in the dictionary, a special character ["None"] is used to represent its components.

7. The Chinese entity recognition method based on vocabulary enhancement and character external information according to claim 3 is characterized in that: The two bidirectional gated recurrent units Bi-GRU used in step (3) dual-path encoding have independent parameters, are independently optimized, and do not interfere with each other; the bidirectional gated recurrent unit is a combination of the forward and reverse GRUs, which have the same network structure except that the input sequence directions are opposite. The forward network is implemented as follows: z t =σ(W z ·[h t-1 ,x t ]) r t =σ(W r ·[h t-1 ,x t ]) Where σ represents the sigmoid function, tanh is the hyperbolic tangent function, r and z represent the reset gate and update gate respectively. h t represents the hidden state at time step t, is the candidate hidden state, W r , W z , W is a trainable weight parameter; the bidirectional hidden state of Bi-GRU at character i is the concatenation of the forward hidden state and the backward hidden state: is the forward hidden state, Backward hidden state.

Images