Chinese named entity recognition method based on Chinese character multi-feature fusion dictionary information

Abstract

The invention provides a Chinese named entity recognition method based on Chinese character multi-feature fusion dictionary information, and the method comprises the steps: based on Chinese character, font and pinyin features, fusing dictionary information, constructing three character and word sequences, obtaining character embedding vectors of characters and words through table look-up, splitting the characters into basic parts, and splitting the words into synthesis parts of the characters. The method comprises the following steps of: extracting corresponding font embedding vectors by utilizing a convolutional neural network, converting characters into pinyin, converting words into initial and final consonants of the characters, extracting corresponding pinyin embedding vectors by utilizing the convolutional neural network, finally performing linear mapping on the characters, the font and the pinyin embedding vectors, and fusing relative position codes in corresponding character and word sequences. The method comprises the following steps: obtaining final character, font and pinyin feature vectors through an improved Transform model, splicing the feature vectors, carrying out linear mapping to obtain a fusion feature vector, and predicting an entity label type through a conditional random field. According to the scheme, the accuracy and efficiency of Chinese named entity recognition can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and particularly to a Chinese named entity recognition method based on the fusion of multi-feature dictionary information of Chinese characters. Background Art

[0002] Named Entity Recognition (NER) is to extract specific types of entities such as people, locations, and proper names from unstructured text. NER provides basic information for some subsequent tasks of natural language processing, such as knowledge graphs, information retrieval, question answering systems, etc. Currently, named entity recognition technology has been widely applied in fields such as social media and healthcare.

[0003] Chinese has its unique language characteristics. Compared with English which uses spaces as word delimiters, Chinese text sequences contain words with semantic information but lack clear word boundaries. By means of word segmentation, word information is introduced, but incorrect word segmentation will cause error propagation. For example, the place name entity "Xi'an North Road" is segmented into "Xi'an" and "North Road", resulting in different semantics.

[0004] In the prior art, some people have proposed Lattice LSTM that introduces lexical information, uses the Long Short-Term Memory network as the model backbone, and utilizes the directed acyclic Lattice structure to fuse word and character information and avoid word segmentation errors; some people have also proposed the FLAT (Flat-LAttice Transformer) model, which designs relative position information based on Transformer to dynamically fuse the Lattice structure; some people have also proposed the MECT (Multi-metadata Embedding based Cross-Transformer) model based on radicals and semantics, which utilizes the multi-source data features of words and radicals and further fuses them through two independent streams of Transformers to enhance semantic understanding.

[0005] The above prior methods focus on capturing partial features of words and characters, have insufficient understanding of low-frequency words and out-of-vocabulary words, lack in-depth mining of multi-dimensional features of Chinese characters such as characters, glyphs, and pinyin, do not make full use of prior knowledge, and lack in-depth fusion of multi-dimensional features of Chinese characters and dictionary information, resulting in insufficient understanding of semantic features by the model. Summary of the Invention

[0006] In view of the above problems, the present invention proposes a Chinese named entity recognition method based on the fusion of multi-feature dictionary information of Chinese characters. By introducing prior knowledge, mining multi-dimensional features of Chinese characters such as characters, glyphs, and pinyin, fusing dictionary information, and performing the recognition of Chinese named entities based on a neural network model.

[0007] A method for Chinese named entity recognition based on multi-feature fusion dictionary information of Chinese characters, the method comprising:

[0008] Step S1, given the input text sequence ,in, express The Middle Using a single character and an open Chinese dictionary L as the dictionary, a prefix tree with a tree structure is constructed.

[0009] Step S2: Traverse the input sequence using the word prefix tree. All character subsequences in the text are matched against dictionary L to find all possible matching words, forming a matching word sequence. This matching word sequence is then concatenated into the corresponding text sequence. Then, a sequence of word combinations is formed. ;

[0010] Step S3, obtain by looking up the embedded table. The character embedding representation of the character sequence and the word embedding representation of the word sequence are obtained by concatenating the character embedding representation and the word embedding representation. Word embedding features ;

[0011] Step S4: Decompose the characters in the character sequence using basic component glyphs to obtain the characters. glyph sequence The words in the word sequence are decomposed using the glyphs of the compound components to obtain the words. glyph sequence ;

[0012] Step S5: Input the glyph sequences of the character sequence and word sequence into the first convolutional neural network CNN1, which then passes through convolutional layers, max pooling layers, and finally fully connected layers to transform them, obtaining glyph embedding representations of the character sequence and word sequence. These representations are then concatenated to obtain... glyph embedding features ;

[0013] Step S6: Obtain the input sequence using the phonetic lookup table. The pinyin and tones of Chinese characters are determined by using initial consonants, final vowels, and tones to form the character sequence. Pinyin sequence For word sequences, the initial consonant, final vowel, and tone are used to obtain the word's... Pinyin sequence ;

[0014] Step S7: Input the pinyin sequences of the character sequence and word sequence into the second convolutional neural network CNN2 to obtain the pinyin embedding representations of the character sequence and word sequence. Then, concatenate the pinyin embedding representations of the character sequence and word sequence to obtain... Pinyin embedding features ;

[0015] Step S8, sequence All characters and words in the text are used as spans and numbered sequentially to form a sequence. Then, the Transformer encoder is used to encode the final fused features;

[0016] Step S9: Input the final fused features into the Conditional Random Field (CRF) decoder for sequence labeling.

[0017] Furthermore, the Transformer encoder consists of multiple encoder layers, each of which includes a self-attention sublayer and a feedforward neural network sublayer.

[0018] Furthermore, Represented as ,in, express The Matching words, sequence Represented as ,in, and They represent The renumbered number The and the first A single character or word can also indicate a range.

[0019] Furthermore, the working principle of each encoder layer is as follows:

[0020] Positional encoding is added to each span (character / word), based on the head and tail position indices. The head and tail position indices of a character are the same. and Indicates span The position of the head and the position of the tail. and Indicates span Based on the head and tail positions, calculate the four relative distances, as shown below:

[0021] ;

[0022] ;

[0023] ;

[0024] ;

[0025] in, Indicates span Head position and span The distance between the head positions, Indicates span Head position and span The distance between the tail positions, Indicates span Tail position and span The distance between the head positions, Indicates span Tail position and span The distance between the tail positions;

[0026] The four relative position distances are encoded as follows:

[0027] ;

[0028] ;

[0029] in, , A vector representing relative position. and An index representing the location encoding dimension. represent , , , The four relative positions shown are × , Indicates the number of heads that attract attention. It is the dimension of each attention head;

[0030] By integrating four positional codes, the span can be calculated. and Relative position encoding between :

[0031] ;

[0032] in, Indicates span Head position and span The relative positional distance encoding between the head positions, Indicates span Tail position and span The relative positional distance encoding between the head positions, Indicates span Head position and span The relative positional distance encoding between the tail positions, Indicates span Tail position and span The relative position encoding between the tail positions, It is a learnable parameter matrix. This represents the activation function, and ⊕ represents the concatenation operation;

[0033] Then embed the words of sequence q. , character embedding and Pinyin embedding The inputs are fed into the improved Transformer, mapped to query, key, and value matrices, and their respective computations are performed. , , The value is represented as follows:

[0034] ;

[0035] in, It is a parameter matrix. represent , , Embedded features;

[0036] Then based on span and The relative positional encoding between them is used to calculate improved attention weights using a self-attention mechanism. and weighted value The calculation method is as follows:

[0037] ;

[0038] ;

[0039] in, For query vector, For key vectors, For value vectors, For span and Relative position encoding between them It is a learnable relative position encoding mapping matrix. It is a learnable key-value vector mapping matrix. It is the parameter matrix after relative position encoding mapping. This represents the matrix transpose operation. It is a normalized exponential function;

[0040] Then the output of the self-attention layer Send to feedforward neural network Layers are used to obtain sequence features. :

[0041] ;

[0042] in, These are the learnable weights of two linear transformations. These are the biases of the two linear transformations, respectively. represent Word sequence features Character sequence features and Pinyin sequence features ;

[0043] Finally, the three sequence features are concatenated and then linearly transformed to obtain the final fused features. :

[0044] ;

[0045] in, , , They are respectively The features of words, characters, and phonetic sequences, These are the learnable weights of a linear transformation. It is the bias of the linear transformation.

[0046] Furthermore, step S9 further includes:

[0047] For sequences The predicted label sequence is The probability of the output predicted sequence y is calculated as follows: :

[0048] ;

[0049] in, Let be the state transition characteristic function of a conditional random field, and calculate the label at the (i-1)th position in the sequence s. Move to the i-th position label The probability, Let be the state feature function, and calculate the label at the i-th position in sequence s. The probability, These are learnable weight parameters;

[0050] Using the Viterbi algorithm, the highest probability label sequence is found through recursion, which is the most likely label sequence to appear, and the final prediction result is obtained.

[0051] Furthermore, when training the model, the negative log-likelihood function is used as the objective function:

[0052] ;

[0053] in, The data is the labeled training set data, and N is the number of data in the training set.

[0054] The beneficial technical effects of this invention are as follows:

[0055] This invention proposes a Chinese named entity recognition method based on multi-feature fusion of dictionary information. This method constructs multiple word sequences based on Chinese character, glyph, and pinyin features, combined with dictionary information. It fuses word information from multiple dimensions to improve the overall recognition performance of the model. Furthermore, it decomposes characters into basic components, words into composite components of characters, converts characters into pinyin, and words into initials and finals of characters. Using a convolutional neural network, it captures word features at different granularities, making full use of prior knowledge of Chinese characters and effectively addressing the problem of insufficient understanding of low-frequency words and out-of-vocabulary words. The method embeds the characters, glyphs, and pinyin into vectors and performs linear mapping. Relative positional encoding is fused into the corresponding word sequences. An improved Transformer model is used to obtain the final feature vectors of characters, glyphs, and pinyin. These feature vectors are concatenated and linearly mapped to obtain a fused feature vector. A conditional random field is then used to predict the entity label type. The improved Transformer model, by fusing positional information, deepens the model's understanding of semantic features, improves model performance, and thus enhances the accuracy and efficiency of Chinese named entity recognition. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a flowchart illustrating a Chinese named entity recognition method based on multi-feature fusion dictionary information of Chinese characters, provided by an embodiment of the present invention.

[0058] Figure 2 This is a schematic diagram of the recursive decomposition process of Chinese characters provided in an embodiment of the present invention;

[0059] Figure 3 This is a schematic diagram illustrating the process of extracting glyph features using CNN according to an embodiment of the present invention;

[0060] Figure 4 This is an index diagram of the head and tail positions of the span contained in the sequence provided in the embodiments of the present invention. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] The following is a brief explanation of the technical terms used in this embodiment:

[0063] CNN: Convolutional Neural Network, a deep learning model that uses local connections and shared weights, and is widely used in image recognition and natural language processing.

[0064] CRF: Conditional Random Field, a discriminative probabilistic model commonly used for tasks such as part-of-speech tagging and named entity recognition.

[0065] Transformer: A neural network architecture proposed by Google, it is a sequence model based on an attention mechanism.

[0066] This embodiment provides a Chinese named entity recognition method based on multi-feature fusion dictionary information of Chinese characters. The overall architecture is as follows: Figure 1 As shown. The method includes the following steps:

[0067] Given a sequence of input text ,in, express The Middle One character.

[0068] Using an open Chinese lexicon as dictionary L, a prefix tree with a tree structure is constructed. Tree nodes store characters in the vocabulary, and tree paths form corresponding words to improve word matching efficiency.

[0069] Traverse the input sequence using a prefix tree. All character subsequences in the dictionary are matched against the dictionary L to find all possible matching words, forming a sequence of matching words. ,in, express The 1 matching term.

[0070] Concatenate the matching word sequence to the corresponding text sequence After that, form a word combination sequence .

[0071] By looking up the embedding table, obtain The character embedding vectors of the character sequence and the word embedding vectors of the word sequence. For In the th character And the th matching word The corresponding embedding vectors are shown as follows:

[0072] ;

[0073] ;

[0074] Among them, Is the character embedding lookup table, Is the word embedding lookup table, Is the character Character embedding vector, Is the matching word Word embedding vector.

[0075] For The character sequence, obtain the character sequence character embedding representation as: , For The word sequence, obtain the word sequence word embedding representation as: . After concatenating the character embedding representation and the word embedding representation, obtain The word and character embedding feature : , Among them, ⊕ represents the concatenation operation, Is the character sequence character embedding representation, Is the word sequence word embedding representation.

[0076] Since Chinese characters evolved from pictographs, a large amount of character semantic information is contained in the glyph structure of Chinese characters. For example, "dao (island)", "feng (peak)", and "yue (Five Sacred Mountains)" all contain the radical "mountain", and semantically are all related to mountains and terrain. In this invention, for low-frequency words and out-of-vocabulary words, the glyph structure analysis is used to enhance the model's reasoning about the character meanings.

[0077] The glyph of Chinese characters can be divided into three hierarchical structures: the whole character, components, and strokes. Chinese character components can be divided into basic components and composite components. For example, for the Chinese character "feng", when disassembled according to basic components, it becomes "‘mountain’, ‘zhi’, ‘feng’", and when disassembled according to composite components, it can be split into "‘mountain’, ‘feng’". The phonetic or semantic features of basic components and composite components all contain a large amount of phonetic and semantic information.

[0078] Using recursive decomposition, the Chinese characters are iteratively extracted multiple times according to their glyph structures until all decompositions are completed. For example, Figure 2 as shown, by decomposing characters and words at different granularities, information redundancy is avoided. For the characters in the character sequence, the glyphs of the basic components are decomposed to obtain the glyph sequence of the characters , for example, the glyph sequence of the Chinese character "峰" is {'山','夂', '丰'}. For the words in the word sequence, the glyphs of the composite components are split to obtain the glyph sequence of the words , for example, the glyph sequence of the word "山峰" is {'山', '山', '夆'}. To simplify the encoding, in this embodiment, different stroke deformations or proportion changes of the components are regarded as components in the same group and are encoded with the same code. For example, the gold radical "钅" and "金" are regarded as components in the same group.

[0079] For example Figure 3 as shown, the glyph sequences of the character sequence and the word sequence are input into the first convolutional neural network CNN1, and successively enter the convolutional layer and the max-pooling layer, and finally are transformed through the fully connected layer to obtain the glyph features of the character sequence and the word sequence, as shown in the following formula:

[0080] ;

[0081] ;

[0082] where represents the th character in 's glyph embedding vector, represents the th matched word

[0083] in 's glyph embedding vector. CNN1 represents the first convolutional neural network, and the convolutional kernel size is 3. For the character sequence of , the glyph embedding representation of the character sequence is obtained as: . For the word sequence of , the glyph embedding representation of the word sequence is obtained as: . After splicing the glyph embedding representation of the character sequence and the glyph embedding representation of the word sequence, the

[0084] ;

[0085] where ⊕ represents the splicing operation, is the glyph embedding representation of the character sequence, is the glyph embedding representation of the word sequence.

[0086] Chinese pinyin is spelled using Roman letters. Chinese characters with similar structures may have similar pronunciations, play similar roles in sentences, and have similar word classes. Pinyin includes initials, finals, and whole-syllable readings. Some Chinese characters have multiple pronunciations and are polysyllabic characters. For example, the character "会" is pronounced as "huì" with meanings such as "to gather", "to meet", "meeting", and pronounced as "kuài" meaning "total". In addition, in a vast amount of social media texts, the usage of various characters and words may be irregular, and the misuse and mixing of homophonic and形似字 (similar-looking characters) can easily cause the model to have deviations in the overall understanding of sentences. The present invention combines Chinese pinyin features with glyph features to deepen the model's understanding of polysyllabic characters, homophonic characters, and similar-looking characters.

[0087] With the help of matching words, different pronunciations of characters are distinguished. Through the pronunciation lookup table, the pinyin and tones of the characters in the input sequence are obtained. For the four tones of flat tone, rising tone, falling-rising tone, and falling tone, they are represented by 1, 2, 3, and 4 respectively. The characters in the character sequence are composed of initial letters, final letters, and tones to obtain the pinyin sequence of the characters. For example, the pinyin sequence of the character "峰" is {'f', 'e', 'n', 'g', '1'}. For the words in the word sequence, they are composed of initials, finals, and tones to obtain the pinyin sequence of the words. For example, the pinyin sequence of the word "山峰" is {'sh', 'an', '1', 'f', 'eng', '1'}. The pinyin sequence of the words. For example, the pinyin sequence of the word "山峰" is {'sh', 'an', '1', 'f', 'eng', '1'}.

[0088] Similar to the extraction of glyph features, the pinyin sequences of the character sequence and the word sequence are input into the second Convolutional Neural Network CNN2 to obtain the pinyin features of the character sequence and the word sequence, as shown in the following formula:

[0089] [[ID=第22行]];

[0090] ;

[0091] For the character sequence, the pinyin embedding representation of the character sequence is obtained as: . For the word sequence, the pinyin embedding representation of the word sequence is obtained as: . After splicing the pinyin embedding representation of the character sequence and the pinyin embedding representation of the word sequence, the pinyin embedding feature of is obtained: :

[0092] ;

[0093] where, ⊕ represents the splicing operation. It should be noted that the term "形似字" in the original text is directly retained in the translation as there is no exact equivalent in English. If there is a more specific English term for it in the context of this text, it can be further adjusted. The pinyin embedding representation of the character sequence, The word sequence is represented by pinyin embedding.

[0094] Will All characters and words in the text are treated as spans and numbered sequentially, forming a sequence. ,in, and They represent The renumbered number The and the first Each character or word can also be considered as a span.

[0095] For the NER task, the encoder part of the Transformer is used for encoding. The encoder part consists of multiple encoder layers. Each encoder layer consists of a self-attention sub-layer and a feedforward neural network sub-layer. The self-attention sub-layer uses the self-attention mechanism to capture the correlation features between different positions within the sequence. The feedforward neural network sub-layer extracts more complex features through non-linear transformations. Each sub-layer then undergoes residual connections and layer normalization. To avoid redundancy, the computation process of the first encoder layer is described as an example; subsequent encoder layers are similar.

[0096] like Figure 4 As shown, positional encoding is added to each span (character, word), based on the head position index and the tail position index, to preserve the original boundary information of the character and word. The head position index and the tail position index of the character are the same. Given... and Indicates span The position of the head and the position of the tail. and Indicates span The head and tail positions.

[0097] To indicate the span and The relationship between these factors is used to calculate the distances to four relative positions. The process is as follows:

[0098] ;

[0099] ;

[0100] ;

[0101] ;

[0102] in, Indicates span Head position and span The distance between the head positions, Indicates span Head position and span The distance between the tail positions, Indicates span Tail position and span The distance between the head positions, Indicates span Tail position and span The distance between the tail positions.

[0103] The four relative position distances are encoded as follows:

[0104] ;

[0105] ;

[0106] in, , A vector representing relative position. Indicates even-numbered dimensions. Representing odd-numbered dimensions, represent , , , The four relative positions shown are and The index representing the positional encoding dimension, in this embodiment The value range is 0-511. × , Indicates the number of heads that attract attention. It is the dimension of each attention head.

[0107] By integrating four positional codes, the span can be calculated. and Relative position encoding between :

[0108] ;

[0109] in, Indicates span Head position and span The relative positional distance encoding between the head positions, , and For similar relative position encoding, It is a learnable parameter matrix. ⊕ represents the activation function, and ⊕ represents the concatenation operation.

[0110] The word embeddings, glyph embeddings, and pinyin embeddings of sequence q are input into the improved Transformer to incorporate word features. In the self-attention sub-layer, the self-attention weights are modified to further improve the Transformer. , , The inputs are fed into a Transformer, mapped to query, key, and value matrices, and their respective values ​​are calculated. , , value:

[0111] ;

[0112] in, It is a parameter matrix. represent , , Embedded features.

[0113] Based on span and The relative positional encoding between them is used to calculate improved attention weights using a self-attention mechanism. and weighted value :

[0114] ;

[0115] ;

[0116] in, For query vector, For key vectors, For value vectors, For span and Relative position encoding between them It is a learnable relative position encoding mapping matrix. It is a learnable key-value vector mapping matrix. It is the parameter matrix after relative position encoding mapping. This represents the matrix transpose operation. This is a normalized exponential function. (Sequence) The corresponding contextual feature matrix is .

[0117] The feedforward neural network sublayer consists of a non-linear activation function and two linear transformation layers. The output of the self-attention layer... Feed into the feedforward neural network Layers are used to obtain sequence features. :

[0118] ;

[0119] Among them, is the learnable weight of the two-layer linear transformation, is the bias of the two-layer linear transformation, represents the word sequence feature of , the glyph sequence feature and the pinyin sequence feature .

[0120] Concatenate the three sequence features and then perform a linear transformation to obtain the final fused feature :

[0121] ;

[0122] Among them, , , are respectively the word, glyph, and pinyin sequence features of is the learnable weight of the linear transformation, is the bias of the linear transformation.

[0123] Input the final fused feature corresponding to the character sequence into the conditional random field (CRF) decoder for sequence labeling. The CRF combines the overall sentence information, pays attention to the dependencies between sequence labels, models the transition probabilities of different labels, and achieves good results in sequence labeling.

[0124] Distinguish according to the position of the character in the word, and adopt the BMES labeling scheme to label the sequence. B, M, and E respectively represent the beginning, middle, and end positions of the character in the word, and S indicates a single character word. Taking the place name entity "Yangtze River Bridge" as an example, "Chang" is labeled as "B-LOC", "Jiang" and "Da" are labeled as "M-LOC", and "Qiao" is labeled as "E-LOC".

[0125] For the sequence , the predicted label sequence is , and the probability of calculating the output prediction sequence y is :

[0126] ;

[0127] Among them, is the state transition feature function of the conditional random field, calculating the probability that the label at the (i - 1)-th position in the sequence s transfers to the label at the i-th position, Let be the state feature function, and calculate the label at the i-th position in sequence s. The probability, , The domain knowledge of the NER task is predefined, and the value is 1 when certain conditions are met, and 0 otherwise. For example, if and ,but Otherwise, it is 0. These are learnable weight parameters; the larger the weight, the more important the feature function.

[0128] Using the Viterbi algorithm, the highest probability label sequence is found recursively; this is the most likely label sequence, yielding the final prediction result. To train the model, the negative log-likelihood function is used as the objective function.

[0129] ;

[0130] in, The data is the labeled training set data, and N is the number of data in the training set.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for Chinese named entity recognition based on multi-feature fusion dictionary information of Chinese characters, characterized in that, The method includes: Step S1, given the input text sequence ,in, express The Middle Using a single character and an open Chinese dictionary L as the dictionary, a prefix tree with a tree structure is constructed. Step S2: Traverse the input sequence using the word prefix tree. All character subsequences in the text are matched against dictionary L to find all possible matching words, forming a matching word sequence. This matching word sequence is then concatenated into the corresponding text sequence. Then, a sequence of word combinations is formed. ; Step S3, obtain by looking up the embedded table. The character embedding representation of the character sequence and the word embedding representation of the word sequence are concatenated to obtain... Word embedding features ; Step S4: Decompose the characters in the character sequence using basic component glyphs to obtain the characters. glyph sequence The words in the word sequence are decomposed using the glyphs of the compound components to obtain the words. glyph sequence ; Step S5: Input the glyph sequences of the character sequence and word sequence into the first convolutional neural network CNN1, which then passes through convolutional layers, max pooling layers, and finally fully connected layers to transform them, obtaining glyph embedding representations of the character sequence and word sequence. These representations are then concatenated to obtain... glyph embedding features ; Step S6: Obtain the input sequence using the phonetic lookup table. The pinyin and tones of Chinese characters are determined by using initial consonants, final vowels, and tones to form the character sequence. Pinyin sequence For word sequences, the initial consonant, final vowel, and tone are used to obtain the word's... Pinyin sequence ; Step S7: Input the pinyin sequences of the character sequence and word sequence into the second convolutional neural network CNN2 to obtain the pinyin embedding representations of the character sequence and word sequence. Then, concatenate the pinyin embedding representations of the character sequence and word sequence to obtain... Pinyin embedding features ; Step S8, sequence All characters and words in the text are used as spans and numbered sequentially to form a sequence. Then, the Transformer encoder is used to encode the final fused features; Step S9: Input the final fused features into the Conditional Random Field (CRF) decoder for sequence labeling.

2. The Chinese named entity recognition method based on multi-feature fusion dictionary information of Chinese characters according to claim 1, characterized in that, The Transformer encoder consists of multiple encoder layers, each of which includes a self-attention sublayer and a feedforward neural network sublayer.

3. The Chinese named entity recognition method based on multi-feature fusion dictionary information of Chinese characters according to claim 2, characterized in that, Represented as ,in, express The Matching words, sequence Represented as ,in, and They represent The renumbered number The and the first A single character or word can also indicate a range.

4. The Chinese named entity recognition method based on multi-feature fusion dictionary information of Chinese characters according to claim 3, characterized in that, The working principle of each encoder layer is as follows: Positional encoding is added to each span (character / word), based on the head and tail position indices. The head and tail position indices of a character are the same. and Indicates span The position of the head and the position of the tail. and Indicates span Based on the head and tail positions, calculate the four relative distances, as shown below: ； ； ； ； in, Indicates span Head position and span The distance between the head positions, Indicates span Head position and span The distance between the tail positions, Indicates span Tail position and span The distance between the head positions, Indicates span Tail position and span The distance between the tail positions; The four relative position distances are encoded as follows: ； ； in, , A vector representing relative position. and An index representing the location encoding dimension. represent , , , The four relative positions shown are × , Indicates the number of heads that attract attention. It is the dimension of each attention head; By integrating four positional codes, the span can be calculated. and Relative position encoding between : ； in, Indicates span Head position and span The relative positional distance encoding between the head positions, Indicates span Tail position and span The relative positional distance encoding between the head positions, Indicates span Head position and span The relative positional distance encoding between the tail positions, Indicates span Tail position and span The relative position encoding between the tail positions, It is a learnable parameter matrix. This represents the activation function, and ⊕ represents the concatenation operation; Then embed the words of sequence q. , character embedding and Pinyin embedding The inputs are fed into the improved Transformer, mapped to query, key, and value matrices, and their respective computations are performed. , , The value is represented as follows: ； in, It is a parameter matrix. represent , , Embedded features; Then based on span and The relative positional encoding between them is used to calculate improved attention weights using a self-attention mechanism. and weighted value The calculation method is as follows: ； ； in, For query vector, For key vectors, For value vectors, For span and Relative position encoding between them It is a learnable relative position encoding mapping matrix. It is a learnable key-value vector mapping matrix. It is the parameter matrix after relative position encoding mapping. This represents the matrix transpose operation. It is a normalized exponential function; Then the output of the self-attention layer Send to feedforward neural network Layers are used to obtain sequence features. : ； in, These are the learnable weights of two linear transformations. These are the biases of the two linear transformations, respectively. represent Word sequence features Character sequence features and Pinyin sequence features ; Finally, the three sequence features are concatenated and then linearly transformed to obtain the final fused features. : ； in, , , They are respectively The features of words, characters, and phonetic sequences, These are the learnable weights of a linear transformation. It is the bias of the linear transformation.

5. The Chinese named entity recognition method based on multi-feature fusion dictionary information of Chinese characters according to claim 4, characterized in that, Step S9 further includes: For sequences The predicted label sequence is The probability of the output predicted sequence y is calculated as follows: : ； in, Let be the state transition characteristic function of a conditional random field, and calculate the label at the (i-1)th position in the sequence s. Move to the i-th position label The probability, Let be the state feature function, and calculate the label at the i-th position in sequence s. The probability, These are learnable weight parameters; Using the Viterbi algorithm, the highest probability label sequence is found through recursion, which is the most likely label sequence to appear, and the final prediction result is obtained.

6. The Chinese named entity recognition method based on multi-feature fusion dictionary information of Chinese characters according to claim 5, characterized in that, When training the model, the negative log-likelihood function is used as the objective function: ； in, The data is the labeled training set data, and N is the number of data in the training set.

Images