Chinese named entity recognition method and system fusing Chinese character root, voice and character features

Abstract

The invention discloses a Chinese named entity recognition method and system fusing Chinese character root, voice and character features, and the method comprises the steps: 1, carrying out the preprocessing of a given character sequence, and obtaining a plurality of features; 2, processing the plurality of features by using a convolutional neural network and a maximum pooling technology to obtain a plurality of feature vectors; 3, fusing the plurality of feature vectors, and sequentially inputting the fused vectors into the RoBERTa model and the bidirectional long and short time memory network to obtain a fused output sequence; and 4, processing the output sequence according to the conditional random field to obtain a maximized tag sequence, and completing the recognition of the Chinese named entity. According to the invention, the accuracy and efficiency of separation boundary detection and entity category determination are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of Chinese named entity recognition, and particularly to a Chinese named entity recognition method and system that integrates Chinese character root, voice, and character features. Background Art

[0002] With the continuous progress and development of technologies in the field of natural language processing (NLP), the demand for more advanced and intelligent Chinese text processing and understanding technologies is also increasing day by day. This is particularly evident in the field of named entity recognition. Named Entity Recognition (NER) is a sequence labeling task that aims to extract structured named entities from unstructured text and accurately identify their boundaries and categories. Named entity recognition plays an important fundamental role in the field of natural language processing (NLP) and is widely used in downstream tasks such as knowledge graph construction, question answering systems, and information retrieval. Traditional Chinese named entity recognition methods mainly focus on character-based or word-based features, but these features cannot accurately capture the rich and multi-dimensional essence of Chinese.

[0003] Chinese characters have unique features. Chinese characters can not only convey semantic information through their own structural components but also convey phonetic information through pronunciation. The above information plays an important role in helping the model understand the context and meaning of Chinese text. There are spaces between English words as word separators, and these separators can help with automatic English word segmentation. Different from English, there are no spaces between Chinese characters as word separators, and Chinese character segmentation mainly relies on the semantic information of the sentence. On the other hand, each Chinese character represents a morpheme or a meaning unit. Chinese characters can represent both sound and meaning, which makes it possible for the same Chinese character to have different pronunciations and meanings in the text, resulting in the entity boundary changing with the change of Chinese character pronunciation in a specific context. This poses a huge challenge to traditional Chinese named entity recognition methods, which mainly rely on the surface features of characters or words. For example, the character "好" is composed of two radicals: "女" and "子". The meaning of this character itself is "good", but its meaning is composed of the meanings of the other radicals. Traditional Chinese named entity recognition methods do not consider the internal structure of Chinese characters, resulting in difficulties in recognizing all the semantic information of Chinese characters. Moreover, some Chinese characters are homophones, that is, they have the same pronunciation but different meanings, which makes the task of Chinese named entity recognition more complex because it needs to distinguish Chinese characters according to context information.

[0004] Traditional Chinese named entity recognition methods mainly use statistical models such as Hidden Markov Model (HMM), Conditional Random Field (CRF), and Support Vector Machine (SVM). These methods regard Chinese characters as single characters, ignoring the internal structure and phonetic information of Chinese characters. This leads to missing important clues for distinguishing named entities when identifying Chinese characters with similar structures or pronunciations. Moreover, traditional methods usually require a large amount of labeled data for training, which is difficult for practical applications. Manually labeling data is both time-consuming and laborious, and is prone to mislabeling. Especially for Chinese, the boundaries between words are not clear, and the meaning of the same Chinese character can change according to different contexts. The dependence on labeled data results in poor generalization of traditional Chinese named entity recognition methods to new domains because there may not be sufficient labeled data in new domains.

[0005] To address the limitations brought by traditional Chinese named entity recognition methods, recent research has focused on methods that integrate multiple linguistic features of Chinese characters to improve the accuracy and robustness of Chinese named entity recognition models. These methods decompose Chinese characters into basic components such as radicals (radical parts), phonetics (pinyin), and single-character features. By leveraging the basic compositional features of Chinese characters and incorporating them into the named entity recognition process, the model can deepen its understanding of the given Chinese text, thereby enhancing the reliability and accuracy of named entity recognition. One method used by some researchers is radical embedding of Chinese characters. Chinese characters originated from pictographs, and their structures can reflect more information. Radicals are the basic components of Chinese characters and can provide meaningful semantic information for the model. Using vector representations in a continuous space for the radicals obtained by decomposing Chinese characters can create associations between Chinese characters and radicals, enabling the model to capture the connection between Chinese characters and radicals. Further, combining radical embedding with character embedding can create more semantically rich representations for each Chinese character, thus improving the accuracy of named entity recognition. For example, consider the characters "松 (sōng)" and "杨 (yáng)". These two characters have the same radical "木 (mù, representing trees)", and are both related to trees. Radicals like "木" help identify entities of the tree category. Also, in English, different words may have the same root or affix, which better represents the semantics of the words. For example, "geologist", "biologist", and "oceanologist" all contain the same suffix "ologist", which helps identify entities of the expert category. The radicals of Chinese characters are similar to the roots and affixes of English.

[0006] The phonetic features represented by Chinese pinyin also play an important role in Chinese named entity recognition. The method used by some researchers is Chinese phonetic embedding. Pinyin is the romanization of Chinese characters, providing a phonetic representation of the pronunciation of Chinese characters. The pinyin system contains five different tones: neutral tone (), high tone (ˉ), rising tone (′), falling-rising tone (ˇ), and falling tone (`). By combining the Chinese pinyin features and character features, it is possible to utilize the phonetic similarities and differences between Chinese characters to solve the difficult problem of distinguishing characters with the same form but different pronunciations. For example, in the phrase "Nanjing Yangtze River Bridge", the Chinese character '长' has two different pronunciations in the pinyin system: 'cháng' and 'zhǎng'. The initial letter, final letter, and tone of the character '长(cháng)' are 'ch', 'ang', and '′' respectively. The initial letter, final letter, and tone of the character '长(zhǎng)' are 'zh', 'ang', and 'ˇ' respectively. When pronounced as 'cháng', it means length, and when pronounced as 'zhǎng', it means leader. For the given example sentence "Nanjing Yangtze River Bridge", the meaning of '长(cháng)' should be length and is used in the phrase '长(cháng)江大桥', which means 'Yangtze River Bridge', thus forming a complete entity, and the starting boundary of the entity should be set before '长(cháng)'. This example shows that phonetic features play an important role in entity boundary detection, can help to more accurately determine the start and end positions of entities, and thus improve the accuracy of entity recognition. Traditional named entity recognition methods only utilize the character features of Chinese characters and are difficult to distinguish characters with the same form but different pronunciations, especially in the context where both are valid entities. By embedding phonetic features into the model, it is possible to utilize pinyin information to help the model recognize these characters, thereby achieving more accurate entity recognition. However, current Chinese named entity recognition technology still has problems of low accuracy and low efficiency in separation boundary detection and entity category determination. Summary of the Invention

[0007] In order to at least partially solve the problems of low accuracy and low efficiency in separation boundary detection and entity category determination of traditional Chinese named entity recognition technology, the present invention provides a Chinese named entity recognition method and system that integrates Chinese character root, phonetic, and character features. Taking the integration embedding of Chinese character root, phonetic, and character features as the research object, according to the characteristics of the external features of Chinese characters, a convolutional neural network (CNN) is used to extract the external features of Chinese characters, and the features extracted by the CNN are combined with a pre-trained model to automatically recognize a large number of named entities, improving the accuracy and efficiency in separation boundary detection and entity category determination.

[0008] To achieve the above object, the technical solution of the present invention is:

[0009] The first aspect of the present invention proposes a Chinese named entity recognition method integrating Chinese character roots, voice and character features, comprising:

[0010] Step 1: Preprocess the given character sequence to obtain multiple features to facilitate subsequent feature extraction;

[0011] Step 2: Use convolutional neural network and maximum pooling technology to process multiple features to obtain multiple feature vectors for further feature extraction;

[0012] Step 3: Fuse multiple feature vectors and input the fused vectors into the RoBERTa model and the bidirectional long short-term memory network in sequence to obtain the fused output sequence for feature fusion;

[0013] Step 4: Process the output sequence according to the conditional random field to obtain the maximized label sequence and complete the recognition of Chinese named entities.

[0014] Furthermore, the given character sequence is preprocessed to obtain multiple features, specifically including:

[0015] Decompose the Chinese characters in a given character sequence into structural components according to the Chinese character decomposition dictionary, and use them as root features of the Chinese characters;

[0016] According to the open source pypinyin library, the Chinese characters in a given character sequence are converted into corresponding pinyin, which is used as the phonetic feature of the Chinese characters;

[0017] The Chinese characters in the given character sequence are taken as the character features of the Chinese characters.

[0018] Furthermore, the step 2 comprises:

[0019] Embed the root features, speech features and character features into corresponding embedding matrices respectively to obtain the original root embedding vector, original speech embedding vector and original character embedding vector, which are convenient for subsequent feature extraction;

[0020] According to the convolutional neural network, feature extraction is performed on the original root feature embedding matrix and the original speech feature embedding matrix respectively, so as to capture local associations through convolution;

[0021] The maximum pooling technology is used to process the output of the convolutional neural network to obtain the root embedding vector and the speech embedding vector, which is convenient for subsequent feature fusion.

[0022] Furthermore, the feature extraction is performed on the original root feature embedding matrix and the original speech feature embedding matrix respectively according to the convolutional neural network, which is specifically expressed by the following formula:

[0023]

[0024] Among them, C r (i, j) is the i-th convolution output in the root feature map, C p (i,j) is the i-th convolution output in the speech feature map, k is the height of the convolution window, d r is the dimension of the original root embedding vector, W r (m,n) is the weight of the root convolution kernel in the mth row and nth column, W p (m,n) is the weight of the speech convolution kernel in the mth row and nth column, R is the root embedding matrix, b r is the bias parameter of the root convolution kernel, j is C r (i, j) and C p (i,j) dimension index, P is the speech embedding matrix, b p is the bias parameter of the speech convolution kernel, d p is the dimension of the original speech embedding vector;

[0025] The maximum pooling technique is used to process the output of the convolutional neural network, which is expressed by the following formula:

[0026]

[0027]

[0028] Among them, h r (i, j) is the root embedding vector of the root feature map at position (i, j) after the maximum pooling operation, h p (i, j) is the speech embedding vector of the speech feature map at position (i, j) after the maximum pooling operation, C r (i+m, j) and C p (i+m, j) are the values ​​of the root convolution output and the speech convolution output in the pooling window respectively, and k is the height of the convolution window.

[0029] Furthermore, the fusing of the multiple feature vectors specifically includes:

[0030] The root embedding vector, the voice embedding vector and the original character embedding vector are concatenated to obtain a fusion vector, which is convenient for fully integrating the features of different information sources (roots, voices and characters);

[0031] The fused vector is linearly transformed using a fully connected layer to obtain a fused embedding vector, which is used to prevent the root, voice and character feature information from being weakened during the fusion process, reduce information loss and enhance the feature expression ability of the model.

[0032] Furthermore, the fusion vector is linearly transformed using the fully connected layer, which is expressed by the following formula:

[0033] E′ i =Fusion(h r , x i ,h p )=W(E i )+b

[0034] in, is the weight matrix of the fully connected layer, b∈R out is the bias vector, d out is the dimension of the output vector, E′ i To fuse the embedding vector, is a vector concatenation operation, Fusion is a fusion operation, h r is the root embedding vector, h p is the speech embedding vector, x i is the original character embedding vector, E i is the fusion vector.

[0035] Furthermore, the step 4 specifically includes:

[0036] Obtain the label sequence based on the conditional random field and the output sequence and construct a scoring function, and calculate the probability of the output sequence generating the label sequence based on the scoring function;

[0037] According to the Viterbi Algorithm algorithm, the label sequence that maximizes the score function is found among all possible label sequences.

[0038] The second aspect of the present invention proposes a Chinese named entity recognition system that integrates Chinese character roots, voice and character features, including:

[0039] The preprocessing module is used to preprocess a given character sequence to obtain multiple features for subsequent feature extraction;

[0040] The feature extraction module is used to process multiple features using convolutional neural networks and maximum pooling technology to obtain multiple feature vectors for further feature extraction;

[0041] The fusion module is used to fuse multiple feature vectors and input the fused vectors into the RoBERTa model and the bidirectional long short-term memory network in sequence to obtain the fused output sequence for easy feature fusion;

[0042] The recognition module is used to process the output sequence according to the conditional random field to obtain the maximized label sequence and complete the recognition of Chinese named entities.

[0043] The third aspect of the present invention proposes an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for Chinese named entity recognition that integrates Chinese character roots, voice and character features as described in the first aspect above is implemented.

[0044] The fourth aspect of the present invention proposes a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the Chinese named entity recognition method that integrates Chinese character roots, voice and character features as described in the first aspect above.

[0045] Beneficial effects of the present invention:

[0046] The present invention analyzes the characteristics of Chinese characters themselves, and aims at the limitations faced by the current Chinese named entity recognition method. First, according to the characteristics of Chinese characters themselves, Chinese characters are decomposed into structural components to construct Chinese character root-level features, and Chinese characters are decomposed into pinyin to construct Chinese character voice features. On this basis, a convolutional neural network is used to extract Chinese character features, local associations are captured by convolution, and the root embedding vector and voice embedding vector are generated by the maximum pooling technology, which are then spliced ​​with the character embedding vector to form a fused embedding vector, and then the fused embedding vector is fused using a fully connected layer. Secondly, the fused embedding vector is input into a bidirectional long short-term memory network for context modeling. Finally, a conditional random field is used for sequence prediction and the label sequence score is returned, thereby realizing the recognition of Chinese named entities. The comparative experiments show that the present invention improves the recognition efficiency and recognition accuracy of separation boundary detection and entity category in the Chinese environment, has better recognition effect, and provides a research idea for more reasonable selection of Chinese character external feature information and further improving the recognition effect and efficiency of the Chinese named entity recognition method. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A flowchart of a Chinese named entity recognition method that integrates Chinese character roots, voice and character features provided in an embodiment of the present invention.

[0048] Figure 2 A specific flow chart of a Chinese named entity recognition method that integrates Chinese character roots, voice and character features provided in an embodiment of the present invention.

[0049] Figure 3 A schematic diagram of the practical application of the Chinese named entity recognition method that integrates Chinese character roots, voice and character features provided by an embodiment of the present invention.

[0050] Figure 4A schematic diagram of a Chinese named entity recognition system that integrates Chinese character roots, voice and character features provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] Example 1

[0053] like Figure 1 and Figure 2 As shown in the figure, the Chinese named entity recognition method integrating Chinese character roots, pronunciation and character features includes:

[0054] S101: Preprocess a given character sequence to obtain multiple features.

[0055] Specifically, in the initial stage, first input a character sequence S to the model. The character sequence S can be a Chinese character or a Chinese sentence, and the length of the character sequence S is not limited. The pypinyin library used in the present invention is a Python library for converting Chinese characters into pinyin. It is based on the Chinese character pinyin data table and provides a simple and powerful API that can meet the needs of various Chinese character conversions to pinyin. The pypinyin library supports a variety of pinyin styles and output formats, and also provides a wealth of parameter options, so that researchers can customize the conversion results according to their needs. The Chinese character decomposition dictionary used in the present invention is a text document that converts Chinese characters into structural components, which can meet the needs of various Chinese character conversions to roots. The preprocessing work of the whole method is mainly to perform word segmentation, remove invalid words, etc. on the training set in the Resume data set. The word segmentation tool is used in the model to perform word segmentation on the input character sequence S, making it a single Chinese character, so that it is convenient to decompose it into the root, pronunciation and character features of Chinese characters later. It can be specifically carried out according to the following process:

[0056] Using a Chinese character decomposition dictionary, each Chinese character is structurally decomposed into its constituent components (such as radicals), which are used as the root features of the Chinese character.

[0057] The open source pypinyin library is used to convert each Chinese character into pinyin, and the Chinese characters are converted into Chinese pinyin, which is used as the phonetic feature of the Chinese character.

[0058] The single Chinese character obtained after word segmentation of the character sequence S is used as the character feature of the Chinese character.

[0059] S102: Process multiple features using a convolutional neural network and maximum pooling technology to obtain multiple feature vectors.

[0060] Specifically, according to the above steps, the character sequence S = {x 1 ,x 2 ,...,x n} The result of preprocessing is to obtain the root, phonetic and character features of Chinese characters. According to the root features of Chinese characters, the root embedding matrix R = [r 1 ,r 2 ,...,r n ], the root embedding matrix R is used to store the root embedding vectors of all Chinese characters. Each Chinese character finds the corresponding root vector in the embedding matrix through the index. For a given input character sequence S = {x 1 ,x 2 ,...,x n Each Chinese character x in i , and its corresponding root feature is converted into a vector representation in the embedding matrix through the mapping function f(·). The original root embedding vector r i It is expressed as:

[0061]

[0062] Among them, the root feature i is the root feature of the i-th Chinese character, r i is the original root embedding vector, d r is the dimension of the original root embedding vector.

[0063] According to the phonetic features of Chinese characters, the phonetic embedding matrix P = [p 1 ,p 2 ,...,p n ], the speech embedding matrix P is used to store the speech embedding vectors of all Chinese characters. Each Chinese character finds its corresponding speech vector in the embedding matrix through the index. For each Chinese character x in the given input character sequence S i , the pinyin corresponding to each Chinese character is converted into a vector in high-dimensional space through the mapping function g(·), and the original root embedding vector p i It is expressed as:

[0064]

[0065] Among them, voice features i is the phonetic feature of the i-th Chinese character, p i is the original root embedding vector, d p is the dimension of the original speech embedding vector.

[0066] Construct a character embedding matrix X=[x 1 ,x 2 ,...,x n ], the character embedding matrix X is used to store the character embedding vectors of all Chinese characters. Each Chinese character finds its corresponding character vector in the embedding matrix through the index. For each Chinese character x in the given input character sequence S i , the character corresponding to each Chinese character is converted into a vector in high-dimensional space through the mapping function y(·), and the original character is embedded in the vector x i It is expressed as:

[0067]

[0068] Among them, character features i is the character feature of the i-th Chinese character, x i is the original character embedding vector, d x is the dimension of the original character embedding vector.

[0069] An effective way to combine Chinese character roots, phonetics, and character features is to use neural networks, such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs). Neural networks can automatically learn complex patterns in data and can automatically capture features from a given Chinese text. By integrating Chinese character roots, phonetics, and character features into a neural network, a model that can capture Chinese text information can be created. For example, a model based on a convolutional neural network can fully learn spatial patterns in feature embedding, and a model based on a recurrent neural network can accurately capture the sequential dependencies between features.

[0070] The present invention selects convolutional neural network CNN for feature combination. The root, voice and character embedding matrix can independently represent the root, voice and character features of Chinese characters. By further inputting the features of two different dimensions of root and voice into the convolutional neural network for local perception and filtering, the local correlation between the root and voice embedding vectors is captured by convolution. This process combines the input root feature matrix R with the root convolution kernel W r , input speech feature matrix P and speech convolution kernel W p The element-by-element product and sum operation are performed respectively. First, the root and speech convolution kernel weights are initialized using uniform distribution to ensure the stability of the model in the initial stage of training.

[0071] Root, speech convolution kernel W r , W p is a learnable weight matrix of size k×d r , k×d p , where k represents the height of the convolution window, d r is the dimension of the root embedding vector, dp is the dimension of the speech embedding vector. r , W p It is expressed as:

[0072]

[0073] Among them, n is the number of input units of the convolution kernel, and μ represents uniform distribution.

[0074] Secondly, the convolution operation is performed by combining the root word and the speech convolution kernel W r , W p Sliding is applied to the input root feature matrix R and the input speech feature matrix P to generate a new feature map C r , C p Specifically, for the i-th position in the input root feature matrix R and the input speech feature matrix P, C r , C p It is expressed as:

[0075]

[0076] Among them, C r (i,j) is the i-th convolution output in the root feature map, C p (i,j) is the i-th convolution output in the speech feature map, W r (m,n) is the weight of the root convolution kernel in the mth row and nth column, W p (m,n) is the weight of the speech convolution kernel in the mth row and nth column. R is the root embedding matrix, b r is the bias parameter of the root convolution kernel, j is C r (i, j) and C p (i,j) dimension index, P is the speech embedding matrix, b p is the bias parameter of the speech convolution kernel, r(i+m-1,n) is the element W at the corresponding position of the root feature matrix R r , P(i+m-1,h) is the element W at the corresponding position of the speech feature matrix P p .

[0077] After the convolutional neural network operation is completed, the purpose is to further compress the size of the root and speech vector feature map and extract the most representative root and speech feature information. The maximum pooling technology is used to process the output of the convolution layer. r The convolution output C of (i, j) and speech features p(i, j), select the regional maximum from each pooling window area to generate a more compact feature representation. After the maximum pooling operation is completed, the generated root and speech two-dimensional features are flattened into a one-dimensional vector to generate the final root embedding vector h r and the speech embedding vector h p Specifically, the pooling window is set to k×k, then the root maximum pooling and speech maximum pooling are expressed as:

[0078]

[0079] Among them, h r (i, j) is the root embedding vector of the root feature map at position (i, j) after the maximum pooling operation, h p (i, j) is the speech embedding vector of the speech feature map at position (i, j) after the maximum pooling operation, C r (i+m,j) and C p (i+m,j) are the values ​​of the root convolution output and the speech convolution output in the pooling window, respectively, and k is the pooling window size.

[0080] S103: Fuse multiple feature vectors, and input the fused vectors into the RoBERTa model and the bidirectional long short-term memory network in sequence to obtain a fused output sequence.

[0081] Specifically, after the maximum pooling operation is completed, the generated root embedding vector h r and the speech embedding vector h p and the original character embedding vector x i The purpose of feature fusion is to fully integrate the features of different information sources (roots, voices and characters) to form a fusion vector E i , which contains the multi-dimensional semantic information of Chinese characters. The feature fusion operation is performed by vector concatenation, which concatenates the root, phonetic and character feature vectors on the same dimension to form a fusion vector. The above process is expressed by the following formula:

[0082]

[0083] Among them, ⊕ represents the vector concatenation operation, h r is the root embedding vector, h p is the speech embedding vector, x i is the original character embedding vector, d r +d p +d x is the fusion vector E i The characteristic dimension, E i is the fusion vector.

[0084] After the feature fusion operation is completed, the fusion vector E i Already contains the root embedding vector h r , speech embedding vector h p and the original character embedding vector x i In order to avoid the weakening of root, voice and character feature information during the fusion process, reduce information loss and enhance the feature expression ability of the model. A fully connected layer is used to integrate the fusion embedding vector E i Perform a linear transformation. Specifically, for the input fusion embedding vector E i , the weight matrix W and the bias vector b are used to perform a linear transformation to generate a fused embedding vector E′ with stronger feature expression ability i , the linear transformation is expressed as follows:

[0085] E′ i =Fusion(h r , x i ,h p )=W(E i )+b

[0086] in, is the weight matrix of the fully connected layer, Fusion is the fusion operation, b∈R out is the bias vector, d out is the dimension of the output vector, E′ i is the fused embedding vector, E i is the fusion vector.

[0087] After the information fusion of the fully connected layer, the fused embedding sequence E = {E′ 1 ,E′ 2 ,...,E′ n As a highly abstract and multi-dimensional information representation, these vectors combine the root, pronunciation and character information of Chinese characters, and further optimize the feature expression through the fully connected layer. 1 ,E′ 2 ,...,E′ n} is input into the pre-trained RoBERTa (Robustly optimized BERT pretraining approach) model for the next step of feature extraction. After being processed by the RoBERTa model, the model generates a new output sequence H = {h 1 ,h 2 ,...,h n}, where h i is the context-sensitive representation corresponding to the i-th element in the input sequence.

[0088] Fusion embedding sequence H = {h 1 ,h 2 ,...,h n After RoBERTa processing, the output sequence H = {h 1 ,h 2 ,...,h n} is passed as the input sequence to the BiLSTM network. The BiLSTM network processes the input sequence H = {h 1 ,h 2 ,...,h n} to generate hidden states in two directions and The final fused output sequence is H′={h′ 1 ,h′ 2 ,...,h′ n}. In the forward LSTM calculation, the hidden state of the current moment i is is based on the hidden state of the previous moment i-1 and the current input h i The recursive relationship is expressed as follows:

[0089]

[0090] in, is the forward hidden state, representing all the information from the start of the sequence to the current time i.

[0091] In the reverse LSTM calculation, the hidden state of the current time i is It is based on the hidden state of the next moment i+1 and the current input h i The recursive relationship is expressed as follows:

[0092]

[0093] in, It is the reverse hidden state, which represents the information gradually accumulated from the end point of the sequence to the starting point of the sequence.

[0094] Finally, in the framework of the bidirectional long short-term memory network, the forward hidden state generated by the forward LSTM Reverse hidden state generated with Reverse LSTM Fusion is performed at each time step i to generate the final hidden state h′ i Specifically, the fusion operation is implemented by vector concatenation, and the fusion operation is expressed as:

[0095]

[0096] Among them, h′ i To fuse the hidden state vector, it combines the forward and backward semantic information.

[0097] S104: Process the output sequence according to the conditional random field to obtain a maximized label sequence, thereby completing the recognition of Chinese named entities.

[0098] Specifically, after being processed by the bidirectional long short-term memory network, the fused output sequence H′={h′ 1 ,h′ 2 ,...,h′ n} contains the bidirectional semantic information corresponding to each character. This output sequence is input into the conditional random field to generate the corresponding predicted label sequence Y = {y 1 ,y 2 ,...,y n First, in the conditional random field, the input sequence X (i.e., the fused output sequence H′ obtained by bidirectional long short-term memory network processing) and a label sequence Y = {y 1 ,y 2 ,...,y n}, define a score function Score(X,Y) to evaluate the match between the label sequence Y and the input sequence X. The score function is expressed as follows:

[0099]

[0100] Among them, ω(y i-1 ,y i ,h′ i ) is the label y i-1 and i The transfer score between the label and the input feature h′ i Then, the conditional random field calculates the probability P(Y|X) that the input sequence X generates a specific label sequence Y according to the score function Score(X,Y). This process is expressed as:

[0101]

[0102] Among them, exp(Score(X,Y)) is the score index corresponding to the label sequence Y, ∑ Y′ exp(Score(X,Y′)) is the score exponential sum of all possible label sequences.

[0103] Finally, the Viterbi Algorithm is used to find the label sequence Y that maximizes the score function Score(X,Y) among all possible label sequences. *, this process is expressed as:

[0104]

[0105] According to the maximized label sequence, the recognition of Chinese named entities is completed.

[0106] like Figure 3 As shown, the present invention combines Chinese character roots, speech features and traditional character-level features, extracts features through convolutional neural networks and maximum pooling technology, fuses the three processed features and sequentially inputs the RoBERTa model and the bidirectional long short-term memory network to obtain a fused output sequence, and finally obtains a maximized label sequence according to the conditional random field to complete the recognition of Chinese named entities. The present invention can integrate the radicals, speech and character features of Chinese characters into a unified framework, and the proposed Chinese named entity recognition method that integrates Chinese character roots, speech and character features promotes further development in this field. The present invention solves the limitations of traditional Chinese named entity recognition methods caused by ignoring the rich semantic information and subtle differences in speech in Chinese characters. By introducing additional Chinese character features, the present invention enhances the model's ability to accurately identify and classify named entities in complex Chinese texts, and promotes the development and progress of natural language processing technology. The present invention not only contributes to the field of Chinese named entity recognition, but also further lays a foundation for the research and development of multilingual and multimodal named entity recognition.

[0107] Example 2

[0108] Based on the above embodiments, the present invention provides an evaluation experiment of a Chinese named entity recognition method integrating Chinese character roots, speech and character features, which specifically includes:

[0109] In the experiment, the Chinese benchmark dataset Resume is used as the dataset. The corpus of this dataset comes from the resume data of Sina Finance, covering a large number of formal expressions. The training set includes 3,800 text data and 13,400 named entities, the validation set includes 460 text data and 1,600 named entities, and the test set includes 480 text data and 1,500 named entities. The Resume dataset contains eight types of named entities: CONT (country), EDU (education background), LOC (location), PER (person), ORG (organization), PRO (occupation), RACE (ethnicity) and TITLE (job title). The Resume dataset was selected to evaluate the performance of the model in a formal context.

[0110] In the experiment, entity-level indicators Precision, Recall, and F1, which are commonly used in Chinese named entity recognition, are used as model performance evaluation criteria to evaluate the effectiveness of the present invention.

[0111] The formulas corresponding to the three evaluation criteria are as follows:

[0112] Precision refers to the measure of the accuracy of the model. It is the ratio between true positives and all identified positives, including true positives (TP) and false positives (FP). The precision metric shows how many predicted entities are correctly labeled:

[0113]

[0114] Among them, Precision is the accuracy, TP is the true positive, and FP is the false positive.

[0115] Recall refers to the ability to measure the model's prediction of the actual positive class. It is the ratio between the predicted true positives and the actual labeled true positives (including true positives (TP) and false negatives (FN)). The recall metric reveals how many of the predicted entities are correct:

[0116]

[0117] Among them, Recall is the recall rate and FN is the false negative.

[0118] F1 refers to the function of precision and recall. It is used to strike a balance between precision and recall:

[0119]

[0120] Next, experiments will be conducted on the test set using the formulas given for precision, recall, and F1, and the Chinese named entity recognition method proposed in the present invention will be compared with other existing methods to reasonably evaluate the effectiveness of the method proposed in the present invention.

[0121] Based on the preprocessing of Chinese character information, Python language is used to Figure 3 The Chinese named entity recognition model shown in the figure is implemented and applied to the given test set. In order to more reasonably analyze the effectiveness of the Chinese named entity recognition method proposed in the present invention, the evaluation criteria defined by precision, recall and F1 are applied to compare the present invention with previous research results and other existing Chinese named entity recognition methods. Table 1 summarizes the relevant evaluation results of each method for Chinese named entity recognition on the test set.

[0122] Table 1 Evaluation results of several Chinese named entity recognition methods on the Resume dataset (%)

[0123]

[0124] As can be seen from Table 1, the Recall and F1 values ​​obtained by the present invention are improved compared with the previous eight methods, and the Precision is improved compared with the previous seven methods. It can be seen that the introduction of the pre-training model makes the character embedding richer and more dynamic, and can more effectively learn and capture semantic information. This result confirms the importance of introducing a pre-training model to improve the performance of Chinese named entities. On this basis, by giving play to the pictographic characteristics of Chinese characters themselves and integrating the roots and phonetic features of embedded Chinese characters, the model can capture the semantic information of Chinese characters in a specific context, better identify entity boundary features, and thus more comprehensively understand the meaning and contextual information of Chinese text. Due to the combination of powerful information of character embedding and richer roots and phonetic features, the model shows better performance. This result confirms the importance of integrating the roots and phonetic features of embedded Chinese characters to improve the performance of Chinese named entities.

[0125] In order to more reasonably prove the effectiveness of introducing external features of Chinese characters in improving model performance, an ablation experiment was conducted on the introduced external features of Chinese characters. The specific experimental results are shown in Table 2. As can be seen from Table 2, when the root feature is removed, the F1 value drops by -0.36%; when the speech feature is removed, the F1 value drops by -0.30%. When the root and speech features are removed at the same time, the F1 value drops by -0.55%. After removing the root feature, the model cannot recognize Chinese characters with the same or similar roots, which weakens the ability to capture the semantic information of Chinese characters, resulting in a decrease in the F1 value of the model. After removing the speech feature, the model cannot recognize heterophonetic characters and cannot accurately determine the starting and ending positions of heterophonetic character entities, resulting in ambiguity in entity boundary detection, thereby reducing the accuracy of entity recognition and causing a decrease in the F1 value of the model. When the word roots and phonetic features are removed at the same time, the impact of the above two problems is superimposed. The model can neither recognize Chinese characters with the same or similar word roots, nor accurately determine the starting and ending positions of heterophonetic character entities. The model has problems in separation boundary detection and entity category determination, resulting in a significant decrease in the F1 value.

[0126] Table 2 Ablation experiment results of the present invention on the Resume dataset (%)

[0127]

[0128] This paper takes the fusion of embedded Chinese character external features as the starting point, and studies the Chinese named entity recognition technology based on convolutional neural networks and bidirectional long short-term memory networks; proposes a Chinese named entity recognition method that integrates Chinese character roots, voice and character features, and verifies the feasibility and effectiveness of the method through experimental tests.

[0129] Example 3

[0130] Based on the above embodiments, Figure 4As shown, the embodiment of the present invention provides a Chinese named entity recognition system integrating Chinese character roots, voice and character features, including:

[0131] The preprocessing module is used to preprocess a given character sequence to obtain multiple features.

[0132] The feature extraction module is used to process multiple features using convolutional neural networks and maximum pooling technology to obtain multiple feature vectors.

[0133] The fusion module is used to fuse multiple feature vectors and input the fused vectors into the RoBERTa model and the bidirectional long short-term memory network in sequence to obtain a fused output sequence.

[0134] The recognition module is used to process the output sequence according to the conditional random field to obtain the maximized label sequence and complete the recognition of Chinese named entities.

[0135] It should be noted that the Chinese named entity recognition system that integrates Chinese character roots, voice and character features provided in the embodiment of the present invention is to implement the Chinese named entity recognition method that integrates Chinese character roots, voice and character features in the above-mentioned embodiment. Its specific functions can be referred to the above-mentioned method embodiments and will not be repeated here.

[0136] Example 4

[0137] Corresponding to the above method, an embodiment of the present invention provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the Chinese named entity recognition method that integrates Chinese character roots, voice and character features as in the above embodiment.

[0138] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute a Chinese named entity recognition method that integrates Chinese character roots, voice and character features as described in the above embodiment.

[0139] In summary, the present invention analyzes the characteristics of Chinese characters themselves, and aims at the limitations faced by the current Chinese named entity recognition method. First, according to the characteristics of Chinese characters themselves, Chinese characters are decomposed into structural components to construct Chinese character root-level features, and Chinese characters are decomposed into pinyin to construct Chinese character voice features. On this basis, a convolutional neural network is used to extract Chinese character features, local associations are captured by convolution, and the root embedding vector and voice embedding vector are generated by the maximum pooling technology, which are then spliced ​​with the character embedding vector to form a fused embedding vector, and then the fused embedding vector is fused using a fully connected layer. Secondly, the fused embedding vector is input into a bidirectional long short-term memory network for context modeling. Finally, a conditional random field is used for sequence prediction and the label sequence score is returned to realize the recognition of Chinese named entities. The comparative experiment shows that the present invention improves the recognition efficiency and recognition accuracy of separation boundary detection and entity category in the Chinese environment, has better recognition effect, and provides a research idea for more reasonable selection of Chinese character external feature information and further improving the recognition effect and efficiency of the Chinese named entity recognition method.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A Chinese named entity recognition method integrating Chinese character roots, phonetics and character features, characterized in that: include: Step 1: Preprocess the given character sequence to obtain multiple features; Step 2: Use convolutional neural network and maximum pooling technology to process multiple features to obtain multiple feature vectors; Step 3: Fuse multiple feature vectors and input the fused vectors into the RoBERTa model and the bidirectional long short-term memory network in sequence to obtain the fused output sequence; Step 4: Process the output sequence according to the conditional random field to obtain the maximized label sequence and complete the recognition of Chinese named entities.

2. The Chinese named entity recognition method integrating Chinese character roots, pronunciation and character features according to claim 1 is characterized in that: The given character sequence is preprocessed to obtain multiple features, including: Decompose the Chinese characters in a given character sequence into structural components according to the Chinese character decomposition dictionary, and use them as root features of the Chinese characters; According to the open source pypinyin library, the Chinese characters in a given character sequence are converted into corresponding pinyin, which is used as the phonetic feature of the Chinese characters; The Chinese characters in the given character sequence are taken as the character features of the Chinese characters.

3. The Chinese named entity recognition method integrating Chinese character roots, speech and character features according to claim 1 is characterized in that: The second step comprises: Embed the root features, speech features and character features into corresponding embedding matrices respectively to obtain the original root embedding vector, the original speech embedding vector and the original character embedding vector; According to the convolutional neural network, feature extraction is performed on the original root feature embedding matrix and the original speech feature embedding matrix respectively; The output of the convolutional neural network is processed using the maximum pooling technique to obtain the root embedding vector and the speech embedding vector.

4. The Chinese named entity recognition method integrating Chinese character roots, speech and character features according to claim 3 is characterized in that: The feature extraction is performed on the original root feature embedding matrix and the original speech feature embedding matrix respectively according to the convolutional neural network, which is specifically expressed by the following formula: Among them, C r (i, j) is the i-th convolution output in the root feature map, C p (i,j) is the i-th convolution output in the speech feature map, k is the height of the convolution window, d r is the dimension of the original root embedding vector, W r (m,n) is the weight of the root convolution kernel in the mth row and nth column, W p (m,n) is the weight of the speech convolution kernel in the mth row and nth column, R is the root embedding matrix, b r is the bias parameter of the root convolution kernel, j is C r (i, j) and C p (i,j) dimension index, P is the speech embedding matrix, b p is the bias parameter of the speech convolution kernel, d p is the dimension of the original speech embedding vector; The maximum pooling technique is used to process the output of the convolutional neural network, which is expressed by the following formula: Among them, h r (i, j) is the root embedding vector of the root feature map at position (i, j) after the maximum pooling operation, h p (i, j) is the speech embedding vector of the speech feature map at position (i, j) after the maximum pooling operation, C r (i+m, j) and C p (i+m, j) are the values ​​of the root convolution output and the speech convolution output in the pooling window respectively, and k is the height of the convolution window.

5. The Chinese named entity recognition method integrating Chinese character roots, pronunciation and character features according to claim 3 is characterized in that: The fusing of multiple feature vectors specifically includes: Concatenate the root embedding vector, the phonetic embedding vector, and the original character embedding vector to obtain a fusion vector; The fused vector is linearly transformed using a fully connected layer to obtain a fused embedding vector.

6. The Chinese named entity recognition method integrating Chinese character roots, pronunciation and character features according to claim 5 is characterized in that: The linear transformation of the fusion vector using the fully connected layer is expressed as follows: E′ i =Fusion(h r ,x i ,h p )=W(E i )+b in, is the weight matrix of the fully connected layer, b∈R out is the bias vector, d out is the dimension of the output vector, E′ i To fuse the embedding vector, is a vector concatenation operation, Fusion is a fusion operation, h r is the root embedding vector, h p is the speech embedding vector, x i is the original character embedding vector, E i is the fusion vector.

7. The Chinese named entity recognition method integrating Chinese character roots, speech and character features according to claim 1 is characterized in that: The step 4 specifically includes: Obtain the label sequence based on the conditional random field and the output sequence and construct a scoring function, and calculate the probability of the output sequence generating the label sequence based on the scoring function; According to the Viterbi Algorithm algorithm, the label sequence that maximizes the score function is found among all possible label sequences.

8. A Chinese named entity recognition system integrating Chinese character roots, phonetics and character features, characterized in that: include: A preprocessing module is used to preprocess a given character sequence to obtain multiple features; A feature extraction module is used to process multiple features using convolutional neural networks and maximum pooling technology to obtain multiple feature vectors; A fusion module is used to fuse multiple feature vectors and input the fused vectors into the RoBERTa model and the bidirectional long short-term memory network in sequence to obtain a fused output sequence; The recognition module is used to process the output sequence according to the conditional random field to obtain the maximized label sequence and complete the recognition of Chinese named entities.

9. An electronic device, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the Chinese named entity recognition method that integrates Chinese character roots, voice and character features as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the Chinese named entity recognition method integrating Chinese character roots, voice and character features as described in any one of claims 1 to 7.

Images