Named entity recognition method and device, computer readable storage medium and processor
By dividing text data into static and non-static vectors and inputting them into two channels of the recognition model, and combining convolutional neural networks, bidirectional long short-term memory networks, and conditional random fields, the word embedding bias problem in named entity recognition is solved, thereby improving recognition performance.
Patent Information
- Application Number
- CN202011630033.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-30
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2040-12-30
AI Technical Summary
In existing named entity recognition technologies, word embedding bias is a common problem, resulting in low recognition performance.
Named entity recognition is performed by dividing text data into a first text vector and a second text vector, which are then input into two channels of the recognition model. Feature extraction is performed using a convolutional neural network and a bidirectional long short-term memory network based on an attention mechanism, and the recognition results are generated using a conditional random field.
By avoiding word embedding bias, the accuracy and performance of named entity recognition are improved.
Smart Images

Figure CN112699684B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information extraction, and in particular to a method and device for named entity recognition, a computer-readable storage medium and a processor. Background Art
[0002] With the increasing popularity of the Internet and the continuous development of computer technology, information extraction has received widespread attention. Information Extraction (IE) mainly structures the unstructured information in the text, extracts specific entities or events from the text, and helps users automatically classify, extract and reconstruct the content of massive text. Named Entity Recognition (NER) is an important subtask of information extraction. Among them, Chinese named entity recognition refers to the computer identifying all the named entities in a piece of Chinese text data according to their categories.
[0003] In recent years, deep learning has been widely used in natural language processing tasks. The deep learning method can combine corpus annotation with word vector features. By reducing the proportion of artificial features in the model, the dependence of the named entity recognition system on large corpora can be reduced; and the complexity of the rule method can be reduced through probabilistic statistics, which effectively improves the performance of the model. The related technology provides a named entity recognition method that combines convolutional neural networks (CNN) and long short-term memory networks (LSTM). First, the text is represented as a word vector, and the local feature information and important features are captured and retained through the subsequent convolutional layer and pooling layer, and then the LSTM layer is entered to extract the global dependency relationship. Finally, the final label output is obtained through the fully connected layer and CRF layer. However, the above scheme is prone to word embedding bias problems, resulting in low named entity recognition performance.
[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention
[0005] Embodiments of the present invention provide a method and apparatus for named entity recognition, a computer-readable storage medium, and a processor to at least solve the technical problem in the related art that word embedding bias is prone to occur in the process of named entity recognition, resulting in low recognition performance.
[0006] According to one aspect of an embodiment of the present invention, a method for named entity recognition is provided, including: acquiring text data; performing feature mapping on the text data to obtain a first text vector and a second text vector; and inputting the first text vector and the second text vector into two channels of a recognition model respectively to perform named entity recognition to obtain a recognition result of the text data.
[0007] Optionally, the first text vector and the second text vector are respectively input into two channels of the recognition model for named entity recognition to obtain a recognition result of the text data, including: inputting the first text vector into a convolutional neural network for feature extraction to obtain a local feature vector of the text data; inputting the second text vector into a bidirectional long short-term memory network based on an attention mechanism for feature extraction to obtain a temporal feature vector of the text data; fusing the local feature vector and the temporal feature vector to obtain a target feature vector; and inputting the target feature vector into a conditional random field for recognition to obtain a recognition result.
[0008] Optionally, the second text vector is input into a bidirectional long short-term memory network based on an attention mechanism for feature extraction to obtain a temporal feature vector of the text data, including: inputting the second text vector into a bidirectional long short-term memory network for feature extraction to obtain an initial feature vector of the text data; inputting the initial feature vector into an attention mechanism layer for weight allocation to obtain a temporal feature vector.
[0009] Optionally, the local feature vector and the time series feature vector are fused to obtain a target feature vector, including: inputting the local feature vector and the time series feature vector into a concatenation layer for fusion to obtain a fused feature vector; inputting the fused feature vector into a fully connected layer for dimensionality conversion to obtain a target feature vector.
[0010] Optionally, the target feature vector is input into a conditional random field for recognition to obtain a recognition result, including: processing the target feature vector by a Viterbi algorithm to obtain a target labeling sequence; and obtaining a recognition result based on the target labeling sequence.
[0011] Optionally, the first text vector is a vector that changes when the recognition model is trained by using the vector as a parameter, and the second text vector is a vector that does not change after being generated by the recognition model.
[0012] Optionally, feature mapping is performed on the text data to obtain a first text vector and a second text vector, including: feature mapping is performed on the text data to obtain a vector for each word in the text data; and the vector for each word is set to obtain a first text vector and a second text vector.
[0013] Optionally, the method further includes: acquiring annotated training corpus; performing feature mapping on the annotated training corpus to obtain a first vector and a second vector; and training the recognition model using the first vector and the second vector.
[0014] Optionally, obtaining annotated training corpus includes: obtaining training corpus; annotating the training corpus to obtain annotated training corpus, wherein the annotation information of the training corpus includes: position annotation and type annotation of words in the training corpus.
[0015] According to another aspect of an embodiment of the present invention, a named entity recognition device is also provided, including: an acquisition module for acquiring text data; a mapping module for performing feature mapping on the text data to obtain a first text vector and a second text vector; and a recognition module for respectively inputting the first text vector and the second text vector into two channels of a recognition model for named entity recognition to obtain a recognition result of the text data.
[0016] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided. The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned named entity recognition method.
[0017] According to another aspect of an embodiment of the present invention, a processor is further provided, and the processor is used to run a program, wherein the program executes the above-mentioned named entity recognition method when running.
[0018] In an embodiment of the present invention, after acquiring the text data, feature mapping can be performed on the text data to obtain a first text vector and a second text vector, and the first text vector and the second text vector are further input into two channels of a recognition model for named entity recognition, respectively, to obtain a recognition result of the text data, thereby achieving the purpose of named entity recognition. It is easy to notice that since the text data can be divided into a first text vector and a second text vector, and the two text vectors are input into a multi-channel recognition model for recognition, the deviation of word embedding is avoided, a more accurate text vector representation is obtained, and the effect of improving the performance of named entity recognition is achieved, thereby solving the technical problem that the word embedding deviation problem is prone to occur in the process of named entity recognition in the related technology, resulting in low recognition performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0020] Figure 1 is a flowchart of a method for named entity recognition according to an embodiment of the present invention;
[0021] Figure 2 is a flowchart of a named entity recognition process according to an embodiment of the present invention;
[0022] Figure 3 is a flow chart of a recognition model training process according to an embodiment of the present invention;
[0023] Figure 4 is a schematic diagram of a named entity recognition device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0024] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0026] The technical names or technical terms appearing in the embodiments of the present invention are explained as follows:
[0027] NLP: Natural Language Processing, or natural language processing, is a field of transformation between computer science and linguistics that studies theories and methods that enable effective communication between humans and computers using natural language.
[0028] NER: The main purpose can be to extract entities from unstructured text. These entities usually include names of people, places, organizations, time, etc.
[0029] RNN: Recurrent Neural Network, recurrent neural network, can be mainly used to process sequence data and prediction problems, such as natural language.
[0030] LSTM: Long Short-term Memory, a type of recurrent neural network, can be used to solve the long-term dependency problem of RNN.
[0031] IDCNN: Iterated Dilated Convolutional Neural Networks, iterative dilated convolutional neural network.
[0032] CRF: Conditional Random Fields, conditional random fields, can be used to model the conditional probability distribution of another set of output random variables under a given set of input random variables. It is a discriminant probabilistic undirected graph model.
[0033] Example 1
[0034] According to an embodiment of the present invention, a method for named entity recognition is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0035] Figure 1 is a flowchart of a method for named entity recognition according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:
[0036] Step S102, obtaining text data.
[0037] The text data in the above steps may be text data that needs to be recognized by named entities, and may be Chinese text or English text. In the embodiment of the present invention, Chinese text is taken as an example for description.
[0038] In an optional embodiment, when a user needs to perform named entity recognition, the user can input text data that needs to be recognized by the named entity.
[0039] Step S104, performing feature mapping on the text data to obtain a first text vector and a second text vector.
[0040] In an optional embodiment, the text data input by the user can first be segmented to divide the text data into multiple words, and feature mapping is performed for each word, and each word is represented by a word vector to achieve mathematization of the language and facilitate computer processing.
[0041] Optionally, the first text vector mentioned above may be a vector that regards the vector as a parameter and changes during network training, and may also be called a non-static text vector; the second text vector may be a vector that does not change after being generated by the model, and may also be called a static text vector.
[0042] In an optional embodiment, there is already a word vector dictionary with good effect on the Internet. The dictionary can be directly used for feature mapping to obtain text vectors. Therefore, the vector obtained by feature mapping through the dictionary can be used as a static text vector. In addition, for different named entity recognition needs, it is often necessary to train with different training corpora. However, the word vector dictionary that already exists on the Internet may not cover all training corpora. Therefore, static text vectors have certain limitations and may lead to word embedding bias problems. In order to avoid the above problems, the uncovered text vectors can be used as non-static text vectors and continuously improved during the training process. By fusing the word vectors of the two channels, not only can the word embedding problem be alleviated, but also specific task information can be captured to obtain a more accurate text vector representation.
[0043] Step S106: input the first text vector and the second text vector into two channels of a recognition model respectively to perform named entity recognition, and obtain a recognition result of the text data.
[0044] The recognition model in the above steps can be composed of CNN, BiLSTM and attention mechanism Attention, wherein CNN and BiLSTM are executed in parallel, and the output result of BiLSTM is input into Attention for processing. At this time, the input of CNN and BiLSTM constitutes the above two channels.
[0045] The recognition result in the above steps may be a tag sequence corresponding to the text data, and the tag sequence may include the classification result of the word, the type of the word, etc., but is not limited thereto.
[0046] In an optional embodiment, if only static text vectors are used to train the recognition model, the limitations of static text vectors in representing text will be magnified during the training process, resulting in word embedding bias problems; if only non-static text vectors are used, the initialization of the non-static text vectors in representing text will affect the recognition results and convergence speed. Therefore, static text vectors and non-static text vectors can be input separately through a dual-channel recognition model, and the final recognition result can be obtained by processing the two text vectors.
[0047] In the above embodiment of the present invention, after acquiring the text data, feature mapping can be performed on the text data to obtain a first text vector and a second text vector, and the first text vector and the second text vector are further input into two channels of the recognition model for named entity recognition, to obtain the recognition result of the text data, thereby achieving the purpose of named entity recognition. It is easy to notice that since the text data can be divided into a first text vector and a second text vector, and the two text vectors are input into a multi-channel recognition model for recognition, the deviation of word embedding is avoided, and a more accurate text vector representation is obtained, thereby achieving the effect of improving the performance of named entity recognition, thereby solving the technical problem that the word embedding deviation problem is prone to occur in the process of named entity recognition in the related technology, resulting in low recognition performance.
[0048] Optionally, in the above embodiment of the present invention, the first text vector and the second text vector are respectively input into two channels of the recognition model for named entity recognition to obtain a recognition result of the text data, including: inputting the first text vector into a convolutional neural network for feature extraction to obtain a local feature vector of the text data; inputting the second text vector into a bidirectional long short-term memory network based on an attention mechanism for feature extraction to obtain a temporal feature vector of the text data; fusing the local feature vector and the temporal feature vector to obtain a target feature vector; and inputting the target feature vector into a conditional random field for recognition to obtain a recognition result.
[0049] In the embodiment of the present invention, for the convolutional neural network, a filter with a length of 3 can be set, the number of filters is 128, the filter can slide over the input feature matrix, the length of the sliding window is the length of the filter, and each length of the filter can obtain 128 feature matrices, called feature maps, and the relu activation function is used to make the 128 feature maps identified as a single-column vector, thereby obtaining the above-mentioned local feature vector c = [c1, c2, ..., c n ].
[0050] It should be noted that, since the transmission of the hidden layer state of the standard long short-term memory neural network is unidirectional from front to back, the memory network at this moment can only see historical information, but has no way to see future information. However, in some problems, the output at the current moment may be related not only to the previous state, but also to the subsequent state. Therefore, in the embodiment of the present invention, a bidirectional long short-term memory model is used, which can capture both positive semantic information and reverse semantic information.
[0051] Furthermore, by introducing the Attention mechanism, the model can pay more attention to a certain word. Compared with the previous encoder-decoder model, the biggest difference of the Attention model is that it no longer requires the encoder to encode all input information into a vector of fixed length. On the contrary, the encoder needs to encode the input into a sequence of vectors, and when decoding, each step will selectively select a subset from the vector sequence for further processing. In this way, when generating each output, it can make full use of the information carried by the input sequence.
[0052] In an optional embodiment, Figure 2 As shown in the figure, the non-static text vector can be input into the CNN network to obtain the local feature vector, and the static text vector can be input into the BiLSTM network to extract the temporal feature information of the input text. Then, the Attention mechanism is used to weight the temporal feature information, so that the feature weights of important words are amplified to obtain the temporal feature vector. The feature vector extracted by CNN is further fused with the feature vector extracted by BiLSTM-Attention at the splicing layer, so that the fused target feature vector has temporal features and local features. The fused target feature vector is input into the CRF, and the optimal sequence is solved by the Viterbi algorithm to label the entire sentence for entity recognition.
[0053] Optionally, in the above embodiment of the present invention, the second text vector is input into a bidirectional long short-term memory network based on an attention mechanism for feature extraction to obtain a temporal feature vector of the text data, including: inputting the second text vector into a bidirectional long short-term memory network for feature extraction to obtain an initial feature vector of the text data; inputting the initial feature vector into the attention mechanism layer for weight allocation to obtain a temporal feature vector.
[0054] It should be noted that for the LSTM model, the long and short-term memory model at time t is as follows:
[0055] i t =σ(W i [h t-1 , x t ]+b i ),
[0056] f t =σ(W f [h t-1 , x t ]+b f ),
[0057] o t =σ(W o [h t-1, x t ]+b fo ),
[0058]
[0059]
[0060] h t =o t *tanh(c t ),
[0061] Among them, σ is the sigmoid function, tanh is the hyperbolic tangent activation function, W is the weight matrix, b is the bias vector, W and b are used as network parameters for training, c t Identifies the state of the memory cell. Indicates the state at time t, which is the intermediate state obtained by the current input. Its main function is to update the state at the current time. t is the output at time t.
[0062] In the embodiment of the present invention, the output of the hidden layer of BiLSTM (i.e., the above-mentioned initial feature vector) is defined as follows: in, and They represent the output representation of the forward LSTM and the backward LSTM at time t respectively. Therefore, the final output of BiLSTM is composed of the past hidden information and the future hidden information.
[0063] Furthermore, the attention value of each word in the text is calculated through the attention mechanism, according to the current hidden state h of the hidden layer t The hidden layer representation can be obtained:
[0064] u t =tanh(W w h t +b w ),
[0065] Among them, W w is the weight matrix of the hidden layer, b w is the bias value.
[0066] Assign a different weight matrix to each output of the previous layer:
[0067]
[0068] Among them, μ w is the word-level context vector.
[0069] For the current hidden state h of the hidden layer t and the weight matrix αt By weighted summation, we can obtain the time series feature vector s containing the importance information of each word in the text:
[0070] s=∑ t α t h t .
[0071] Optionally, in the above embodiment of the present invention, the local feature vector and the time series feature vector are fused to obtain a target feature vector, including: inputting the local feature vector and the time series feature vector into a concatenation layer for fusion to obtain a fused feature vector; inputting the fused feature vector into a fully connected layer for dimensionality conversion to obtain a target feature vector.
[0072] In an optional embodiment, Figure 2 As shown in the figure, the local feature vector c output by the CNN layer and the temporal feature vector s output by the BiLSTM-Attention layer can be fused through the concatenation layer to obtain the fused feature vector a:
[0073]
[0074] The fused feature vector is further input into the fully connected layer, and the d-dimensional vector obtained by the concatenation layer is converted into a k-dimensional vector with the same dimension as the number of defined labels, that is, the target feature vector. The conversion formula is as follows:
[0075] l=w·a+b.
[0076] Optionally, in the above embodiment of the present invention, the target feature vector is input into the conditional random field for recognition to obtain a recognition result, including: processing the target feature vector through the Viterbi algorithm to obtain a target labeling sequence; and obtaining a recognition result based on the target labeling sequence.
[0077] It should be noted that CRF, as one of the most reliable sequence annotations, has achieved good performance in named entity tasks. In sequence annotation, the CRF model acts on the structure of the entire sentence, rather than an independent single position. The state feature is used in the CRF layer as the state score representation of the current node, and the transition matrix is represented by the transition score from the previous node to the current node. The CRF layer can automatically set some legal constraints between labels. The label of the first word in the sentence can only be "B-" or "O", not "I-"; in the label "B-label I-label I-label" in the sentence, "label" should be the same named entity label. These constraints are automatically learned by the CRF model from the training data. Therefore, the advantage of CRF is that it can model implicit states and learn the characteristics of state sequences. By adding constraints between labels, it can better conform to the correctness of language logic and ultimately generate a language model that conforms to humans.
[0078] For a given sentence x = [x1, x2, ..., x n ], the annotation sequence of the sentence is y = [y1, y2, ..., y n ]'s score:
[0079]
[0080] The score s(x, y) consists of two parts: the labeling score P and the transition score A between labels, where A is the transition score matrix, A i,j represents the transfer score between label i and label j (y0, y n+1 Represent the beginning and end of the sentence respectively), P is the score matrix output by the BiLSTM layer (n×k dimensions, k is the number of label types), P i,j It means that x=[x1,x2,…,x n ] is the score of the i-th word labeled with the j-th label. Then x=[x1,x2,…,x n ] is marked as y = [y1, y2, ..., y n The probability of ] is:
[0081]
[0082] Among them, Y x Represents a sentence x = [x1, x2, ..., x n ] corresponds to all the annotation schemes.
[0083] During training, the named entity recognition model is trained by maximizing the logarithmic probability function log(p(y|x)); during prediction, the Viterbi algorithm is used to obtain the annotation sequence with the highest score:
[0084]
[0085] Optionally, in the above embodiment of the present invention, feature mapping is performed on the text data to obtain a first text vector and a second text vector, including: feature mapping is performed on the text data to obtain a vector for each word in the text data; and the vector of each word is set to obtain a first text vector and a second text vector.
[0086] In an optional embodiment, the text data can be segmented to obtain each word, and the words can be further converted into vectors to obtain the vector of each word, and then set manually to determine the static text vector and the non-static text vector, where the non-static text vector is represented by v1 and the static text vector is represented by v2.
[0087] Optionally, in the above embodiment of the present invention, the method further includes: obtaining annotated training corpus; performing feature mapping on the annotated training corpus to obtain a first vector and a second vector; and training the recognition model using the first vector and the second vector.
[0088] In an optional embodiment, Figure 3 As shown in the figure, the recognition model training process is as follows: obtain the labeled training corpus, convert the words in the corpus into non-static text vectors and static text vectors, input the non-static text vectors and static text vectors into the recognition model based on CNN-BiLSTM-Attention, output the BIO label, and use the back-propagation method to update the parameters in the recognition model.
[0089] Optionally, in the above embodiment of the present invention, obtaining annotated training corpus includes: obtaining training corpus; annotating the training corpus to obtain annotated training corpus, wherein the annotation information of the training corpus includes: position annotation and type annotation of words in the training corpus.
[0090] In an optional embodiment, annotating the corpus is a manual annotation process, and the corpus positions the training corpus in the manner of BIO (Begin, Inside, Other), where Begin indicates the first character mark belonging to an entity word; Inside indicates an entity word but not the first character; Other indicates a character that does not belong to an entity word. If the character is an entity word, a corresponding type annotation (i.e., the above-mentioned label) is added to indicate the type of the entity word, where the type annotation includes but is not limited to: the type annotation of a person's name is PER, the type annotation of an organization name is ORG, and the type annotation of a place name is LOC. If an entity recognition unit is the beginning of an entity, it is marked as (B-lable); if an entity recognition unit is an entity intermediate word, it is marked as (I-lable).
[0091] Example 2
[0092] According to an embodiment of the present invention, a named entity recognition device is also provided, which can execute the named entity recognition method in the above embodiment. The specific implementation scheme and application scenario are the same as those in the above embodiment and will not be described in detail here.
[0093] Figure 4 is a schematic diagram of a named entity recognition device according to an embodiment of the present invention. Figure 4 As shown, the device comprises:
[0094] An acquisition module 42, used for acquiring text data;
[0095] A mapping module 44, used for performing feature mapping on the text data to obtain a first text vector and a second text vector;
[0096] The recognition module 46 is used to input the first text vector and the second text vector into two channels of the recognition model respectively to perform named entity recognition, so as to obtain a recognition result of the text data.
[0097] Optionally, in the above embodiment of the present invention, the recognition module includes: a first extraction unit, used to input the first text vector into a convolutional neural network for feature extraction to obtain a local feature vector of the text data; a second extraction unit, used to input the second text vector into a bidirectional long short-term memory network based on an attention mechanism for feature extraction to obtain a temporal feature vector of the text data; a fusion unit, used to fuse the local feature vector and the temporal feature vector to obtain a target feature vector; and a recognition unit, used to input the target feature vector into a conditional random field for recognition to obtain a recognition result.
[0098] Optionally, in the above embodiment of the present invention, the second extraction unit includes: an extraction subunit, used to input the second text vector into a bidirectional long short-term memory network for feature extraction to obtain an initial feature vector of the text data; and an allocation unit, used to input the initial feature vector into an attention mechanism layer for weight allocation to obtain a time series feature vector.
[0099] Optionally, in the above embodiment of the present invention, the fusion unit includes: a fusion subunit, used to input the local feature vector and the time series feature vector into the splicing layer for fusion to obtain a fused feature vector; a conversion subunit, used to input the fused feature vector into the fully connected layer for dimensional conversion to obtain a target feature vector.
[0100] Optionally, in the above embodiment of the present invention, the recognition module includes: a processing unit, used to process the target feature vector by a Viterbi algorithm to obtain a target labeling sequence; and a recognition unit, used to obtain a recognition result based on the target labeling sequence.
[0101] Optionally, in the above embodiment of the present invention, the mapping module includes: a mapping unit, used to perform feature mapping on the text data to obtain a vector of each word in the text data; a setting unit, used to set the vector of each word to obtain a first text vector and a second text vector.
[0102] Optionally, in the above embodiment of the present invention, the acquisition module is also used to acquire labeled training corpus; the mapping module is also used to perform feature mapping on the labeled training corpus to obtain a first vector and a second vector; the device also includes: a training module, used to train the recognition model using the first vector and the second vector.
[0103] Optionally, in the above embodiment of the present invention, the acquisition module includes: an acquisition unit, used to acquire training corpus; and a labeling unit, used to label the training corpus to obtain labeled training corpus, wherein the labeling information of the training corpus includes: position labeling and type labeling of words in the training corpus.
[0104] Example 3
[0105] According to an embodiment of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the named entity recognition method in the above-mentioned embodiment 1.
[0106] Example 4
[0107] According to an embodiment of the present invention, a processor is further provided, and the processor is used to run a program, wherein the named entity recognition method in the above-mentioned embodiment 1 is executed when the program is run.
[0108] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0109] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0110] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0111] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0112] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0113] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.
[0114] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A method for named entity recognition, It is characterized in that include: Get text data; Performing feature mapping on the text data to obtain a first text vector and a second text vector; Inputting the first text vector and the second text vector into the first channel and the second channel of the recognition model respectively for named entity recognition, and obtaining the recognition result of the text data, wherein the first text vector is a vector that changes when the recognition model is trained by taking the vector as a parameter, and the second text vector is a vector that does not change after being generated by the recognition model, the first channel is a CNN model, and the second channel is a BiLSTM-Attention model; The first text vector and the second text vector are respectively input into the first channel and the second channel of the recognition model for named entity recognition to obtain the recognition result of the text data, including: inputting the first text vector into a convolutional neural network for feature extraction to obtain a local feature vector of the text data; inputting the second text vector into a bidirectional long short-term memory network based on an attention mechanism for feature extraction to obtain a temporal feature vector of the text data; fusing the local feature vector and the temporal feature vector to obtain a target feature vector; inputting the target feature vector into a conditional random field for recognition to obtain the recognition result; Among them, the second text vector is input into a bidirectional long short-term memory network based on an attention mechanism for feature extraction to obtain a temporal feature vector of the text data, including: inputting the second text vector into a bidirectional long short-term memory network for feature extraction to obtain an initial feature vector of the text data; inputting the initial feature vector into an attention mechanism layer for weight allocation to obtain the temporal feature vector.
2. The method according to claim 1, It is characterized in that The local feature vector and the time series feature vector are fused to obtain a target feature vector, including: Inputting the local feature vector and the temporal feature vector into a concatenation layer for fusion to obtain a fused feature vector; The fused feature vector is input into the fully connected layer for dimension conversion to obtain the target feature vector.
3. The method according to claim 1, It is characterized in that Inputting the target feature vector into a conditional random field for recognition to obtain the recognition result includes: Processing the target feature vector by using a Viterbi algorithm to obtain a target labeling sequence; Based on the target annotation sequence, the recognition result is obtained.
4. The method according to claim 1, It is characterized in that Performing feature mapping on the text data to obtain a first text vector and a second text vector includes: Performing feature mapping on the text data to obtain a vector for each word in the text data; The vector of each word is set to obtain the first text vector and the second text vector.
5. The method according to claim 1, It is characterized in that The method further comprises: Get the labeled training corpus; Performing feature mapping on the labeled training corpus to obtain a first vector and a second vector; The recognition model is trained using the first vector and the second vector.
6. The method according to claim 5, It is characterized in that Get the labeled training corpus, including: Get training corpus; The training corpus is annotated to obtain the annotated training corpus, wherein the annotation information of the training corpus includes: position annotation and type annotation of words in the training corpus.
7. A named entity recognition device, It is characterized in that include: An acquisition module, used to acquire text data; A mapping module, used for performing feature mapping on the text data to obtain a first text vector and a second text vector; A recognition module, used for inputting the first text vector and the second text vector into a first channel and a second channel of a recognition model respectively for named entity recognition, to obtain a recognition result of the text data, wherein the first text vector is a vector that changes when the recognition model is trained by taking the vector as a parameter, and the second text vector is a vector that does not change after being generated by the recognition model, the first channel is a CNN model, and the second channel is a BiLSTM-Attention model; The first text vector and the second text vector are respectively input into the first channel and the second channel of the recognition model for named entity recognition to obtain the recognition result of the text data, including: inputting the first text vector into a convolutional neural network for feature extraction to obtain a local feature vector of the text data; inputting the second text vector into a bidirectional long short-term memory network based on an attention mechanism for feature extraction to obtain a temporal feature vector of the text data; fusing the local feature vector and the temporal feature vector to obtain a target feature vector; inputting the target feature vector into a conditional random field for recognition to obtain the recognition result; Among them, the second text vector is input into a bidirectional long short-term memory network based on an attention mechanism for feature extraction to obtain a temporal feature vector of the text data, including: inputting the second text vector into a bidirectional long short-term memory network for feature extraction to obtain an initial feature vector of the text data; inputting the initial feature vector into an attention mechanism layer for weight allocation to obtain the temporal feature vector.
8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for named entity recognition according to any one of claims 1 to 6.
9. A processor, It is characterized in that The processor is used to run a program, wherein the program executes the named entity recognition method according to any one of claims 1 to 6 when running.
