A Chinese agricultural named entity recognition method based on domain dictionary
By constructing a dictionary in the agricultural field and using the BiLSTM-CRF model, combined with the channel attention module, the problems of insufficient feature extraction capabilities and insensitive word segmentation in Chinese agricultural naming entity recognition are solved, and the accuracy and feature extraction capabilities of naming entity recognition are significantly improved. .
Patent Information
- Application Number
- CN202210650342.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-09
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-06-09
AI Technical Summary
In the Chinese agricultural named entity recognition task, the feature extraction ability based on deep learning models is insufficient, and the word segmentation tool is insensitive to agricultural text, resulting in frequent word segmentation errors, affecting the accuracy of entity recognition.
Using a domain dictionary-based method, text is physically annotated through BMESO tags, character vectorization is used using Word2Vec, agricultural dictionary is constructed, and sequence encoding is enhanced through BiLSTM and channel attention modules, decoding is combined with CRF, and vocabulary information is integrated. To improve feature extraction capabilities.
It significantly improves the accuracy and feature extraction ability of Chinese agricultural naming entity recognition, improves the adaptability of word segmentation tools to agricultural texts, and enhances the model's sensitivity and recognition performance to agricultural texts.
Smart Images

Figure CN115017907B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of agronomy, relates to information extraction, and specifically to a Chinese agricultural named entity recognition method based on a domain dictionary. Background Art
[0002] Named entity recognition is the basis and key task of other natural language processing tasks such as relation extraction. It extracts entities with specific meanings from unstructured texts and classifies them into predetermined categories. In the field of Chinese agriculture, Chinese agricultural named entity recognition aims to identify the boundaries and types of agricultural-related entities, such as diseases, pests, pesticides, etc., from unstructured Chinese agricultural texts. It is a key technology for automatically mining knowledge from massive Chinese agricultural texts, and is also the basis for downstream tasks such as building agricultural knowledge graphs and building agricultural intelligent question-answering systems.
[0003] Traditional methods for Chinese agricultural named entity recognition tasks can be divided into rule-based, dictionary matching-based, machine learning-based methods, etc. Although these methods can achieve good results, they rely heavily on time-consuming and labor-intensive pattern matching and feature engineering, and their generalization is not ideal. With the in-depth application of deep learning in the field of named entity recognition, Chinese agricultural named entity recognition has developed to the stage based on deep learning.
[0004] For models based on deep learning, introducing word segmentation information into the model has a positive effect on the recognition of entity boundaries, which is conducive to the model learning more agricultural text features. Due to the characteristics of Chinese agricultural texts such as the large number of professional vocabulary and strong domain characteristics, word segmentation tools are prone to word segmentation errors when processing agricultural texts. Therefore, current research based on deep learning mostly adopts character-based models to avoid introducing word segmentation errors into the model. Therefore, how to improve the problem of word segmentation tools being insensitive to agricultural texts and construct a domain dictionary, integrate agricultural vocabulary information into the model to accurately identify agricultural-related named entities, and construct a model with stronger feature extraction capabilities are technical problems that technicians in this field urgently need to solve. Summary of the invention
[0005] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a Chinese agricultural named entity recognition method based on a domain dictionary to solve the technical problem that the feature extraction capability of the recognition method in the prior art needs to be further improved.
[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions to achieve the above problems:
[0007] A Chinese agricultural named entity recognition method based on a domain dictionary is performed in the following steps:
[0008] Step 1: Annotation of original Chinese agricultural text:
[0009] Select the BMESO tag to annotate the original Chinese agricultural text, and follow the entity category after B, M, E, and S;
[0010] In the BMESO tag, B represents the beginning of an entity word (Begin), M represents the middle of an entity word (Middle), E represents the end of an entity word (End), S represents a single word as an entity word (Single), and O represents other words (Other);
[0011] The entity categories include pesticides, pests, diseases and crops;
[0012] Step 2: Vectorization of original Chinese agricultural text:
[0013] The word embedding technology Word2Vec is used to train Chinese agricultural text into character vectors, and the input sequence X = (x1, x2, x3, ..., x n )∈V c , V c are characters, each of which is represented by a trained dense vector: where e c represents a character embedding lookup table;
[0014] Step 3: Construction of agricultural vocabulary:
[0015] First, the original Chinese agricultural text is segmented into strings using the N-Gram segmentation method to obtain candidate words. Then, the junk strings are filtered out in turn according to the three statistics of word frequency, mutual information and adjacency entropy to obtain a new word set. The new word set is added to the built-in dictionary of the Jieba word segmentation tool for enhancement. The enhanced Jieba word segmentation tool is used to segment the original Chinese agricultural text to obtain the agricultural field word set.
[0016] Step 4: Construction of agricultural dictionary:
[0017] Perform word embedding operation on the agricultural domain word set obtained in step 3 to obtain an agricultural domain dictionary;
[0018] Step 5: Character vector enhancement:
[0019] The agricultural field dictionary constructed in step 4 is applied to the model input sequence S = (x1, x2, x3, ..., x n )∈V c Perform dictionary matching on each character in the dictionary, and integrate the vocabulary vector obtained in step 4 corresponding to the matched agricultural field dictionary into the character vector obtained in step 2 corresponding to the character, so as to realize character vector enhancement;
[0020] Step 6: Sequence encoding:
[0021] BiLSTM is used as the sequence encoding layer, and the enhanced character vector obtained in step 5 is input into the sequence encoding layer for sequence encoding to obtain the sequence encoding result, that is, the feature map H∈R C×W ;
[0022] Step 7: Sequence coding enhancement:
[0023] Construct a channel attention module based on the channel attention module for the feature map H∈R C×W Perform sequence coding enhancement to obtain the sequence coding result K∈R C×W ;
[0024] Step 8: Use CRF to encode the sequence result K∈R C×W For decoding processing, CRF can encode the sequence result K∈R according to the result of the original Chinese agricultural text annotation in step 1. C×W The probability of the BMESO label corresponding to each character in the text is calculated, and the maximum probability is solved by the standard Viterbi algorithm to obtain the label of each character, thus realizing Chinese agricultural named entity recognition.
[0025] The present invention also has the following technical features:
[0026] Step three specifically includes the following steps:
[0027] Step 3.1, N-gram segmentation:
[0028] The specific process of N-gram segmentation is as follows: the content in the original Chinese agricultural text is operated with a sliding window of size n according to bytes, forming a sequence of byte segments of length n. Each byte segment is called a gram. The occurrence frequency of all grams is counted and filtered according to a pre-set threshold to form a key gram list.
[0029] Step 3.2, statistics calculation:
[0030] The word frequency, mutual information and left and right adjacent entropy are calculated for the candidate words to form a candidate word filtering system based on word frequency, mutual information, left adjacent entropy and right adjacent entropy;
[0031] Step 3.3, candidate word screening:
[0032] After calculating the frequency, mutual information and adjacency entropy of each candidate word, the corresponding thresholds are set in turn to filter the candidate words. The final filtering result is defined as a new word set and added to the built-in dictionary of the dictionary-based Jieba word segmentation tool. The Jieba word segmentation tool is strengthened to improve its adaptability to agricultural texts.
[0033] Step 3.4, agricultural domain word set construction:
[0034] The enhanced Jieba word segmentation tool is used to segment the original Chinese agricultural text to obtain the agricultural field word set.
[0035] In step 4, the specific process is as follows: using the same method as step 2, use Word2Vec to train each word in the agricultural field word set into a word vector, and obtain an input sequence X = (x1, x2, x3, ..., x n )∈V c , V c is the domain word set, x is the word in the word set, and each word is represented by a trained dense vector: where e c Represents a word embedding lookup table; after the words in the agricultural field word set are trained into vectors, they constitute an agricultural field dictionary.
[0036] In step 5, the specific process is:
[0037] Step 501, perform vocabulary matching on each character in the model input sequence;
[0038] Step 501 specifically includes: matching the word containing the character from the agricultural field dictionary constructed in step 4, in order to retain all character information and its matched word information, according to each character x in the model input sequence i The words are divided into B(x i ), M(x i ), E(x i ), S(x i ) Four vocabulary sets are constructed as follows:
[0039]
[0040]
[0041]
[0042]
[0043] in:
[0044] B(x i ) represents the B vocabulary set, i.e. the starting position vocabulary set;
[0045] M(x i ) represents the M vocabulary set, i.e. the middle position vocabulary set;
[0046] E(x i) represents the E vocabulary set, i.e. the ending position vocabulary set;
[0047] S(x i ) represents the S vocabulary set, i.e., the single-word position vocabulary set;
[0048] L stands for dictionary;
[0049] w represents the matched word. If a character set is empty, it is represented by "None";
[0050] Step 502, after the character is matched to the corresponding vocabulary, the vocabulary vector needs to be integrated into the corresponding character vector representation to obtain an enhanced character representation, so that the encoding layer can fully utilize the character and vocabulary information;
[0051] Step 502 is specifically implemented by a static weighting method based on statistics, that is, the weight of the matched words is represented by their word frequency, and the data set used for static word frequency statistics is the data used for training and development of the model;
[0052] The following formula is the weighted representation v of the vocabulary set S that a character matches S (S) is calculated as follows:
[0053]
[0054]
[0055] in:
[0056] S represents the S vocabulary set;
[0057] z(w) is the frequency of the word in the vocabulary set in the static data statistics;
[0058] e w is the word embedding lookup table;
[0059] v S (B) v S (M), v S (E) is calculated in the same way as v S (S) is calculated in the same way;
[0060] Finally, the weighted representations of the four vocabulary sets are combined into a one-dimensional feature, namely e s (B,M,E,S)=[v s (B);v s (M); v s (E); v s (S)], and then concatenated to the representation of the character vector, that is, x c ←[x c ;e s(B,M,E,S)] to obtain the final character enhancement vector.
[0061] In step seven, the specific process is:
[0062] Step 7.1, construct the channel attention module:
[0063] In the channel attention module, for the image feature matrix A∈R C×H×W , first transform it into a size R C×N The matrix of A and A is then transposed and matrix multiplied to obtain the characteristic matrix X∈R by the following formula C×C :
[0064]
[0065] in:
[0066] x ji represents the influence of channel i on channel j;
[0067] C represents the number of channels of the image;
[0068] H represents the width of the image;
[0069] W represents the length of the image;
[0070] N represents H×W;
[0071] In the channel attention module, matrix multiplication is performed on the feature matrix X and the image feature matrix A, and element-wise addition is performed on the feature matrix A to obtain the output matrix
[0072] Step 7.2: Based on the channel attention module, the feature map H∈R C×W Perform sequence encoding enhancement:
[0073] The feature map H∈R obtained in step 6 with a length of W C×W As a picture with length W, width 1 and number of channels C, the feature map H∈R is unsqueezed. C×W Expand the dimension and transform it into the image feature matrix A∈R C×H×W ;
[0074] For the image feature matrix A∈R C×H×W :
[0075] First, transform it into a size R C×N The matrix of A and A is then transposed and matrix multiplied to obtain the characteristic matrix X∈R by the following formula C×C ;
[0076]
[0077] Secondly, the image feature matrix A is enhanced by convolutional neural networks conv1, conv3 and conv5 with convolution kernels 1, 3 and 5 in series to enhance the ability to extract local features of the sequence to obtain the feature matrix D = conv5 (conv3 (conv1 (A))), where D ∈ R C×W×H ;
[0078] Again, perform matrix multiplication on the feature matrix X and the image feature matrix A, and perform element-wise addition operation on the image feature matrix A to obtain the output matrix where E∈R C×W×H ;
[0079] Finally, the output matrix E is squeezed to obtain the feature map H∈R C×W The sequence coding result K∈R after sequence coding enhancement C×W .
[0080] Compared with the prior art, the present invention has the following technical effects:
[0081] (I) The present invention strengthens the word segmentation tool to construct an agricultural dictionary, and integrates vocabulary information into the character vector through the character vector strengthening layer, so that the model can make full use of character information and vocabulary information.
[0082] (II) The present invention proposes a sequence coding enhancement module to improve the feature extraction capability of the model.
[0083] (III) Based on the character-based BiLSTM-CRF model, the present invention introduces the agricultural field word set construction process to improve the problem that the word segmentation tool is insensitive to agricultural text word segmentation, so as to construct an agricultural field dictionary.
[0084] (IV) The present invention introduces a character vector enhancement module, applies it to the agricultural field dictionary, and integrates vocabulary information into character information.
[0085] (V) The present invention can further improve the performance of named entity recognition methods for Chinese agricultural texts. BRIEF DESCRIPTION OF THE DRAWINGS
[0086] Figure 1 It is a schematic diagram of a model flow chart of an embodiment of the present invention.
[0087] Figure 2 It is a schematic diagram of the channel attention module.
[0088] Figure 3 It is a schematic diagram of the sequence coding enhancement module.
[0089] Figure 4 It is a schematic diagram of the model flow of comparative example 1.
[0090] Figure 5 It is a schematic diagram of the model flow of Example 2.
[0091] Figure 6 It is a schematic diagram of the model flow of Example 3.
[0092] The specific contents of the present invention are further explained in detail below in conjunction with embodiments. DETAILED DESCRIPTION
[0093] It should be noted that, unless otherwise specified, all software, modules and layers in the present invention are software, modules and layers known in the prior art.
[0094] In accordance with the above technical scheme, specific embodiments of the present invention are given below. It should be noted that the present invention is not limited to the following specific embodiments, and all equivalent changes made on the basis of the technical scheme of this application fall within the protection scope of the present invention.
[0095] Example:
[0096] This embodiment provides a Chinese agricultural named entity recognition method based on a domain dictionary, such as Figure 1 As shown, the method is performed according to the following steps:
[0097] Step 1: Annotation of original Chinese agricultural text:
[0098] The BMESO tag is selected to annotate the original Chinese agricultural text with entity categories following B, M, E, and S.
[0099] In the BMESO tag, B represents the beginning (Begin) of an entity word, M represents the middle (Middle) of an entity word, E represents the end (End) of an entity word, S represents a single word as an entity word (Single), and O represents other words (Other).
[0100] The entity categories include pesticides, pests, diseases and crops.
[0101] This embodiment is based on the annotation of Chinese agricultural text containing 8590 sentences.
[0102] In this embodiment, B-Disease indicates the beginning of the disease entity.
[0103] Step 2: Vectorization of original Chinese agricultural text:
[0104] The word embedding technology Word2Vec is used to train Chinese agricultural text into character vectors, and the input sequence X = (x1, x2, x3, ..., x n )∈Vc , V c are characters, each of which is represented by a trained dense vector: where e c Represents a character embedding lookup table.
[0105] In this embodiment, the word embedding technology Word2Vec adopts the known word embedding technology Word2Vec.
[0106] Step 3: Construction of agricultural vocabulary:
[0107] Firstly, the original Chinese agricultural text is segmented into strings by the N-Gram word segmentation method to obtain candidate words. Then, the junk strings are filtered out in turn according to the three statistics of word frequency, mutual information and adjacency entropy to obtain a new word set. The new word set is added to the built-in dictionary of the Jieba word segmentation tool for enhancement. The enhanced Jieba word segmentation tool is used to segment the original Chinese agricultural text to obtain the agricultural field word set.
[0108] Step three specifically includes the following steps:
[0109] Step 3.1, N-gram segmentation:
[0110] The specific process of N-gram word segmentation is as follows: the content in the original Chinese agricultural text is subjected to a sliding window operation of size n according to bytes, forming a sequence of byte segments of length n. Each byte segment is called a gram. The occurrence frequency of all grams is counted and filtered according to a pre-set threshold to form a key gram list.
[0111] In this embodiment, the 2-gram segmentation result of the Chinese agricultural text "agricultural disease identification" is "agriculture, industry disease, disease, disease identification, identification".
[0112] The present invention firstly segments the Chinese agricultural text by using the N-gram method to obtain candidate character strings, namely candidate words.
[0113] Step 3.2, statistics calculation:
[0114] The word frequency, mutual information and left and right adjacent entropy are calculated for the candidate words to form a candidate word filtering system based on word frequency, mutual information, left adjacent entropy and right adjacent entropy.
[0115] in:
[0116] The word frequency is the frequency of the word appearing. If a word appears frequently, it means that the word is often used and is a common word. Otherwise, it means that the word is not a common word. Words with low frequency of appearance can be removed.
[0117] There is a certain correlation between the characters in a word, so the greater the correlation between characters or words, the greater the probability that the characters or words form a word. Mutual information can calculate the degree of mutual dependence between two objects, and the larger the mutual information value, the greater the degree of dependence between the two objects. Therefore, mutual information can be used to calculate the internal probability of a new word.
[0118] Adjacency entropy is a current method for determining the left and right boundaries of new words. Adjacency entropy can measure the uncertainty of the left and right adjacent characters of a candidate new word. The greater the uncertainty, the more information its adjacent characters contain, and the higher the probability of its forming a word.
[0119] Step 3.3, candidate word screening:
[0120] After calculating the word frequency, mutual information and adjacency entropy for each candidate word, the corresponding thresholds are set in turn to filter the candidate words. The final filtering results are defined as a new word set and added to the built-in dictionary of the dictionary-based Jieba word segmentation tool. The Jieba word segmentation tool is strengthened to improve its adaptability to agricultural texts.
[0121] Step 3.4, agricultural domain word set construction:
[0122] The enhanced Jieba word segmentation tool is used to segment the original Chinese agricultural text to obtain the agricultural field word set.
[0123] Step 4: Construction of agricultural dictionary:
[0124] The agricultural field dictionary is obtained by performing word embedding operation on the agricultural field word set obtained in step 3.
[0125] The specific process of step 4 is as follows: Using the same method as step 2, use Word2Vec to train each word in the agricultural field word set into a word vector, and obtain an input sequence X = (x1, x2, x3, ..., x n )∈V c , V c is the domain word set, x is the word in the word set, and each word is represented by a trained dense vector: where e c Represents a word embedding lookup table; after the words in the agricultural field word set are trained into vectors, they constitute an agricultural field dictionary.
[0126] Step 5: Character vector enhancement:
[0127] The agricultural field dictionary constructed in step 4 is applied to the model input sequence S = (x1, x2, x3, ..., x n )∈V cDictionary matching is performed for each character in the dictionary, and the vocabulary vector obtained in step 4 corresponding to the matched agricultural field dictionary is integrated into the character vector obtained in step 2 corresponding to the character to achieve character vector enhancement.
[0128] The specific process of step five is:
[0129] Step 501: perform vocabulary matching on each character in the model input sequence.
[0130] Step 501 specifically includes: matching the word containing the character from the agricultural field dictionary constructed in step 4, in order to retain all character information and its matched word information, according to each character x in the model input sequence i The words are divided into B(x i ), M(x i ), E(x i ), S(x i ) Four vocabulary sets are constructed as follows:
[0131]
[0132]
[0133]
[0134]
[0135] in:
[0136] B(x i ) represents the B vocabulary set, i.e. the starting position vocabulary set;
[0137] M(x i ) represents the M vocabulary set, i.e. the middle position vocabulary set;
[0138] E(x i ) represents the E vocabulary set, i.e. the ending position vocabulary set;
[0139] S(x i ) represents the S vocabulary set, i.e., the single-word position vocabulary set;
[0140] L stands for dictionary;
[0141] w represents the matched word. If a character set is empty, it is represented by "None".
[0142] In this embodiment, taking the input sequence "plant disease" as an example, the character "物" is lexically matched with a pre-constructed word set, and two words, "plant disease" and "plant", are matched. Then the four dictionaries corresponding to the character "物" are: B = {"None"}, M = {"plant disease"}, E = {"plant"}, S = {"None"}. The character "病" is lexically matched with the pre-constructed dictionary, and two words, "disease" and "病", are matched. Then the four word sets corresponding to the character "病" are: B = {"disease"}, M = {"None"}, E = {"None"}, S = {"病"}.
[0143] Step 502, after the character is matched to the corresponding word, it is necessary to integrate the word vector into the corresponding character vector representation to obtain an enhanced character representation, so that the encoding layer can make full use of character and word information.
[0144] In step 502, it is specifically implemented by a static weighting method based on statistics, that is, the weight of the matched word is represented by its word frequency, and the data set used for static word frequency statistics is the data used by the model for training and development.
[0145] The following formula is the weighted representation v S (S) of the word set S matched by a certain character:
[0146]
[0147]
[0148] Where:
[0149] S represents the word set S;
[0150] z(w) is the frequency of the word in the word set in the static data statistics;
[0151] e w is the word embedding lookup table;
[0152] v S (B), v S (M), v S (E) are calculated in the same way as v S (S).
[0153] Finally, the weighted representation forms of the four word sets are combined into a one-dimensional feature, that is, e s (B, M, E, S) = [v s (B); v s (M); v s (E); v s (S)], and then concatenated to the representation of the character vector, that is, xc ←[x c ;e s (B,M,E,S)] to obtain the final character enhancement vector.
[0154] Step 6: Sequence encoding:
[0155] BiLSTM is used as the sequence encoding layer, and the enhanced character vector obtained in step 5 is input into the sequence encoding layer for sequence encoding to obtain the sequence encoding result, that is, the feature map H∈R C×W .
[0156] RNN (basic recurrent neural network) cannot learn long-distance dependencies well due to gradient explosion during training.
[0157] LSTM (Long Short-Term Memory Network) introduces memory cells to record state information based on RNN, and updates hidden states and memory cells through three gate structures: input gate, forget gate, and output gate. LSTM is a unidirectional recurrent neural network that can only obtain unidirectional context information of the target word.
[0158] BiLSTM (Bidirectional Long Short-Term Memory Network) is widely used in NLP (Natural Language Processing) tasks and performs well in dealing with sequence encoding problems. BiLSTM can fully obtain the context information of the target character from both the forward and reverse directions, thereby obtaining a deeper feature representation. Therefore, this embodiment uses BiLSTM as the sequence encoding layer.
[0159] First, let’s introduce the implementation details of LSTM. LSTM consists of a forget gate, an input gate, and an output gate. These three gates are used to control the information flow. Suppose the character x at time step t t After being processed by the dictionary construction method disclosed in the present invention, the enhanced character representation is obtained. Then the one-way LSTM calculation is as follows:
[0160]
[0161]
[0162]
[0163]
[0164]
[0165] h t =o t ⊙tanh(c t )
[0166] in:
[0167] σ represents the sigmoid activation function;
[0168] tanh is the hyperbolic tangent activation function;
[0169] i, f, o, and c represent the input gate, forget gate, output gate, and memory cell at time t, respectively;
[0170] W and b represent the weight matrix and bias vector in the corresponding gate respectively;
[0171] x t represents the input vector at time t;
[0172] represents the candidate memory cell at time t;
[0173] h t is the hidden state of the LSTM unit, that is, the output of each unit;
[0174] ⊙ is calculated by element points.
[0175] Then BiLSTM can be formally expressed as the following formula. The output of the forward and backward LSTM is concatenated to obtain the character x at time step t t The encoded representation is used as the input of the next CRF layer.
[0176]
[0177]
[0178]
[0179] Step 7: Sequence coding enhancement:
[0180] Construct a channel attention module based on the channel attention module for the feature map H∈R C×W Perform sequence coding enhancement to obtain the sequence coding result K∈R C×W .
[0181] Step 7.1, construct the channel attention module:
[0182] like Figure 2 As shown in the figure, the channel attention module is often used in the field of semantic segmentation to enhance the model's ability to extract dependencies between different pixels in an image.
[0183] In the channel attention module, for the image feature matrix A∈R C×H×W , first transform it into a size R C×N The matrix of A and A is then transposed and matrix multiplied to obtain the characteristic matrix X∈R by the following formula C×C:
[0184]
[0185] in:
[0186] x ji represents the influence of channel i on channel j;
[0187] C represents the number of channels of the image;
[0188] H represents the width of the image;
[0189] W represents the length of the image;
[0190] N represents H×W.
[0191] In the channel attention module, matrix multiplication is performed on the feature matrix X and the image feature matrix A, and element-wise addition is performed on the feature matrix A to obtain the output matrix
[0192] Step 7.2: Based on the channel attention module, the feature map H∈R C×W Perform sequence encoding enhancement:
[0193] The feature map H∈R obtained in step 6 with a length of W C×W As a picture with length W, width 1 and number of channels C, the feature map H∈R is unsqueezed. C×W Expand the dimension and transform it into the image feature matrix A∈R C×H×W .
[0194] like Figure 3 As shown, for the picture feature matrix A∈R C×H×W :
[0195] First, transform it into a size R C×N The matrix of A and A is then transposed and matrix multiplied to obtain the characteristic matrix X∈R by the following formula C×C .
[0196]
[0197] Secondly, the image feature matrix A is enhanced by convolutional neural networks conv1, conv3 and conv5 with convolution kernels 1, 3 and 5 in series to enhance the ability to extract local features of the sequence to obtain the feature matrix D = conv5 (conv3 (conv1 (A))), where D ∈ R C×W×H .
[0198] Again, perform matrix multiplication on the feature matrix X and the image feature matrix A, and perform element-wise addition operation on the image feature matrix A to obtain the output matrix where E∈R C×W×H .
[0199] Finally, the output matrix E is squeezed to obtain the feature map H∈R C×W The sequence coding result K∈R after sequence coding enhancement C×W .
[0200] Step 8: Use CRF to encode the sequence result K∈R C×W For decoding processing, CRF can encode the sequence result K∈R according to the result of the original Chinese agricultural text annotation in step 1. C×W The probability of the BMESO label corresponding to each character in is calculated, and the maximum probability is solved by the standard Viterbi algorithm to obtain the label of each character.
[0201] In this embodiment, CRF is a conditional random field, and CRF is used to encode the sequence result K∈R C×W The specific method for decoding processing adopts the known conditional random field conventional processing method.
[0202] In this embodiment, the method of the present invention may be compiled into a program code, the code may be stored in a computer-readable storage medium, the program code may be transmitted to a processor, and the method of the present invention may be executed by the processor.
[0203] Comparative Example 1:
[0204] This comparative example provides a method for recognizing named entities in Chinese agriculture, wherein a character-based BiLSTM-CRF model is provided, that is, the character enhancement layer and the sequence encoding enhancement layer are removed from the model of the embodiment of the present invention, which is the baseline model of the embodiment of the present invention, and the structure is as follows: Figure 4 As shown (the model schematic diagram of the present invention is as shown Figure 1 shown).
[0205] Comparative Example 2:
[0206] This comparative example provides a Chinese agricultural named entity recognition method, in which a dictionary-based BiLSTM-CRF model is provided, that is, the sequence coding enhancement layer is removed on the basis of the model of the embodiment of the present invention, and the structure is as follows Figure 5 shown.
[0207] Comparative Example 3:
[0208] This comparative example provides a Chinese agricultural named entity recognition method, in which a dictionary-based BiLSTM-CRF model is provided, that is, the sequence encoding enhancement layer in the model of the embodiment of the present invention is replaced with an unimproved channel attention mechanism, and the structure is as follows Figure 6 shown.
[0209] Comparative Example 4:
[0210] This comparative example provides a Chinese agricultural named entity recognition method. Step 3 of this comparative example enhances the Jieba word segmentation. This comparative example uses the unenhanced Jieba word segmentation during the implementation process, and the rest is the same as the method of the embodiment of the present invention.
[0211] Performance Test:
[0212] The commonly used criteria for evaluating named entity recognition models are mainly precision, recall, and F1 score, and the calculation formula is:
[0213]
[0214]
[0215]
[0216] Table 1 is a comparison between different models. It can be seen from the table that the method proposed in the present invention has achieved the best results in the three evaluation criteria of precision, recall rate, and F1 score (i.e., the number of comprehensive evaluation copies), which fully demonstrates the superiority of the present invention. Comparative Example 2 has achieved a 3.47 improvement in F1 score compared with Comparative Example 1, which shows the effectiveness of constructing an agricultural field dictionary and integrating vocabulary information into character representation through a character enhancement layer when processing agricultural named entity recognition tasks. Comparative Example 3 has achieved a 0.85 improvement in F1 score compared with Comparative Example 2, indicating that the use of a channel attention mechanism can effectively enhance the feature extraction ability of the model. Experiment 4, that is, the method of the present invention, has achieved a 0.72 improvement in F1 score compared to Experiment 3, indicating the effectiveness of the sequence encoding enhancement module proposed by the present invention based on the channel attention mechanism and the integration of the three-level convolutional neural network. The embodiment of the present invention has achieved an F1 score improvement of 0.94 compared to Comparative Example 4, indicating the effectiveness of the present invention in enhancing the Jieba word segmentation tool for agricultural texts using a series of operations in step 3.
[0217] Table 1 Effects of the present invention and two comparative methods
[0218] experiment method Accuracy Recall F1 score 1 Comparative Example 1 83.35 85.17 84.26 2 Comparative Example 2 86.34 88.35 87.73 3 Comparative Example 3 87.15 90.04 88.58 4 Embodiments of the present invention 87.14 91.60 89.30 5 Comparative Example 4 87.26 89.48 88.36
Claims
1. A Chinese agricultural named entity recognition method based on domain dictionary, characterized in that: The method proceeds as follows: Step 1: Annotation of original Chinese agricultural text: Select the BMESO tag to annotate the original Chinese agricultural text, and follow the entity category after B, M, E, and S; In the BMESO tag, B indicates the beginning of an entity word, M indicates the middle of an entity word, E indicates the end of an entity word, S indicates that a single word is an entity word, and O indicates other words; The entity categories include pesticides, pests, diseases and crops; Step 2: Vectorization of original Chinese agricultural text: The word embedding technology Word2Vec is used to train Chinese agricultural text into character vectors, and the given length is n The input sequence X=(x 1 , x 2 , x 3 ,……,x n )∈V c , V c are characters, each of which is represented by a trained dense vector: x i c = e c (x i ) ,in e c represents a character embedding lookup table; Step 3: Construction of agricultural vocabulary: First, the original Chinese agricultural text is segmented into strings using the N-Gram segmentation method to obtain candidate words. Then, the junk strings are filtered out in turn according to the three statistics of word frequency, mutual information and adjacency entropy to obtain a new word set. The new word set is added to the built-in dictionary of the Jieba word segmentation tool for enhancement. The enhanced Jieba word segmentation tool is used to segment the original Chinese agricultural text to obtain the agricultural field word set. Step 4: Construction of agricultural dictionary: Perform word embedding operation on the agricultural domain word set obtained in step 3 to obtain an agricultural domain dictionary; Step 5: Character vector enhancement: Apply the agricultural field dictionary constructed in step 4 to the model input sequence S=(x 1 , x 2 , x 3 ,……,x n )∈V c Perform dictionary matching on each character in the dictionary, and integrate the vocabulary vector obtained in step 4 corresponding to the matched agricultural field dictionary into the character vector obtained in step 2 corresponding to the character, so as to realize character vector enhancement; In step 5, the specific process is: Step 501, perform vocabulary matching on each character in the model input sequence; Step 501 specifically includes: matching the word containing the character from the agricultural field dictionary constructed in step 4, in order to retain all character information and its matched word information, according to each character in the model input sequence x i The words are divided into the starting position, middle position, end position or single word position in the different words it matches. B(x i ) , M(x i ) , E (x i ) , S(x i ) Four vocabulary sets are constructed as follows: , , , ; in: B(x i ) express B The vocabulary set is the starting position vocabulary set; M(x i ) express M Lexical set, that is, the set of middle position lexical sets; E(x i ) express E The vocabulary set is the ending position vocabulary set; S(x i ) express S Vocabulary set, i.e., the vocabulary set of single-word positions; L Represents a dictionary; w Indicates the matched words. If a character set is empty, it is represented by "None"; Step 502, after the character is matched to the corresponding vocabulary, the vocabulary vector needs to be integrated into the corresponding character vector representation to obtain an enhanced character representation, so that the encoding layer can fully utilize the character and vocabulary information; Step 502 is specifically implemented by a static weighting method based on statistics, that is, the weight of the matched words is represented by their word frequency, and the data set used for static word frequency statistics is the data used for training and development of the model; The following formula is the vocabulary set that a certain character matches S The weighted representation of v S (S) Calculation method: ; in: S express S Vocabulary collection; z(w) is the frequency of occurrence of words in the vocabulary set in static data statistics; e w is the word embedding lookup table; v S (B) v S (M), v S (E) The calculation method of v S (S) is calculated in the same way; Finally, the weighted representations of the four vocabulary sets are combined into a one-dimensional feature, namely , and then concatenated to the representation of the character vector, that is , thus obtaining the final character enhancement vector; Step 6: Sequence encoding: BiLSTM is used as the sequence encoding layer, and the enhanced character vector obtained in step 5 is input into the sequence encoding layer for sequence encoding to obtain the sequence encoding result, that is, the feature map ; Step 7: Sequence coding enhancement: Construct a channel attention module based on the channel attention module to the feature map Perform sequence coding enhancement to obtain sequence coding results ; Step 8: Use CRF to encode the sequence results For decoding processing, CRF can encode the sequence results based on the results of the original Chinese agricultural text annotation in step 1 The probability of the BMESO label corresponding to each character in the text is calculated, and the maximum probability is solved by the standard Viterbi algorithm to obtain the label of each character, thus realizing Chinese agricultural named entity recognition; In step seven, the specific process is: Step 7.1, construct the channel attention module: In the channel attention module, for the image feature matrix , first transform it into a size The matrix is then converted into A and A The transpose of the matrix multiplication is used to obtain the characteristic matrix : in: Indicates channel For Channel The impact of C Indicates the number of channels of the image; H Indicates the width of the image; W Indicates the length of the image; N express H × W ; In the channel attention module, the feature matrix X and the image feature matrix A Do matrix multiplication and add it to the image feature matrix A Perform element-wise addition to get the output matrix ; Step 7.2: Attention module for feature maps based on channel Perform sequence encoding enhancement: The length is W The feature map obtained in step six Considered as length W、 The width is 1 and the number of channels is C The picture is unsqueeze operation to remove the feature map Expand the dimension and turn it into a picture feature matrix ; For the image feature matrix : First, transform it into a size The matrix is then converted into A and A The transpose of the matrix multiplication is used to obtain the characteristic matrix ; ; Secondly, through the convolutional neural network with convolution kernels of 1, 3 and 5 in series conv1 , conv3 and conv5 , for the image feature matrix A Perform feature enhancement processing to enhance the ability to extract local features of the sequence and obtain the feature matrix ; Again, for the feature matrix X With the image feature matrix A Do matrix multiplication and add it to the image feature matrix A Perform element-wise addition to get the output matrix ; Finally, the output matrix Perform a squeeze operation to obtain a feature map Sequence encoding results after sequence encoding enhancement .
2. The Chinese agricultural named entity recognition method based on domain dictionary as claimed in claim 1 is characterized in that: Step three specifically includes the following steps: Step 3.1, N-gram segmentation: The specific process of N-gram segmentation is as follows: the content of the original Chinese agricultural text is divided into bytes. n The sliding window operation forms a length of n Each byte segment is called a gram. The occurrence frequencies of all the gram are counted and filtered according to the preset threshold to form a key gram list. Step 3.2, statistics calculation: The word frequency, mutual information and left and right adjacent entropy are calculated for the candidate words to form a candidate word filtering system based on word frequency, mutual information, left adjacent entropy and right adjacent entropy; Step 3.3, candidate word screening: After calculating the frequency, mutual information and adjacency entropy of each candidate word, the corresponding thresholds are set in turn to filter the candidate words. The final filtering result is defined as a new word set and added to the built-in dictionary of the dictionary-based Jieba word segmentation tool. The Jieba word segmentation tool is strengthened to improve its adaptability to agricultural texts. Step 3.4, agricultural domain word set construction: The enhanced Jieba word segmentation tool is used to segment the original Chinese agricultural text to obtain the agricultural field word set.
3. The Chinese agricultural named entity recognition method based on domain dictionary as claimed in claim 1 is characterized in that: In step 4, the specific process is: using the same method as step 2, use Word2Vec to train each word in the agricultural field word set into a word vector, and obtain a given length of n The input sequence X=(x 1 , x 2 , x 3 ,……,x n )∈V c , V c is the domain vocabulary, x are the words in the word set, each word is represented by a trained dense vector: x i c =e c (x i ) ,in e c Represents a word embedding lookup table; after the words in the agricultural field word set are trained into vectors, they constitute an agricultural field dictionary.
Citation Information
Patent Citations
Cross-border ethnic culture entity recognition method and device based on word set feature weighting
CN113935324A
Systems and methods for adaptive proper name entity recognition and understanding
US20180068661A1
