Semi-supervised Chinese place name identification method combining active learning and self-training

Abstract

The invention relates to the field of geographic information extraction methods, and discloses an active learning and self-training combined semi-supervised Chinese geographic name recognition method, which comprises the following steps: S1, acquiring a corpus text containing a geographic name as a data set, S2, carrying out diversity classification on corpus data by using an LDA topic model according to a topic of the text, and S3, carrying out classification on the data set; s3, selecting a sample with a high uncertainty score, namely, high information, according to a prediction result of the model by an active learning method based on an LTP (Lowest Token Probability) uncertainty query strategy, S4, screening out a small number of samples with diversity and high information for manual labeling according to the steps S2 and S3, and S5, combining a small number of manually labeled samples and a large number of unlabeled samples to obtain a large number of non-labeled samples. And continuously iteratively updating model parameters by adopting a BERT model-based self-training method with high-confidence coefficient tokens selection to obtain an optimal place name recognition model. According to the method, the LDA topic model is used for carrying out diversity classification on corpus data.

Description

Technical Field

[0001] The present invention relates to the field of geographic information extraction methods, and in particular to a semi-supervised Chinese place name recognition method combining active learning and self-training. Background Art

[0002] Place name recognition is a subset of named entity recognition (NER). Its purpose is to identify the boundaries of location names in texts. It can identify a large amount of geographic information from the massive unstructured texts that appear on the Internet platform. In data sets such as social media, natural language texts with place name tags are very sparse, and there are phenomena such as irregular expression of place name information, unclear entity boundaries, and simplified expression of place names. Compared with English, Chinese text does not have clear word boundaries, such as spaces, which makes the boundary recognition of place names complicated. For example, "I plan to travel to Chengdu next year" does not have obvious word segmentation symbols like English, which makes the boundary recognition of place names complicated. Chinese place names are usually composed of multiple words such as geographical location nouns, landmark building nouns, and place type nouns, such as "Shanghai Oriental Pearl Tower". This complex structure increases the difficulty of recognition. Chinese place names have national characteristics, such as "Urumqi" and "Arshan" and other place names with national characteristics. These words have certain uniqueness in language. Chinese place names are often represented by abbreviations or abbreviations, such as "Yu" for Chongqing. In addition, China is vast and has a huge number of place names, which poses challenges to Chinese place name recognition. The mainstream place name entity recognition methods can be divided into four categories: rule-based matching methods, place name dictionary matching methods, machine learning-based methods, and deep learning-based methods.

[0003] Place name entity recognition based on rule matching defines rules through the vocabulary, syntax and spelling features of place names to identify whether the text is a place name. For example, Zhang Xueying et al. (Zhang Xueying et al. 2010) used a large-scale place name dictionary and address database as data sources, analyzed the word formation features of place names and the structural patterns of addresses, constructed a feature word library and parsing rule library, and designed the RBAI Chinese address element parsing algorithm. Malmasi et al. (Malmasi and Dras 2016) proposed an unsupervised location detection model based on grammatical parse trees and place name directory information. The model is based on language technology and rules, and uses n-gram matching technology to match the place name database GeoNames to extract place names from noun phrases.

[0004] Gazetteers usually contain information such as the standard written form of place names, geographical location, administrative region, demographic data, and historical background. In many studies, gazetteers play an important role in place name recognition. For example, Milleville et al. (Milleville, Verstockt, and van de Weghe 2020) proposed a place name recognition algorithm with place name detection and place name matching as the core. Gazetteers are used to match the identified place name entities with real-world place names, thereby improving the quality of place name annotations in maps. Al-Olimat et al. (Al-Olimat et al. 2020) used a method combining n-gram statistics and gazetteers to identify location detection in Twitter texts, and proposed a place name extraction tool called LNEx to process place name abbreviations and extended gazetteers.

[0005] Machine learning methods mainly use the language information on the annotated training corpus to train the model through manually defined features, such as word features, context features, part-of-speech features, stop word features, language features, word vector features, etc. The trained language model automatically identifies entities in the text. This type of method does not require too much professional knowledge and linguistic knowledge, and has relatively little dependence on specific language characteristics. For example, Mao Bo et al. (Mao Bo and Teng Wei 2020) deeply analyzed the characteristics of complex Chinese place names to achieve accurate and automatic recognition of complex Chinese place names, trained conditional random field models to recognize place names, and combined rules to correct and supplement the CRF results. In order to further improve the recognition accuracy of complex place names, a complex place name recognition algorithm based on information entropy and point mutual information is designed. The algorithm uses the place name database to generate an association dictionary, and based on the dictionary, calculates the association between adjacent words in the text, thereby determining the boundary between complex place names and contexts, and finally realizing the recognition of complex place names.

[0006] Deep learning has attracted widespread attention in the field of place name entity recognition due to its deep feature learning ability and strong generalization ability. For example, Ma et al. (Tao et al. 2022) proposed a deep neural network BERT-BiLSTM-CRF based on the BERT model. The input text sequence is encoded through the BERT model to obtain a high-dimensional representation of each word; then BiLSTM is used to further capture the long-distance dependencies in the sequence, and finally CRF is used to optimize the final label sequence to achieve the classification of Chinese place names. Tao et al. (Tao et al. 2022) proposed using the ALBERT model for word vector extraction. ALBERT reduces the number of model parameters through parameter sharing, making the model more lightweight, and has obvious advantages in reducing model complexity, speeding up training, and improving efficiency. Zhao et al. (Zhao et al. 2024) proposed a place name recognition model EIBC based on a multi-layer deep learning framework, using an ERNIEGram-IDCNN-BiLSTM-CRF deep learning network to extract Chinese place names, achieving excellent results. Some scholars have also proposed weakly supervised deep learning methods to identify place names. The method mainly solves the dependency problem of manually annotated data by automatically generating training sentences. For example, Wang et al. (Wang, Hu, and Joseph 2020) proposed to generate labeled training data from Wikipedia articles, and Qiu et al. (Qiu et al. 2022) generated training data based on word sets and related word frequencies in different texts. Summary of the invention

[0007] The purpose of the present invention is to provide a semi-supervised Chinese place name recognition method combining active learning and self-training, which has the advantage of requiring only a small amount of manually labeled data. While achieving a good recognition effect, the method can greatly reduce the workload of manual labeling.

[0008] To achieve the above object, the present invention provides the following technical solution: a semi-supervised Chinese place name recognition method combining active learning and self-training, wherein the recognition method comprises the following steps:

[0009] S1: Get the corpus text containing place names as a dataset

[0010] The corpus text containing place names includes the Chinese NER public dataset, which includes training set, test set and validation set;

[0011] S2: Use the LDA topic model to classify the corpus data according to the diversity of the text topics

[0012] LDA stands for Latent Dirichlet Allocation. It regards each text as a probability distribution consisting of a set of topics, and each topic is a probability distribution consisting of a set of specific words in the text. Each word in the text is distributed with a certain probability. The topic of each text is represented by a set of words with the highest frequency, thus forming a three-layer Bayesian generative model consisting of text, topic and word.

[0013] Specifically, the corpus is defined as D = (d1, d2, ..., d Z ), consists of Z texts, and a text consists of N d The sequence of words is denoted as d = (w1, w2, ..., w n ), where n∈(1, 2, ... N d ), d∈(1, 2, ..., Z). LDA models D according to the following generative process:

[0014] 1) Select N d ~Poisson(ξ), where Poisson(ξ) is the Poisson distribution;

[0015] 2) Select the parameter θ of the text topic distribution d ~Dir(α), where Dir(α) is the Dirichlet distribution;

[0016] 3) For a word sequence of N d Each word w in the text n :

[0017] a) Choose a topic n ~Mult(O), where Mult(θ d ) is a multinomial distribution;

[0018] b) From the topic t n The word distribution p(w n |t n ,β) to sample a word w n .

[0019] The LDA model contains some basic assumptions: 1) It is assumed that the dimension k of Dir(α) is known and fixed; 2) It is assumed that the probability of a word is parameterized by a matrix β of dimension k×V; 3) The Poisson distribution is not important for the derivation of the LDA model, and other distributions can be used as needed. Therefore, given α and β, the generation probability of the corpus D is calculated and obtained as follows:

[0020]

[0021] The corpus-level parameters α and β are sampled only once in the process of generating the corpus; the text-level variable θ d, sample each text once; the vocabulary-level variable t dn 、w dn Each word in each text is sampled once.

[0022] Use the LDA topic model to classify all unlabeled samples into diverse categories according to topics, and the number of topics is determined by the C_v (Coefficient of variance) score;

[0023] S3: Active learning method based on LTP (Lowest Token Probability) uncertainty query strategy selects samples with higher uncertainty scores, i.e., more informative samples, according to the prediction results of the model.

[0024] The LTP score is calculated as shown below:

[0025]

[0026] Where y* represents the most likely label sequence for x, and A is a parameter called the transfer matrix, which is the parameter learned by the model.

[0027] The active learning method is used to screen samples based on the LDA topic diversity classification results. The query strategy adopted by active learning is the LTP uncertainty query strategy, and the classifier is BiLSTM-CRF. Before active learning screening, 100-200 manually labeled training samples are randomly screened to initialize the classifier, and finally samples with both diversity and high information are screened for manual labeling.

[0028] S4: According to steps S2 and S3, a small number of samples with both diversity and high information are selected for manual labeling.

[0029] Before data screening, all labels of the training set will be removed. When manual labeling is required, the labels of the public dataset will be used as the manual labeling results.

[0030] S5: Use the self-training method with high confidence tokens selection based on the BERT model to continuously iterate and update the model parameters to obtain the best place name recognition model

[0031] The self-training method with high confidence tokens selection based on the BERT model is as follows:

[0032] 1) Using the samples selected for manual annotation based on the LDA topic model and the LTP uncertainty active learning strategy as the Bert-based self-training initialization model, by minimizing The formula for training the model with the following loss is as follows:

[0033]

[0034] 2) Using the trained initialization model, predict the sample data that has not been manually labeled to obtain pseudo-label training data;

[0035] 3) Initialize the teacher model and the student model using the model parameters in step S1;

[0036] 4) Perform T rounds of iterative self-training, calculate soft labels in each iteration, and optimize and update the new model only on a selected set of high-confidence tokens to finally obtain the place name recognition model with the best performance based on the test results.

[0037] The formula for generating soft false labels is as follows:

[0038]

[0039] in M is the number of data with labels, θ is the parameter of the NER model, and X m is the vector of the mth sample sentence, n is the nth token in the mth sentence, and C is the type.

[0040] Tokens are selected based on the confidence of the prediction. At the tth iteration, the model loss formula is minimized by selecting a set of high-confidence tokens in the mth sentence as follows:

[0041]

[0042] where ε∈(0,1) is the tuning threshold.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The present invention first uses the LDA topic model to classify the diversity of the corpus data, and then uses the active learning method based on the LTP uncertainty query strategy to further screen the samples, and screens out a small number of samples with both diversity and high information for manual annotation. Then, combined with a small number of manually annotated samples and a large number of unlabeled samples, a self-training method with high-confidence tokens selection based on the BERT model is used to continuously iterate and update the model parameters to improve the accuracy and generalization ability of the place name recognition model. The present invention conducts experiments on the three data sets of MSRA, Ontonote4 and Boson, annotating 1000, 800 and 800 samples respectively. The comparative experimental results show that the F1 values ​​on the above three data sets are 0.9309, 0.7732 and 0.7914 respectively, which are all higher than the baseline model. The method proposed in the present invention achieves good place name recognition performance with a small amount of annotations, greatly reducing the workload of manual annotation. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic flow chart of the method of the present invention;

[0046] Figure 2 It is the C_v Coherence score map of the three data sets;

[0047] Figure 3 is the influence diagram for the high confidence threshold selection. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] An implementation example of a semi-supervised place name recognition method combining active learning and self-training includes the following steps:

[0050] S1: Obtain Chinese corpus text containing place names as a dataset. The collected datasets include MSRA, Boson and Ontonote4, which are divided into training set, test set and validation set. All labels of the training set will be removed before data screening. When manual annotation is required, the annotations of the public dataset are used as the manual annotation results.

[0051] S2: Classify the corpus according to the diversity of topics through the LDA topic model;

[0052] S3: Based on the LTP uncertainty active learning strategy, high-information samples are selected from the diverse classification results. The core idea of ​​the active learning method of the uncertainty query strategy is that the model actively selects its most uncertain samples to obtain annotations, thereby improving learning efficiency and model performance;

[0053] S4: The self-training method with high confidence tokens selection based on the BERT model is as follows:

[0054] The BIO tagging scheme is used, where B represents the start position of the entity, I represents the internal and end position of the entity, and O represents the non-entity part of the sentence instance. Specifically, M is defined as the number of data with labels, and C is the type. X represents the vector representation of a sentence with a length of N. X = (x1, x2...x N ), the corresponding label sequence is expressed as Y = (y1, y2......y N ), train an entity recognition model parameterized by θ to minimize the classification loss so that the model correctly predicts the given label.

[0055] 1) Use the samples selected for manual annotation according to the LDA topic model and the LTP uncertainty active learning strategy as the training set for the BERT-based self-training initialization model, and train such a model by minimizing the loss on the training data.

[0056] The formula is as follows:

[0057]

[0058] Where θ is the parameter of the model. l(·,·) is the loss function.

[0059] 2) Using the trained initialization model, predict the sample data that has not been manually labeled to obtain soft label training data;

[0060] 3) Initialize the teacher model and student model using the model parameters in step S1

[0061] 4) Perform T rounds of iterative self-training, and calculate soft false labels in each iteration. The formula for generating soft false labels is as follows:

[0062]

[0063] in

[0064] 5) By minimizing Update the student model for the next iteration based on the loss of KL divergence

[0065]

[0066] 6) Select tokens based on prediction confidence. At the tth iteration, a set of tokens whose confidence in the mth sentence satisfies the tuning threshold ε∈(0,1) is selected by the following formula;

[0067]

[0068] The student model is optimized by minimizing the loss only on the selected tokens, as follows:

[0069]

[0070] 7) According to the prediction performance of the Chinese place name recognition model, the model parameters are tuned.

[0071] Number of experimental datasets

[0072] The experimental data sets of the present invention mainly include three data sets: MSRA, OntoNotes4 and Boson. The details of the data sets are shown in Table 1. The MSRA data set is an entity recognition data set in the news field annotated by Microsoft Asia Research, and is also one of the data sets for the entity recognition task of SIGNANbackoff2006. The data set contains more than 50,000 Chinese entity recognition annotated data, and the entity categories are divided into three categories: people, places, and institutions.

[0073] OntoNotes Release 4.0 is a large-scale semantically annotated dataset jointly created and maintained by Cornell University, University of Pennsylvania and News Corporation (New York Times), including 250,000 words of Chinese radio news, 150,000 words of Chinese radio dialogues and 150,000 words of Chinese online texts. The dataset contains 24,371 Chinese entity recognition annotated data, and the entity categories are divided into people, geopolitical entities, organizations and place names. The present invention regards the geopolitical entities in the dataset as place names.

[0074] The Boson dataset is the named entity recognition data provided by Boson. It is encoded in UTF-8 and each line is a paragraph annotation. It includes 2,000 paragraphs and more than 10,000 Chinese sentences. The annotated entity types include six categories: time, place, person name, organization name, company name, and product name.

[0075] The proposed method consists of two stages: (1) data screening combining diversity and uncertainty; (2) self-training with high-confidence selection. The overall structure of the method is shown in Figure 1 shown.

[0076] In the data screening stage, in order to avoid the problem that active learning causes the model to only focus on specific data areas and ignore other important samples, the present invention uses the LDA topic model to classify the corpus data and distinguish the original data in terms of topic diversity. Then, based on each topic category, the LTP query strategy in the active learning method is used to select samples with higher uncertainty scores (i.e., high dispersion) through the prediction results of the model. Through the screening at this stage, the most valuable samples are selected and submitted to experts for annotation. Then, the BERT model is initialized with the obtained annotated data, and the remaining unlabeled data is used for model prediction to obtain pseudo-label data, which is used for the next self-training method based on high-confidence label selection. The parameters of the teacher-student model are continuously trained and iterated through the self-training method, thereby improving the accuracy and generalization ability of the final place name recognition model.

[0077] Table 1 Dataset details

[0078]

[0079] Evaluation Metrics

[0080] The evaluation indicators of this experiment use precision (Precision, P), recall (Recall, R) and F1 score (F1). Among them, precision mainly evaluates the proportion of true positive samples in the samples predicted by the model as positive examples, recall mainly evaluates the proportion of positive samples predicted correctly by the algorithm in all positive samples, and F1 value is the harmonic mean of Precision and Recall, which is a commonly used evaluation indicator. The calculation formulas of Precision, Recall and F1 values ​​are as follows:

[0081]

[0082] Among them, true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN) are the four categories into which the prediction results are divided based on the model prediction results and the actual labels of the samples.

[0083] Experimental results and analysis

[0084] This section compares the method of the present invention with several baseline methods, mainly including HMM, BERT-softmax, MeanTeacher, VAT and ChineseTR, among which HMM is a traditional machine learning method, BERT-softmax is a deep learning fully supervised method, both methods use all the labeled data in the training set as training data, MeanTeacher and VAT are two classic semi-supervised methods, in both methods, the BERT model is used to learn the vector representation of sentences, and the same amount of training data is randomly selected for training. In these semi-supervised methods, a small amount of data is selected in the data screening step for manual labeling. The amount of data screened in the MSRA, Ontonote4 and Boson data sets is 1000, 800 and 800 respectively, and ChineseTR is a place name recognition model based on weakly supervised deep learning.

[0085] From the experimental results (Table 2), it can be seen that thanks to sufficient manually annotated data, the place name features can be better learned. The fully supervised method BERT-softmax performs best on the three data sets. The F1 scores of the method of the present invention on the three data sets are 0.9309, 0.7732 and 0.7914 respectively. Compared with other semi-supervised baseline methods (Mean Teacher and VAT), better results are achieved, and the experimental results are close to the BERT-softmax method, especially on the Ontonote4 set, which almost reaches the effect of the fully supervised learning method.

[0086] Table 2 Experimental results

[0087]

[0088] The method of the present invention uses the screened data to initialize the model and perform supervised training in self-training to achieve prediction of unlabeled data, and selects the most confident predicted label for model loss optimization in the subsequent self-training iterative training. Compared with the Mean Teacher and VAT methods, the pseudo-label data generated by label data prediction is prone to introduce more noise, resulting in performance degradation. The method of the present invention can better ensure the accuracy of pseudo-label annotation and reduce the impact of noise labels on the model. Through this method, the amount of training data can be effectively expanded, so that the model has better performance and generalization ability.

[0089] Experiments on the MSRA and Boson datasets show that the method of the present invention has obvious advantages over the weakly supervised method ChineseTR. The ChineseTR method automatically generates training data mainly based on word sets and related word frequencies from different texts. This method does not use manually annotated data. The generated sentences have logical errors due to the lack of context-related semantic information, which makes it difficult for the model to learn the contextual semantic information in the training set or the learned semantic information contains more noise. The method of the present invention selects a small amount of data by combining diversity and uncertainty, and uses it for model training after manual annotation. The remaining corpus is further annotated, and a self-training method with high-confidence labels is selected to reduce the impact of noise data on the model. Therefore, the method of the present invention has the advantage of only requiring a small amount of manual cost to obtain better place name recognition results, and the method of the present invention can be easily expanded and applied to other fields where annotated data is lacking, greatly reducing the cost of manual annotation.

[0090] Ablation and comparative experiments

[0091] In order to verify the effectiveness of each part of the method proposed in the present invention, ablation and comparison experiments were carried out on three experimental data (see Table 3). Method 1 in the table refers to LDA topic model + LTP (no self-training), which only uses the screened labeled data as the experimental data training model, and no longer uses the remaining unlabeled data for self-training; Method 2 refers to random screening + self-training, which uses randomly screened data for manual labeling; Method 3 refers to LTP + self-training, which only uses the active learning LTP strategy to screen out data with uncertain signal height for manual labeling; Method 4 refers to LDA + self-training, which only uses the LTP topic model to classify the corpus, and randomly selects the same amount of data from each class for manual labeling; Method 5 refers to LTP + cosine similarity + GCE noise learning, which uses the active learning LTP strategy to screen highly informative training samples, and then uses the world2vec word vector to calculate the sentence cosine similarity method to ensure the diversity of the screened samples, and then based on the BERT vector representation, combined with the GCE (Zhang and Sabuncu 2018) loss function for noise label noise robustness learning; Method 6 refers to LTP + cosine similarity + self-training, using the same data screening method as method 5, combined with the same self-training method as the present invention to train the model. The present invention analyzes the experimental results of ablation and comparison experiments from three aspects. The method of the present invention refers to LDA topic model+LTP+self-training.

[0092] 1) Verify the effectiveness of combining diversity classification with uncertainty sampling

[0093] The same self-training method and different manually labeled data screening methods were used to verify the effectiveness of using LDA and LTP for data screening. From the comparison between method 2 to method 4 in the experimental result table 3 and the method of the present invention, it can be concluded that the method of the present invention using the LTP active learning method combined with the LDA topic model improves the F1 scores of the three data sets by 3.29%, 4.56%, and 5.54% respectively compared with the random screening method of method 2. The F1 scores of the present invention on the three data sets are improved by 2.33%, 3.05%, and 1.36% respectively compared with method 3 which only uses the LTP active learning strategy based on uncertainty sampling. Compared with method 4 which only uses the LDA topic model for diversity classification and then random sampling, the F1 scores on the three data sets are improved by 1.98%, 2.77%, and 6.91% respectively. The reason for the above results is that only using sampling based on uncertainty ignores the relationship between sentences, resulting in insufficient diversity of the sampling results. Similarly, only considering the diversity screening of samples will ignore the information contained in the sentence itself, resulting in a low information content of the sampling results, thereby reducing the performance of the entire model.

[0094] 2) Comparison with the diversity screening method of cosine similarity

[0095] Method 6 uses the same uncertainty screening method and self-training method as the method of the present invention, but uses cosine similarity to screen the diversity of data. Comparative experiments were conducted on the accuracy and recall rate of the three data sets with the method of the present invention. From the experimental results, it can be seen that both have their own advantages in the accuracy and recall rate of the three data sets, but the performance on the comprehensive indicator F1 score shows that the data screening method using LTP+topic model is slightly better than the LTP+cosine similarity method, especially on the Boson data set, the former is 0.55% higher than the latter in F1 score, which shows that the present invention has better effect in screening diverse samples using LDA topic model, and the diversity sample screening method of the present invention takes less time than the method of using world2vec word vector to calculate sentence cosine similarity, and has more advantages in efficiency.

[0096] 3) Verify the effectiveness of the self-training method

[0097] A comparative experiment was conducted on whether to use the self-training method to improve the generalization ability and accuracy of the model. From the experimental results, it was found that compared with method 1 that did not use self-training and continued training, the F1 scores of the method of the present invention on the three data sets were improved by 0.74%, 1.79%, and 0.07%, respectively. Method 5 used the GCE loss function used to process noise labels in deep learning, and continued to perform noise robustness learning on the pseudo-label data generated by the prediction to train the model. Compared with method 6 using the self-training method of the present invention, there was no obvious advantage. Although GCE is robust to noise data in theory, its generalization ability in practical applications is limited by the characteristics of the data set and the type of noise.

[0098] Table 3 Ablation experiment results

[0099]

[0100] Note: Method 1 in the table refers to LDA topic model + LTP (without self-training); Method 2 refers to random screening + self-training; Method 3 refers to LTP + self-training; Method 4 refers to LDA + self-training; Method 5 refers to LTP + cosine similarity + GCE noise learning; Method 6 refers to LTP + cosine similarity + self-training; The method of the present invention refers to LDA topic model + LTP + self-training.

[0101] 4.5 Parameter study

[0102] 1) Determination of the number of topics in the LDA model

[0103] Coherence is an indicator for evaluating the quality of topic models, and its purpose is to measure the coherence of a set of words that are considered similar in a topic. In the LDA model, each topic is represented as a set of words, and coherence mainly evaluates whether this set of words has a certain correlation. The present invention uses the C_v (Coefficient of variance) score to select the appropriate number of topics. This method is based on a sliding window, performs a one-set segmentation on the topic words, compares any two words in a set into word pairs, and uses normalized pointwise mutual information (NPMI) and cosine similarity to indirectly obtain coherence. The C_v Coherence scores of the three data sets are shown in Figure 2. Figure 2 shown.

[0104] Since the goal of the method of the present invention is not to accurately classify the text, the process of selecting the number of topics is simplified. The number of topics corresponding to the highest score is selected. From the figure, it can be seen that the best choice for the number of topics of the MSRA dataset is 15, the number of topics of the Ontonote4 dataset is 20, and the number of topics of the Boson dataset is 25.

[0105] 2) High confidence label threshold selection

[0106] For the threshold ε of high confidence label selection in the self-training phase, the present invention is explored on the Ontonote4 dataset. The results are as follows Figure 3 As shown in the figure, the results show that different threshold selections have an important impact on the performance of the model. As the threshold increases, the accuracy and F1 value of the model show a trend of first increasing, then decreasing, and finally increasing again. As can be seen from the figure, the accuracy and F1 score reach the best value when the ε value is 0.9.

[0107] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0108] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A semi-supervised Chinese place name recognition method combining active learning and self-training, characterized by: The identification method includes the following steps: S1: Get the corpus text containing place names as a dataset The corpus text containing place names includes the Chinese NER public dataset, which includes training set, test set and validation set; S2: Use the LDA topic model to classify the corpus data according to the diversity of the text topics LDA stands for Latent Dirichlet Allocation. It regards each text as a probability distribution consisting of a set of topics, and each topic is a probability distribution consisting of a set of specific words in the text. Each word in the text is distributed with a certain probability. The topic of each text is represented by a set of words with the highest frequency, thus forming a three-layer Bayesian generative model consisting of text, topic and word. Specifically, the corpus is defined as D = (d1, d2, ..., d Z ), consists of Z texts, and a text consists of N d The sequence of words is denoted as d = (w1, w2, ..., w n ), where n∈(1, 2, ... N d ), d∈(1, 2, ..., Z). LDA models D according to the following generative process: 1) Select N d ~Poisson(ξ), where Poisson(ξ) is the Poisson distribution; 2) Select the parameter θ of the text topic distribution d ~Dir(α), where Dir(α) is the Dirichlet distribution; 3) For a word sequence of N d Each word w in the text n : a) Choose a topic n ~Mult(θ), where Mult(θ d ) is a multinomial distribution; b) From the topic t n The word distribution p(w n |t n ,β) to sample a word w n . The LDA model contains some basic assumptions: 1) It is assumed that the dimension k of Dir(α) is known and fixed; 2) It is assumed that the probability of a word is parameterized by a matrix β of dimension k×V; 3) The Poisson distribution is not important for the derivation of the LDA model, and other distributions can be used as needed. Therefore, given α and β, the generation probability of the corpus D is calculated and obtained as follows: The corpus-level parameters α and β are sampled only once in the process of generating the corpus; the text-level variable θ d , sample each text once; the vocabulary-level variable t dn 、w dn Each word in each text is sampled once. Use the LDA topic model to classify all unlabeled samples according to the diversity of topics, and the number of topics is determined by the C_v (Coefficient of variance) score; S3: Active learning method based on LTP (Lowest Token Probability) uncertainty query strategy selects samples with higher uncertainty scores, i.e., more informative samples, according to the prediction results of the model. The LTP score is calculated as shown below: where y * to represent the most likely label sequence for x, and A is a parameter called the transfer matrix, which is learned by the model. The active learning method is used to screen samples based on the LDA topic diversity classification results. The query strategy adopted by active learning is the LTP uncertainty query strategy, and the classifier is BiLSTM-CRF. Before active learning screening, 100-200 manually labeled training samples are randomly screened to initialize the classifier, and finally samples with both diversity and high information are screened for manual labeling. S4: According to steps S2 and S3, a small number of samples with both diversity and high information are selected for manual labeling. Before data screening, all labels of the training set will be removed. When manual labeling is required, the labels of the public dataset will be used as the manual labeling results. S5: Use the self-training method with high confidence tokens selection based on the BERT model to continuously iterate and update the model parameters to obtain the best place name recognition model The self-training method with high confidence tokens selection based on the BERT model is as follows: 1) Using the samples selected for manual annotation based on the LDA topic model and the LTP uncertainty active learning strategy as the Bert-based self-training initialization model, by minimizing The formula for training the model with the following loss is as follows: 2) Using the trained initialization model, predict the sample data that has not been manually labeled to obtain pseudo-label training data; 3) Initialize the teacher model and the student model using the model parameters in step S1; 4) Perform T rounds of iterative self-training, calculate soft labels in each iteration, and optimize and update the new model only on a selected set of high-confidence tokens to finally obtain the place name recognition model with the best performance based on the test results. The formula for generating soft false labels is as follows: in M is the number of data with labels, θ is the parameter of the NER model, and X m is the vector of the mth sample sentence, n is the nth token in the mth sentence, and C is the type. Tokens are selected based on the confidence of the prediction. At the tth iteration, the model loss formula is minimized by selecting a set of high-confidence tokens in the mth sentence as follows: where ε∈(0,1) is the tuning threshold.

Images