A Named Entity Recognition Technology for Environmental Impact Assessment Based on Few-Shot Learning

Abstract

This invention discloses a named entity recognition (NER) technology based on few-shot learning in the environmental impact assessment (EIA) domain. The method includes: acquiring and preprocessing corpora from EIA documents; manually annotating the preprocessed corpora to obtain annotated and unannotated samples; manually collecting and organizing entities in the EIA domain and saving them in an entity database in the form of a vocabulary; expanding the entity database; establishing a named entity recognition model; training the named entity recognition model in stages using the annotated and unannotated samples; obtaining the trained named entity recognition model; and in the prediction stage, correcting the prediction results of the named entity recognition model using the expanded entity database. This invention comprehensively utilizes a small number of manually annotated samples and a manually organized entity database as supervision signals, gradually expanding pseudo-label data in stages, and using mixed data to improve the model's generalization ability, thus training a relatively efficient NER model with less manually annotated data.

Description

Technical Field

[0001] This invention relates to a named entity recognition technology, and more particularly to a named entity recognition technology for the environmental impact assessment field based on few-shot learning. Background Technology

[0002] With the rapid development of artificial intelligence technology, intelligent assisted writing technology has been widely applied in various fields of human production and life, such as automatic contract generation, legal document proofreading, and essay correction. Named Entity Recognition (NER), as a fundamental problem in natural language processing, is a prerequisite and core component of intelligent assisted writing systems, responsible for extracting entities with specific meanings from unstructured text. Its recognition results determine the accuracy of the revision suggestions provided by the intelligent assisted writing system, thus directly affecting user satisfaction. In the field of environmental assessment, because the types of entities to be predicted are novel, labeled data is lacking, and manually labeled samples are expensive, conventional NER technology is not applicable.

[0003] For few-sample NER techniques, the common approach involves two steps. The first step uses a small number of manually labeled samples as supervisory signals to extract useful structural or pseudo-label information from a large amount of unlabeled data, thus transforming the unlabeled data into usable data. The second step trains the NER model using both labeled and transformed data. The most commonly used NER model structure consists of a pre-trained encoder, a bidirectional long-short-term memory (BiLSTM) network, and a conditional random field (CRF). There are many specific training methods for the model. Based on the different methods of information extraction in the first step and training in the second step, the commonly used methods can be divided into the following three categories:

[0004] Method 1 (Reference paper: Snell J, Swersky K, Zemel R. Prototypical networks for few-shot learning[J]. 2017.):

[0005] This method can be transferred to few-shot NER tasks, leveraging meta-learning to solve the few-shot NER problem. The scheme uses a prototypical network, assuming that all entities of the same type have close embedding vector distances in the representation space, thus using the center of these vectors as the embedding vector representation of the entity type. During the prediction phase, the entity type with the shortest distance is selected as the predicted value by comparing the distance between the word's embedding vector and the candidate entity type's embedding vector.

[0006] Method 2 (Reference paper: Jiang H, Zhang D, Cao T, et al. Named entity recognition with small strongly labeled and large weakly labeled data[J]. 2021.):

[0007] This method can be transferred to few-shot NER tasks, addressing the problem from a data perspective. The scheme uses distant supervision, transforming unlabeled data into noisy pseudo-labeled data based on certain assumptions. To ensure model accuracy, the pseudo-labeled data needs to be denoised. Finally, the labeled and pseudo-labeled data are combined to train the NER model using supervised learning.

[0008] Method 3 (Reference: Jiang H, Zhang D, Cao T, et al. Named entity recognition with small strongly labeled and large weakly labeled data[J]. 2021.):

[0009] This method can be transferred to few-shot NER tasks, addressing the few-shot NER problem from a generalization perspective. The scheme uses self-training, which can progressively improve the model's generalization in stages and iterations. In each iteration, high-quality samples are used to ensure the accuracy of the teacher model. Then, high-quality samples are mixed with pseudo-labeled data, and training noise is added to train a student model with stronger generalization.

[0010] Existing few-sample NER methods can use a small number of labeled samples combined with a large-scale unlabeled corpus for joint training to obtain a high-precision NER model. However, these methods are either based on overly simplistic assumptions or have a limited approach to using mixed data, making them unsuitable for practical applications such as environmental impact assessment. Specifically, meta-learning-based methods assume that the embedding vectors of entities belonging to the same entity type are close in the representation space. However, in real-world scenarios, even entities of the same type contain specific semantics, and their distribution in the representation space is unlikely to be guaranteed to cluster together. Therefore, it cannot be guaranteed that the entity type closest to the entity embedding vector is the true entity type. Methods that first transform pseudo-labeled data and then denoise it rely on limited rule assumptions to obtain pseudo-labels, which are often severely lacking or partially biased. Compared to the true labels, the obtained pseudo-labeled data may have a distribution bias that even denoising cannot fundamentally correct, potentially severely affecting the accuracy of the learned model. Self-training methods iteratively expand the pseudo-labeled data with high-precision data in stages to gradually improve the model's generalization ability and reduce potential data distribution bias. However, this method is limited by the process of filtering out pseudo-labeled data. A common approach is to calculate the difference between the distribution of labeled data and the distribution of predicted labeled data as a confidence score, and only select data with high confidence scores for the next stage of training. However, the labels of manually labeled data may not match the true distribution, leading to biased confidence scores and severely affecting the accuracy of the learning model. Summary of the Invention

[0011] Purpose of the invention: The technical problem to be solved by the present invention is to provide a named entity recognition technology in the field of environmental impact assessment based on few-sample learning, which addresses the shortcomings of the existing technology.

[0012] To address the aforementioned technical problems, this invention discloses a named entity recognition technology for the environmental impact assessment field based on few-shot learning, comprising the following steps:

[0013] Step 1: Obtain the corpus from environmental impact assessment documents and preprocess the corpus; manually annotate the preprocessed corpus, annotating at least 10 samples for each entity type to obtain manually annotated samples and unannotated samples.

[0014] Step 2: Manually collect and organize entities in the field of environmental impact assessment, and save them in the form of a thesaurus to the entity database; expand the entity database.

[0015] Step 3: Establish a Named Entity Recognition (NER) model (Reference: Luo Y, Xiao F, Zhao H. Hierarchical Contextualized Representation for Named Entity Recognition[J].2019.). The model consists of a pre-trained encoder, a bidirectional long short-term memory network (BiLSTM), and a conditional random field (CRF). The method for obtaining the pre-trained encoder includes: obtaining a pre-trained encoder in a general domain, and pre-training the corpus in the environmental impact assessment domain to obtain the pre-trained encoder.

[0016] Step 4: Train the Named Entity Recognition (NER) model in stages using manually labeled and unlabeled samples; obtain the trained Named Entity Recognition (NER) model.

[0017] Step 5: In the prediction stage, the prediction results of the Named Entity Recognition (NER) model are corrected using the expanded entity database; the final recognition result is obtained, and the Named Entity Recognition in the Environmental Impact Assessment field based on few-shot learning is completed.

[0018] In step 1 of this invention, the method for preprocessing the corpus includes:

[0019] Preprocess the corpus: delete incomplete sentences, clean up sentences with complex structures containing formulas, deduplicate the corpus and uniformly convert the encoding; manually screen the corpus, if it contains entities of the target entity type, keep it for later use, otherwise remove the next corpus, until at least 10 corpus entries are screened for each target entity type.

[0020] In step 1, the methods for manually annotating the preprocessed corpus include:

[0021] The preprocessed corpus is manually annotated: The preprocessed corpus is manually annotated using the BIO method (BIO is an annotation method that labels each word as BX, IX, or O. Where BX represents the beginning of entity X, IX represents the rest of entity X, and O represents not belonging to any entity). (Reference: Sang EFTK, DeMeulder F. Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition[J].arXiv,2003.) The annotated corpus is the manually annotated sample, and the unannotated corpus is the unannotated sample.

[0022] Step 2 includes methods for expanding the entity database, including:

[0023] Step 2-1: Obtain entity t from the entity database and construct entity vocabulary T;

[0024] Step 2-2: Randomly select a sufficient number of sentences (sufficient number means the number of samples needed to expand the entity library. Assuming the entity library is expected to be expanded by K times, then 5K samples need to be randomly selected to form a sample set S) from the unlabeled samples obtained in Step 1 as the sample set S. The sample set S contains samples s. Compare with the entity vocabulary T to calculate the number of entities contained in samples s, and sort all samples in the sample set from largest to smallest according to the number. Select the first K samples to construct the original sample set S′ for entity expansion.

[0025] Steps 2-3 involve data augmentation of samples s′ in the original sample set S′ using synonym substitution and back-translation methods to obtain the augmented samples s′. p ; Calculate s′ p The perplexity and cosine similarity with the original sample s′, only if s′ p The perplexity is below the threshold S ppl Furthermore, the cosine similarity with the original sample is higher than the threshold S. sim At that time, s′ p Keep qualified enhanced samples for future use; otherwise, discard them.

[0026] Steps 2-4: Compare with qualified enhanced samples s′ p Compared with the original sample s′, examine the modified continuous text region t. span Calculate t span The part of speech of the noun is determined if the probability of it being a noun is higher than the threshold p. noun , then it means t span It is a new entity that will enhance sample s′ p Save the original sample s′ to a medium for later use, and store t span Add to entity library;

[0027] Perform steps 2-3 and 2-4 on all samples in the original sample set S′.

[0028] In step 3, the methods for obtaining the pre-trained encoder include:

[0029] Step 3-1: Obtain the pre-trained encoder for the general domain. pre The pre-training task involves using the preprocessed corpus described in step 1 to pre-train the pre-trained encoder for the general domain for two more rounds. cont Save for future use.

[0030] In step 3, the methods for obtaining the pre-trained encoder include:

[0031] Step 3-2: Obtain the original sample set S′ and the augmented sample set S′ obtained in Step 2. p For the pre-trained encoder saved in step 3-1 cont The pre-trained encoder was trained for two rounds using the Masked Entity Language Modeling (Masked Entity) task. Specifically, entities were masked using the Masked Language Modeling (Masked LM) strategy before being predicted (see: Devlin J, Chang MW, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[J]. 2018.). Entity semantic knowledge was then injected into the pre-trained encoder to obtain the pre-trained encoder. entity .

[0032] In step 3, the methods for obtaining the pre-trained encoder include:

[0033] Step 3-3: Use the pre-trained encoder obtained in step 3-2. entity A Named Entity Recognition (NER) model is assembled with a Bi-directional Long Short-Term Memory (BiLSTM) network and a Conditional Random Field (CRF). During the training phase, manually labeled samples are used, and the embedding vectors of entities are concatenated to the embedding vectors of the manually labeled samples. The NER model is fine-tuned using supervised training, with the loss function being the negative log-likelihood loss.

[0034] Step 4 involves training the Named Entity Recognition (NER) model in stages using manually labeled and unlabeled samples.

[0035] Step 4-1, Obtain manually labeled samples S fewsh For each entity type, select 10 corresponding samples to construct a labeled small sample training set. Select sentences from the unlabeled samples described in step 1 (randomly select 5 times the number of unlabeled samples from the corpus, without special processing), and construct an unlabeled training set.

[0036] Step 4-2, using a small training set The NER model is trained using supervised learning, and the trained model is used as a teacher model and saved for future use.

[0037] Step 4-3, on the unlabeled training set The teacher model is used to make predictions and generate pseudo-labels, forming a pseudo-label dataset S. pseudo ; For S pseudo Each pseudo-label data s in pseudo Calculate the confidence score (a common confidence score measure is to compare the KL divergence between the labeled entity label distribution and the predicted label distribution), and sort the pseudo-label data in descending order of confidence score; select the top N pseudo-label data with high confidence scores to add to the labeled dataset. The expanded labeled dataset is obtained; where the value of N is related to the training of the teacher model. The size ratio is 3 to 5 times;

[0038] Step 4-4: Copy the structure and network parameters of the teacher model to obtain the student model, obtain the expanded labeled dataset, and introduce noise to train the student model. The introduced noise is either gradient noise during the training of the student model or data noise introduced after inserting, shuffling, or deleting the training data.

[0039] Steps 4-5: Use the student model as the teacher model for the next iteration, repeat steps 4-2 to 4-4 to train a new student model, and use the student model obtained after 2 or 3 iterations as the final Named Entity Recognition (NER) model.

[0040] Step 5, the method for correcting the prediction results of the Named Entity Recognition (NER) model using an entity database includes:

[0041] Step 5-1, input the target sample s of the entity to be predicted. pred Using a trained Named Entity Recognition (NER) model, candidate entities t are obtained through prediction. cand .

[0042] Step 5, the method for correcting the prediction results of the Named Entity Recognition (NER) model using an entity database includes:

[0043] Step 5-2: Obtain the entities and candidate entities t from the entity database. cand Compare and calculate with candidate entity t cand The entity t with the largest overlap ratio po And the corresponding overlap ratio p overlap If p overlap Greater than threshold S po And entity t po In s pred If it exists, then the prediction result of the corrected model is t. po Otherwise, no correction is made, and the prediction result remains t. candCompleted named entity recognition in the environmental impact assessment field based on few-shot learning.

[0044] Beneficial effects:

[0045] From a technical perspective, the technical solution of this invention (1) uses a small number of manually labeled samples and a collected entity library to guide the training of the NER model, and uses an algorithm to automatically expand the entity library from the corpus, providing more supervision information for model training at a lower cost; (2) it uses multiple auxiliary tasks to pre-train the encoder, injects environmental impact assessment domain-specific knowledge and entity knowledge into the encoder, and uses a small number of manually labeled samples to fine-tune the NER model, so that the embedding vector representation obtained after the entity is encoded by the encoder contains more semantic information, thus being more effective for NER tasks in the environmental impact assessment domain; (3) it uses a self-training method to comprehensively utilize a small number of manually labeled samples and unlabeled samples to iteratively and in stages to train the NER model. This method can make full use of the supervision information in labeled samples and the semantic information in unlabeled samples, effectively improving the generalization of the model. At the same time, this method can also limit the degree of pseudo-label data distribution shift in each iteration, preventing the introduction of excessive noise that reduces the accuracy of the model; (4) it uses the entity library to correct the prediction results of the model, further improving the accuracy of the final prediction results.

[0046] From an application perspective, the technical solutions of this invention (1) by using entity libraries with lower manual costs to assist labeled samples with higher manual costs, can significantly reduce the number of manually labeled samples required while providing the model with an equal amount of supervisory information, thus saving a large amount of labor costs. (2) It can automatically expand the entity library using large-scale unlabeled corpora, greatly reducing the labor costs required to maintain the integrity of the entity library. (3) It uses an iterative self-training method to support the continued training of the model from incremental data, without having to start training from scratch, thereby helping to update and maintain the model in a timely manner as it develops over time, and improving the model's recognition ability. Attached Figure Description

[0047] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0048] Figure 1 This is a schematic diagram of the overall process of the present invention.

[0049] Figure 2 This is a schematic diagram of the entity library expansion process in this invention.

[0050] Figure 3 This is a schematic diagram of the encoder pre-training process in this invention.

[0051] Figure 4 This is a schematic diagram of the self-training process in this invention.

[0052] Figure 5 This is a schematic diagram of the correction process for the model output in this invention. Detailed Implementation

[0053] The following describes a specific embodiment with reference to the accompanying drawings. This paper implements a method for training a NER model in the field of environmental impact assessment that comprehensively utilizes a small number of manually labeled samples and a manually organized entity database. The NER model adopts the most commonly used structure, namely, it consists of a pre-trained encoder, BiLSTM, and CRF. Figure 1 As shown, the specific steps are described below:

[0054] Step 101: Use a web crawler to download environmental impact assessment (EIA) application forms, review documents, and other attachments from government websites and EIA forums. Open the documents, read the plain text content, and retain only sentences longer than 10 characters to construct a corpus. Preprocess the corpus by deleting incomplete sentences, cleaning up sentences containing complex structures such as formulas, deduplicating, and uniformly converting the encoding before storing it on a medium. Read EIA-related corpus from the medium, manually filter the corpus, and keep any corpus containing the target entity type for later use; otherwise, move on to the next corpus, until at least 10 corpus entries are selected for each target entity type. Manually annotate the selected corpus using the BIO (Browser-Incremental) annotation method. The annotated corpus represents a small sample of manually annotated corpus entries. BIO is a commonly used annotation method that annotates each word as BX, IX, or O. Here, BX represents the beginning of entity X, IX represents the rest of entity X, and O represents not belonging to any entity.

[0055] Step 102 involves having business experts in the environmental impact assessment (EIA) field collect and organize as many commonly used entities as possible in the EIA field, and save them to an entity database in the form of a thesaurus; the process of automatically expanding the entity database using algorithms, such as... Figure 2 As shown:

[0056] Step 201: Read the entities t collected by experts from the entity database and construct the entity vocabulary T.

[0057] Step 202: Randomly select sentences from the environmental impact assessment document corpus to construct a sample set S. Assuming that the entity database is expected to be expanded by K times, 5K samples need to be randomly selected to form the sample set S. Calculate the number of entities contained in the sample s by referring to the entity vocabulary T, and sort all samples in the sample set S from largest to smallest according to the number of entities contained. Select the first K samples to construct the original sample set S′ for entity expansion.

[0058] Step 203: Perform data augmentation on samples s′ in the original sample set S′, for example, by using synonym replacement, back translation, etc., to obtain the augmented samples s′. p ; Calculate s′ p The perplexity and cosine similarity with the original sample s′, only if s′ pThe perplexity is below the threshold S ppl Furthermore, the cosine similarity with the original sample is higher than the threshold S. sim At that time, s′ p Keep qualified enhanced samples for future use; otherwise, discard them.

[0059] Step 204, compare with qualified enhanced sample s′ p Compared with the original sample s′, examine the modified continuous text region t. span Calculate t span The part of speech of the noun is determined if the probability of it being a noun is higher than the threshold p. noun Then it is considered that t span It is a new entity that will enhance sample s′ p Save the original sample s′ to a medium for later use, and store t span Add to the entity library. Perform steps 203 and 204 on all samples in the original sample set.

[0060] Step 103: In this embodiment, the NER model adopts the most general structure, consisting of a pre-trained encoder, BiLSTM, and CRF. This step obtains a pre-trained encoder for the general domain and pre-trains it on the environmental impact assessment corpus. The steps for encoder pre-training are as follows: Figure 3 As shown:

[0061] Step 301: Download the pre-trained general-domain encoder from the open-source website. pre For example, downloading Chinese BERT as an encoder from https: / / huggingface.co / pre The Masked Language Modeling (Masked LM) pre-training task is used to further pre-train the code on the preprocessed environmental impact assessment (EIA) domain corpus for two more rounds, thereby learning EIA-specific knowledge. The pre-trained encoder is then used... cont Save for future use.

[0062] Step 302: Obtain the original sample set S′ and the enhanced sample set S′ saved during the entity library expansion phase. p Retrieve the encoder saved in the previous step. cont The algorithm is pre-trained for two rounds using the Masked Entity Language Modeling (Masked Entity) task. This involves masking entities and then predicting them, similar to Masked LM, with the aim of injecting entity semantic knowledge into the pre-trained encoder. The pre-trained encoder is then used... entity Save for future use.

[0063] Step 303: Use the pre-trained encoder from the previous step. entity The model is assembled with BiLSTM and CRF to form a NER model. A small number of manually labeled samples are obtained. During the training phase, the entity embedding vectors are concatenated to the sample embedding vectors. Supervised training is used to fine-tune the entire NER model, where the loss function is negative log-likelihood loss (NLL loss). After one or two rounds of fine-tuning, the encoder is saved at this point. final , as a backup for the final encoder.

[0064] Step 104 involves iteratively training the NER model in stages using both manually labeled and unlabeled samples, as follows: Figure 4 As shown:

[0065] Step 401: Obtain a small number of manually labeled samples S fewshot For each entity type, 10 corresponding samples are selected to ensure sample balance, and a labeled small sample training set is constructed. Based on the number of labeled samples, five times the number of unlabeled samples are randomly selected from the environmental impact assessment corpus to construct an unlabeled training set.

[0066] Step 402, use the small sample training set constructed in the previous step. The NER model was trained using supervised learning, employing the most common architecture consisting of a pre-trained encoder, BiLSTM, and CRF. The trained model was then saved as a teacher model for future use.

[0067] Step 403, using the unlabeled training set constructed in the previous step The teacher model is used to predict and generate pseudo-labels, forming a pseudo-label dataset S. pseudo ; For S pseudo Each pseudo-label data s pseudo To calculate the confidence score, a common method is to compare the Kullback-Leibler divergence between the labeled entity label distribution and the predicted label distribution, and then sort the pseudo-label data in descending order of confidence score; the top N pseudo-label data with the highest confidence scores are then added to the labeled dataset. This expands the labeled dataset. The value of N is related to the training of the teacher model. The size is proportional, for example, it can be 3 to 5 times.

[0068] Step 404: Copy the structure and network parameters of the teacher model to obtain the student model, and obtain the expanded labeled dataset from the previous step. Introducing noise into the training of student models yields models with stronger generalization performance. The introduced noise can be gradient noise during training, or noise introduced by inserting, shuffling, or deleting data from the training data.

[0069] Step 405: Use the student model as the teacher model for the next iteration, and repeat steps 402, 403, and 404 to train a new student model. This process can be repeated for 2 or 3 rounds. The resulting student model is the final NER model.

[0070] Step 105: In the prediction phase, the actual prediction result is the NER model prediction result after correction to the entity database. The correction process is as follows: Figure 5 As shown:

[0071] Step 501, input the sample s of the entity to be predicted. pred Using the trained NER model, candidate entities t are obtained through prediction. cand .

[0072] Step 502: Obtain the entities and candidate entities t from the entity database. cand In comparison, calculate candidate entity t cand The entity t with the largest overlap ratio po And the corresponding overlap ratio p overlap If p overlap Greater than a certain threshold S po And entity t po In s pred If it exists, then the prediction result of the corrected model is t. po Otherwise, no correction is made, and the prediction result remains t. cand .

[0073] In its specific implementation, this application provides a computer storage medium and a corresponding data processing unit. The computer storage medium is capable of storing a computer program, which, when executed by the data processing unit, can run the invention's content regarding named entity recognition technology in the environmental impact assessment field based on few-shot learning, as well as some or all of the steps in various embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0074] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented using computer programs and their corresponding general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of computer programs, i.e., software products. These computer program software products can be stored in a storage medium and include several instructions to cause a device containing a data processing unit (which may be a personal computer, server, microcontroller, MUU, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0075] This invention provides a concept and method for named entity recognition technology in the field of environmental impact assessment based on few-shot learning. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of this invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A named entity recognition technology for the environmental impact assessment field based on few-shot learning, characterized in that, Includes the following steps: Step 1: Obtain the corpus from environmental impact assessment documents and preprocess the corpus; manually annotate the preprocessed corpus, annotating at least 10 samples for each entity type to obtain manually annotated samples and unannotated samples; Step 2: Manually collect and organize entities in the field of environmental impact assessment, and save them in the form of a thesaurus to the entity database; expand the entity database. Step 3: Establish a Named Entity Recognition (NER) model. The model consists of a pre-trained encoder, a bidirectional long short-term memory network (BiLSTM), and a conditional random field (CRF). The method for obtaining the pre-trained encoder includes: obtaining a pre-trained encoder in a general domain and pre-training the corpus in the environmental impact assessment domain to obtain the pre-trained encoder. Step 4: Train the Named Entity Recognition (NER) model in stages using manually labeled and unlabeled samples; obtain the trained Named Entity Recognition (NER) model. Step 5: In the prediction stage, the prediction results of the Named Entity Recognition (NER) model are corrected using the expanded entity database; the final recognition result is obtained, and the named entity recognition in the environmental impact assessment field based on few-shot learning is completed. Step 2, the method for expanding the entity library, includes: Step 2-1: Obtain entity t from the entity database and construct entity vocabulary T; Step 2-2: Randomly select sentences from the unlabeled samples obtained in Step 1 as sample set S, which contains samples s; Compare with the entity vocabulary T to calculate the number of entities contained in sample s, and sort all samples in the sample set from largest to smallest according to the number; Select the first K samples to construct the original sample set S′ for entity expansion. Steps 2-3 involve data augmentation of samples s′ in the original sample set S′ using synonym substitution and back-translation methods to obtain the augmented samples s′. p ; Calculate s′ p The perplexity and cosine similarity with the original sample s′, only if s′ p The perplexity is below the threshold S ppl Furthermore, the cosine similarity with the original sample is higher than the threshold S. sim At that time, s′ p Keep qualified enhanced samples for future use; otherwise, discard them. Steps 2-4: Compare with qualified enhanced samples s′ p Compared with the original sample s′, examine the modified continuous text region t. span Calculate t span The part of speech of the noun is determined if the probability of it being a noun is higher than the threshold p. noun , then it means t span It is a new entity that will enhance sample s′ p Save the original sample s′ to a medium for later use, and store t span Add to entity library; Perform steps 2-3 and 2-4 on all samples in the original sample set S′.

2. The named entity recognition technology for environmental impact assessment based on few-shot learning according to claim 1, characterized in that, In step 1, the methods for preprocessing the corpus include: Preprocess the corpus: delete incomplete sentences, clean up sentences with complex structures containing formulas, deduplicate the corpus and uniformly convert the encoding; manually screen the corpus, if it contains entities of the target entity type, keep it for later use, otherwise remove the next corpus, until at least 10 corpus entries are screened for each target entity type.

3. The named entity recognition technology for environmental impact assessment based on few-shot learning according to claim 2, characterized in that, In step 1, the methods for manually annotating the preprocessed corpus include: The preprocessed corpus is manually annotated: The corpus obtained after preprocessing is manually annotated using the BIO method. The annotated corpus is the manually annotated sample, and the unannotated corpus is the unannotated sample.

4. The named entity recognition technology for environmental impact assessment based on few-shot learning according to claim 3, characterized in that, In step 3, the methods for obtaining the pre-trained encoder include: Step 3-1: Obtain the pre-trained encoder for the general domain. pre The pre-training task involves using the preprocessed corpus described in step 1 to further pre-train the pre-trained encoder for the general domain for two more rounds. cont Save for future use.

5. The named entity recognition technology for environmental impact assessment based on few-shot learning according to claim 4, characterized in that, In step 3, the methods for obtaining the pre-trained encoder include: Step 3-2: Obtain the original sample set S′ and the augmented sample set S′ obtained in Step 2. p For the pre-trained encoder saved in step 3-1 cont The pre-trained encoder is trained for two rounds using a masked entity language modeling (Masked LM) strategy. This involves masking entities using the Masked Language Modeling (Masked LM) strategy and then predicting the entities, injecting semantic knowledge of the entities into the pre-trained encoder. This results in the pre-trained encoder. entity .

6. The named entity recognition technology for environmental impact assessment based on few-shot learning according to claim 5, characterized in that, In step 3, the methods for obtaining the pre-trained encoder include: Step 3-3: Use the pre-trained encoder obtained in step 3-2. entity A named entity recognition (NER) model is assembled with a bidirectional long short-term memory network (BiLSTM) and a conditional random field (CRF). During the training phase, manually labeled samples are used, and the embedding vectors of entities are concatenated to the embedding vectors of the manually labeled samples. The NER model is fine-tuned using supervised training, with the loss function being the negative log-likelihood loss.

7. The named entity recognition technology for environmental impact assessment based on few-shot learning according to claim 6, characterized in that, Step 4 involves training the Named Entity Recognition (NER) model in stages using manually labeled and unlabeled samples. Step 4-1, Obtain manually labeled samples S fewshot For each entity type, select 10 corresponding samples to construct a labeled small sample training set. Select statements from the unlabeled samples described in step 1 to construct an unlabeled training set. Step 4-2, using a small training set The NER model is trained using supervised learning, and the trained model is used as a teacher model and saved for future use. Step 4-3, on the unlabeled training set The teacher model is used to make predictions and generate pseudo-labels, forming a pseudo-label dataset S. pseudo ; For S pseudo Each pseudo-label data s in pseudo Calculate the confidence score and sort the pseudo-label data in descending order of confidence score; select the top N pseudo-label data with high confidence scores and add them to the labeled dataset. The expanded labeled dataset is obtained; where the value of N is related to the training of the teacher model. The size ratio is 3 to 5 times; Step 4-4: Copy the structure and network parameters of the teacher model to obtain the student model, obtain the expanded labeled dataset, and introduce noise to train the student model. The introduced noise is either gradient noise during the training of the student model or data noise introduced after inserting, shuffling, or deleting the training data. Steps 4-5: Use the student model as the teacher model for the next iteration, repeat steps 4-2 to 4-4 to train a new student model. The student model obtained after 2 or 3 iterations is used as the final Named Entity Recognition (NER) model.

8. The named entity recognition technology for environmental impact assessment based on few-shot learning according to claim 7, characterized in that, Step 5, the method for correcting the prediction results of the Named Entity Recognition (NER) model using an entity database includes: Step 5-1, input the target sample s of the entity to be predicted. pred Using a trained Named Entity Recognition (NER) model, candidate entities t are obtained through prediction. cand .

9. The named entity recognition technology for environmental impact assessment based on few-shot learning according to claim 8, characterized in that, Step 5, the method for correcting the prediction results of the Named Entity Recognition (NER) model using an entity database includes: Step 5-2: Obtain the entities and candidate entities t from the entity database. cand Compare and calculate with candidate entity t cand The entity t with the largest overlap ratio po And the corresponding overlap ratio p overlap If p overlap Greater than threshold S po And entity t po In s pred If it exists, then the prediction result of the corrected model is t. po Otherwise, no correction is made, and the prediction result remains t. cand Completed named entity recognition in the environmental impact assessment field based on few-shot learning.

Images