Water taking permission management data named entity identification method and device
By pre-processing text and labeling of water intake permit management materials in the Yangtze River Basin, combining data augmentation technology and deep learning models, the problem of difficulty in identifying named entities in the water intake permit management materials is solved, and higher recognition accuracy and generalization capabilities are achieved.
Patent Information
- Application Number
- CN202411737824.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-03-25
AI Technical Summary
There are problems of information fragmentation and high complexity in the water intake permit management materials in the Yangtze River Basin, which makes it difficult to identify entities and it is difficult for the existing technology to effectively identify named entities in specific fields.
The BIO annotation method is used to physically label the water withdrawal permission management data to form an initial data set, and a professional data set is constructed through data augmentation techniques such as synonym replacement, random insertion, and random deletion. Then, the named entity recognition model of the feature representation layer, the context feature extraction layer, the bidirectional long and short-term memory neural network layer, and the conditional random field hierarchy cascade is trained to identify the entities in the data to be input.
It effectively avoids the problem that water withdrawal licensed entities are destroyed during the data enhancement process, enhances the variability and scalability of the data set, improves the accuracy and generalization ability of named entity recognition, and can more accurately extract the features of long-sequence water withdrawal licensed texts.
Smart Images
Figure CN119692349A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing, and in particular to a method and device for identifying named entities in water withdrawal permit management data. Background Art
[0002] Water abstraction permit management is an important system of water resources management. It is a basic means to implement unified management of water resources, regulate the relationship between water supply and demand, and strengthen water resources protection. Water abstraction permit management plays a vital role in achieving sustainable use of water resources and supporting sustainable economic and social development. Water abstraction permit management data involves information on various water conservancy objects, including the temporal and spatial distribution of water resources, the geographical location and parameters of water withdrawal and return projects, water user information and other related content. The amount of information involved is large and the complexity is high, which makes the review work cumbersome when approving water abstraction permits and reviewing water use supervision.
[0003] A wealth of textual materials have been accumulated in the field of water abstraction permit management in the Yangtze River Basin. The water abstraction permit management textual materials contain various types of information fragments with great potential utilization value. Making full use of water abstraction permit management materials can effectively improve the efficiency of water abstraction permit management. Compared with the difficulty of managing and interpreting water abstraction permit documents in the Yangtze River Basin, the organizational structure of structured text is clearer. In order to convert the massive water abstraction permit documents in the Yangtze River Basin into structured text and enable better utilization of water resource information, it is necessary to develop a named entity recognition program that can automatically identify and extract water resource data and information from water permit management materials.
[0004] The named entity recognition task of the Yangtze River water abstraction permit management data presents special problems. Water abstraction permit management data are usually compiled by different units and authors, and show variability in text features and patterns. As text data in a specific field, water abstraction permit management data is difficult to recognize complex content such as specific place names and water resource concepts. Summary of the invention
[0005] The present invention provides a method and device for identifying named entities in water withdrawal permit management data, so as to solve the defects in the prior art.
[0006] In a first aspect, the present invention provides a method for identifying named entities in water withdrawal permit management information, comprising: performing text preprocessing on the water withdrawal permit management information, and using the BIO annotation method to perform entity annotation on the preprocessed text to form an initial data set; performing data enhancement on the initial data set to expand samples and construct a professional data set in the field of water withdrawal permits; using the professional data set in the field of water withdrawal permits to train a named entity recognition model for water withdrawal permit management; the named entity recognition model for water withdrawal permit management is cascaded by a feature representation layer, a context feature extraction layer, a bidirectional long short-term memory neural network layer, and a conditional random field layer; the water withdrawal permit management information to be identified is input into the trained named entity recognition model for water withdrawal permit management to determine the entity recognition result.
[0007] According to the method for identifying named entities in water abstraction permit management data provided by the present invention, data enhancement of the initial data set includes a synonym replacement operation; wherein the synonym replacement operation includes: compiling a corresponding dictionary for each type of entity in the water abstraction permit management data, and directly replacing non-water abstraction permit management entities with synonyms; for water abstraction permit management entities, randomly selecting an entity from the dictionary of the corresponding category of the entity according to the entity category to replace the original entity.
[0008] According to the method for named entity recognition of water abstraction permit management materials provided by the present invention, data enhancement of the initial data set includes random insertion, random deletion and random exchange operations; wherein the random insertion operation includes: randomly selecting a synonym of a non-entity in a sentence, and randomly inserting it into a position other than the entity in the sentence; the random deletion operation includes: randomly selecting a word in the sentence, and if it is an entity, reselecting it, and if it is a non-entity, deleting it; the random exchange operation includes: randomly selecting two non-entity words in the sentence for exchange.
[0009] According to the method for named entity recognition of water permit management materials provided by the present invention, data enhancement of the initial data set also includes sentence order exchange, style transfer and context enhancement operations; wherein the sentence order exchange operation includes: randomly exchanging the order of two sentences; the style transfer operation includes: changing the writing style of the text through a pre-trained language model without changing the basic semantics and entities; the context enhancement operation includes: expanding the context of the text through a pre-trained language model to extend the sentence length.
[0010] The method for identifying named entities in water abstraction permit management data provided by the present invention further comprises: the feature representation layer is used to transform the water abstraction permit text sequence W=(W1, W2, ..., W n ) is converted into a water withdrawal permission vector sequence E = (E1, E2, ..., E n); wherein n represents the number of words in the water permit text corresponding to the water permit text sequence; the context feature extraction layer is a RoBERTa pre-trained language model for receiving the water permit vector sequence E = (E1, E2, ..., E n ), output a vector representation sequence T=(T1, T2, ..., T n ); the bidirectional long short-term memory neural network layer is used to receive the vector representation sequence T = (T1, T2, ..., T n ), outputs the named entity prediction result; the conditional random field layer is used to optimize the named entity prediction result based on the constraint relationship between BIO tags and output the final entity prediction result.
[0011] According to the method for identifying named entities in water withdrawal permit management data provided by the present invention, the constraint relationship between BIO tags includes: a B tag cannot be followed by a B tag, but must be followed by an I tag.
[0012] In a second aspect, the present invention further provides a device for identifying named entities in water withdrawal permit management information, comprising:
[0013] The first processing module is used to perform text preprocessing on the water withdrawal permit management data, and use the BIO annotation method to perform entity annotation on the preprocessed text to form an initial data set;
[0014] The second processing module is used to perform data enhancement on the initial data set to expand samples and construct a professional data set in the field of water abstraction permit;
[0015] The third processing module is used to train a named entity recognition model for water permit management using a professional data set in the field of water permit; the named entity recognition model for water permit management is composed of a cascade of a feature representation layer, a context feature extraction layer, a bidirectional long short-term memory neural network layer, and a conditional random field layer;
[0016] The fourth processing module is used to input the water withdrawal permit management data to be identified into the trained water withdrawal permit management named entity recognition model to determine the entity recognition result.
[0017] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method for identifying named entities in water withdrawal permit management data as described in any one of the above-mentioned methods are implemented.
[0018] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for identifying named entities in water withdrawal permit management data as described in any one of the above-mentioned methods.
[0019] The method and device for identifying named entities in water withdrawal permit management data provided by the present invention have the following beneficial effects compared with the prior art:
[0020] (1) In view of the defect that the simple data enhancement commonly used in data enhancement technology will cause the water abstraction permit entity to be destroyed and affect the validity of the data set when it is applied to the Yangtze River water abstraction permit information, the present invention provides a hybrid data enhancement method based on a dictionary and a pre-trained model. By constructing a dictionary for each type of water abstraction permit entity, the problem of water abstraction permit entity being destroyed can be effectively avoided; the data enhancement strategy combined with the pre-trained model enhances the variability and scalability of the data set, and shows a strong data enhancement capability in the field of water abstraction permits with few available data sets.
[0021] (2) In order to solve the problems of weak entity recognition and poor generalization ability of the Yangtze River water abstraction permit data by constructing rule template methods and machine learning methods, the named entity recognition model for water abstraction permit management data provided by the present invention has stronger generalization ability and can extract long-sequence water abstraction permit text features. When faced with water abstraction permit professional entities, more accurate results can be obtained based on the contextual features of the entity. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0023] Figure 1 It is a flow chart of a method for identifying named entities in water withdrawal permit management data provided by the present invention;
[0024] Figure 2 It is a structural diagram of the named entity recognition model for water withdrawal permit management provided by the present invention;
[0025] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0027] It should be noted that, in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0028] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.
[0029] Combine the following Figure 1-Figure 3 The invention describes a method and device for identifying named entities in water withdrawal permit management data provided by an embodiment of the invention.
[0030] Figure 1 : is a flow chart of the method for identifying named entities in water withdrawal permit management data provided by the present invention, such as Figure 1 As shown, including but not limited to the following steps:
[0031] Step 101: Preprocess the water abstraction permit management data, and use the BIO annotation method to perform entity annotation on the preprocessed text to form an initial data set.
[0032] This stage includes operations such as removing irrelevant characters, standardizing text formats (such as converting to lowercase), and word segmentation, with the aim of making the text more suitable for subsequent processing.
[0033] Step 102: Perform data enhancement on the initial data set to expand samples and construct a professional data set in the field of water abstraction permits.
[0034] By transforming the existing data set, such as replacing synonyms, reorganizing sentences, adding noise, etc., more diverse training samples can be generated to increase the generalization ability of the model and reduce the risk of overfitting.
[0035] Specifically, data enhancement of the initial data set includes synonym replacement, random insertion, random deletion, random swap, sentence order swap, style transfer and context enhancement operations, specifically:
[0036] The synonym replacement operation includes: compiling a corresponding dictionary for each type of entity in the water abstraction permit management data, and directly replacing non-water abstraction permit management entities with synonyms; for water abstraction permit management entities, randomly selecting an entity from the dictionary of the corresponding category of the entity to replace the original entity according to the entity category.
[0037] The random insertion operation includes: randomly selecting a synonym of a non-entity in a sentence and randomly inserting it into a position other than the entity in the sentence;
[0038] The random deletion operation includes: randomly selecting a word in the sentence, if it is an entity, reselecting it, if it is a non-entity, deleting it;
[0039] The random exchange operation includes: randomly selecting two non-entity words in the sentence to exchange;
[0040] The operation of sentence order exchange includes: randomly exchanging the order of two sentences;
[0041] The style transfer operation includes: changing the writing style of the text through the pre-trained language model without changing the basic semantics and entities;
[0042] Context-enhanced operations: Expand the context of the text through a pre-trained language model to extend the sentence length.
[0043] Step 103: Use a professional data set in the field of water abstraction permit to train a named entity recognition model for water abstraction permit management; the named entity recognition model for water abstraction permit management is composed of a cascade of a feature representation layer, a context feature extraction layer, a bidirectional long short-term memory neural network layer, and a conditional random field layer.
[0044] Among them, the feature representation layer is used to convert text into a form that can be processed by the machine learning model; the context feature extraction layer is used to capture local and global context information in the text, which may include convolutional neural networks (CNN) or other suitable models; the bidirectional long short-term memory neural network layer can capture sequence information (temporal features) from both the forward and backward directions, which helps to improve the recognition accuracy of entity boundaries; conditional random field layer (CRF): The CRF layer is located at the top layer of the model and is used to further optimize the recognition results of the entity.
[0045] Step 104: Input the water abstraction permit management data to be identified into the trained water abstraction permit management named entity recognition model to determine the entity recognition result.
[0046] The present invention combines the powerful representation ability of deep learning with the advantages of traditional sequence labeling models. It can effectively cope with complex named entity recognition tasks in water permit management data, significantly improve the recognition accuracy, and provide strong technical support for water permit management.
[0047] The technical solution of the present invention is further described below in conjunction with specific embodiments.
[0048] The first step is text preprocessing.
[0049] Due to the professionalism and variability of the Yangtze River water withdrawal permit management data, the present invention uses the "Water Withdrawal Permit Management Measures" and other national standards to perform text preprocessing on the water withdrawal permit management data, including text conversion and removal of special characters, and divides and classifies each type of entity in detail according to the text features of the Yangtze River water withdrawal permit management data. After the preprocessing, more standardized water withdrawal permit management data is obtained to facilitate the training of the entity recognition model.
[0050] After the preprocessing, the water abstraction permit management data uses the BIO annotation method to annotate the entities in the sentences to form an initial data set. The BIO tag is used to mark whether each word or character in the text belongs to a specific type of entity and their position in the entity. BIO is the abbreviation of the following three tags:
[0051] B-Beginning: indicates the beginning of a named entity.
[0052] I-Inside: Indicates the part that is inside the named entity but not at the beginning of the entity.
[0053] O-Outside: Indicates the part that does not belong to any named entity.
[0054] For example, the label of "Changshu Power Plant Phase II Project is located in Changshu City, Jiangsu Province" after BIO annotation is "'B''I''I''I''I''I''I''I''O''O''B''I''I''I''I''I''I'", "Changshu Power Plant Phase II Project" belongs to the project entity, and "Changshu City, Jiangsu Province" belongs to the location entity. The BIO label after adding the corresponding entity type is "'B-Project''I-Project''I-Project''I-Project''I-Project''I-Project''I-Project''I-Project''O''O''B-Address''I-Address''I-Address''I-Address''I-Address''I-Address'". The initial labeled dataset has few samples and needs to be further expanded through data augmentation methods.
[0055] The second step is data enhancement.
[0056] The main purpose of data augmentation methods is to expand the water permit data obtained by text preprocessing by introducing reasonable noise, while avoiding manual labeling and effectively improving model performance. A commonly used data augmentation technique is called Easy Data Augmentation (EDA). EDA includes four key strategies: random replacement, randomly selecting words from the original text and replacing them with synonyms; random insertion, randomly inserting words at different positions in the text data; random swapping, randomly selecting words from the text data and swapping their respective positions; random deletion, randomly deleting some words from the text data. Since the EDA method may change the original entity, directly applying the EDA method to the named entity recognition (NER) task will produce suboptimal results, affect the effectiveness and reduce the overall quality of the dataset.
[0057] In order to avoid the adverse impact of the data enhancement process on the reliability of the data set, the present invention improves the EDA method and proposes a hybrid data augmentation method (Hybrid Data Augmentation, EDA) for text materials in the field of water abstraction permits. The improved HDA method includes four data enhancement modes: synonym replacement, random insertion, random deletion and random exchange:
[0058] (1) Synonym replacement: A corresponding dictionary is compiled for each type of entity under water permit management. For entities not under water permit management, synonyms are used directly for replacement. For water permit management entities, the BIO tag is used to determine their category. Subsequently, an entity is randomly selected from the dictionary of the corresponding category of the entity to replace the original entity. For example, "Changshu Power Plant Phase II Project is located in Changshu City, Jiangsu Province.", where "Changshu Power Plant Phase II Project" is a project entity, then a project entity is randomly selected from the dictionary of project entities to replace it, such as "Shengzhong Reservoir Phase II Expansion Project".
[0059] (2) Random insertion: According to the BIO tag of each word, randomly select non-entity synonyms in the sentence and insert them into random positions in the sentence except for entities. For example, "Changshu Power Plant Phase II Project is located in Changshu City, Jiangsu Province.", where "Changshu Power Plant Phase II Project" and "Changshu City, Jiangsu Province" are water permit management entities, and "located" is a non-entity word. Synonyms of "located" include "geographic location", "in" and "located in". Synonyms can be inserted at the beginning of the sentence, changing the sentence to "Changshu Power Plant Phase II Project located in Changshu City, Jiangsu Province".
[0060] (3) Random deletion: Randomly select a word. If the corresponding BIO tag of the word is an entity, reselect it. If it is a non-entity, delete the word. For example, in "Changshu Power Plant Phase II Project is located in Changshu City, Jiangsu Province.", only "located" is a non-entity. After deleting it, we get "Changshu Power Plant Phase II Project is located in Changshu City, Jiangsu Province."
[0061] (4) Random exchange: select two non-entity words based on the BIO tag to exchange. For example, "The maximum water intake flow rate of the first phase of Changshu Power Plant is 49.8m 3 / s, and the annual permitted water withdrawal is 750 million m 3 ”, after random exchange, we can get “The annual permitted water intake of the first phase of Changshu Power Plant is 49.8m 3 / s, the maximum water intake flow is 750 million m 3 . "
[0062] Based on the above traditional EDA method data enhancement strategy, the present invention proposes several new data enhancement strategies:
[0063] (5) Sentence order swap: Use commas as separators to randomly swap two sentences to enhance data diversity. For example, “Changshu Power Plant is located in Changshu City, Jiangsu Province. The total annual designed water intake of the plant is 1.772 billion m3.” 3 After the exchange, we can get "The total annual designed water intake of Changshu Power Plant is 1.772 billion m 3 , located in Changshu City, Jiangsu Province".
[0064] (6) Style transfer: The writing style or tone of a text is changed through a pre-trained language model without changing the basic semantics and entities. For example, "Jiangsu Changshu Power Generation Co., Ltd. is located in Bixi New District, Changshu City, Jiangsu Province, about 24 km from the urban area of Changshu City." can be changed to "Jiangsu Changshu Power Generation Co., Ltd. is located in the beautiful Bixi New District, Changshu City, Jiangsu Province, about 24 km from the center of Changshu City."
[0065] (7) Context enhancement: Use the pre-trained language model to expand the context of the text and extend the length of the sentence. For example, "The second phase of the Changshu Power Plant is located in Changshu City, Jiangsu Province." After expansion, we can get "The second phase of the Changshu Power Plant is located in Changshu City, Jiangsu Province, with a superior geographical location, providing important support for the region's energy supply and economic development."
[0066] The above HDA method is used to perform data enhancement on the water abstraction permit data obtained through text preprocessing, expand the number of samples, and construct a professional dataset in the field of water abstraction permit.
[0067] The third step is to construct a named entity recognition model for water permit management.
[0068] After completing the construction of a professional dataset in the field of water abstraction permits using the data augmentation method, it is necessary to build a named entity recognition model for water abstraction permits and use the professional dataset in the field of water abstraction permits obtained through data augmentation to train and adjust the model parameters.
[0069] Figure 2 is a schematic diagram of the structure of the named entity recognition model for water withdrawal permit management provided by the present invention, such as Figure 2 As shown in the figure, the water permit management named entity recognition model is composed of a water permit text feature representation layer (which can be called a feature representation layer), a water permit text context feature extraction layer (which can be called a context feature extraction layer), a bidirectional sequence modeling layer (which can be called a bidirectional long short-term memory neural network layer) and a sequence annotation optimization layer (which can be called a conditional random field layer) in cascade order. The water permit management text sequence in the water permit management data set is input into the water permit management named entity recognition model, and the label corresponding to each word is obtained by prediction. The loss function model is further constructed according to the label corresponding to each word, and the water permit management named entity recognition model is obtained through optimization training.
[0070] (1) Water abstraction permit text feature representation layer
[0071] The water permit text feature representation layer is an important part of the water permit model, responsible for converting the input text data into a high-dimensional vector representation. The pre-trained model can only process numerical data and cannot directly read the water permit text sequence W = (W1, W2, ..., W n ), the water abstraction permit text needs to be converted into a vector. Water abstraction permit text sequence W = (W1, W2, ..., W n ) represents a sentence of text, such as "The second phase of the Changshu Power Plant is located in Changshu City, Jiangsu Province." Each element in the text sequence represents a word in the sentence. The input embedding of the pre-trained model consists of three parts: word embedding, position embedding, and segment embedding. Each word in the water permit sentence is mapped to a high-dimensional vector space. The position embedding provides position information for each word, and the segment embedding is used to distinguish different sentences. The output of the embedding layer is the result of adding the three embeddings to form the final vector sequence E = (E1, E2, ..., E n ).
[0072] (2) Water abstraction permit text context feature extraction layer
[0073] In order to extract the context and semantic features of the water permit text, the pre-trained language model RoBERTa is introduced to construct the water permit text context feature extraction layer. The pre-trained model is a transformer-based model. The overall framework is composed of a stack of multiple layers of transformer encoders. Each encoder layer is composed of a layer of muti-head-attention and a layer of feedforward neural network. The water permit vector sequence E obtained by the water permit text feature representation layer is (E1, E2, ..., E n ) will be sent to the water abstraction permit text context feature extraction layer, and features will be extracted through muti-head-attention and feedforward neural networks, and a vector representation sequence T = (T1, T2, ..., T n ).
[0074] (3) Bidirectional sequence modeling layer
[0075] The process of extracting features in the pre-trained model layer focuses on the semantic features of the sentence, while ignoring the temporal features. The bidirectional long short-term memory neural network is a special variant of the recurrent neural network model. The introduction of memory units solves the gradient vanishing and explosion problems of the recurrent neural network, and shows better results in capturing the long-term dependencies and temporal features in the water permit text. The long short-term memory neural network contains three gate structures: forget gate, input gate, and output gate.
[0076] The vector sequence T with water abstraction permit context information output by the water abstraction permit text context feature extraction layer is (T1, T2, ..., T n ) is input to the bidirectional long short-term memory neural network layer, and the forget gate of the long short-term memory neural network receives the output vector h of the previous moment t-1 and the input vector T at the current time t t , determines how much output information should be discarded at the last moment, T t Corresponding vector sequence T = (T1, T2, ..., T n ). The calculation formula is as follows:
[0077] f t =σ(W f ·[h t-1 , T t ]+b f ) (1)
[0078] Among them, W f is the forget weight matrix, b f is the forget bias, x t is the input at time t, f tis the forgetting value, 1 means “completely retained”, and 0 means “completely forgotten”.
[0079] The input gate receives the output vector h at the previous moment t-1 and the current input vector T t , determines how much output information should be retained at the last moment. The input gate normalizes the value used for updating through the sigmoid function to control the current candidate state How much water abstraction permit information needs to be retained? The calculation formula is as follows.
[0080] i t =σ(W i ·[h t-1 , T t ]+b i ) (2)
[0081]
[0082] Among them, i t is the output of the input gate at time t, W i represents the weight matrix of the i-th state, b i represents the deviation vector in state i, W c represents the weight matrix in state C, b c represents the deviation vector, Represents the candidate vector generated at time t.
[0083] According to the previous moment C t-1 and the forgetting value f t , current time C t and the input value i t , discard unnecessary information, keep the required new information, and get the following updated state:
[0084]
[0085] The output gate outputs the vector h according to the previous moment t-1 and the current input vector T t , decide what information needs to be output. First, the initial output is obtained by the third normalization of the sigmoid function, and then C is processed by tanh. t The value of is in the range of [-1, 1]. Finally, the final output is obtained by multiplying it with the output of the sigmoid function. The calculation formula is as follows:
[0086] o t =σ(W o ·[h t-1 , x t ]+b o ) (5)
[0087] h t =o t *tanh(C t ) (6)
[0088] Among them, t is the initial output at time t, W o represents the weight matrix in state o, b o Represents the deviation vector in state o. t is the output of the hidden layer at time t.
[0089] The bidirectional sequence modeling layer finally outputs the score sequence S = (S1, S2, ..., S n ), and select the label with the highest score as the final result.
[0090] (4) Sequence annotation optimization layer
[0091] NER is a sequence tagging task, and there are strong constraints between adjacent tags. However, the bidirectional long short-term memory neural network only considers the long-term contextual information in the water permit sentence, ignoring the dependencies and constraints between BIO tags. For example, the "B" tag cannot be followed by "B", but should be followed by the "I" tag. Because the "B" tag indicates the beginning of a new entity, an entity cannot contain only one word.
[0092] The sequence annotation optimization layer fully considers the constraint relationship between labels, obtains the optimal solution of the label sequence, ensures the rationality of the output sequence, and reduces output errors. Therefore, in this paper, the sequence annotation optimization layer is selected as the final decoding layer to obtain the predicted label sequence of the water abstraction permit named entity, improve the accuracy of water abstraction permit entity recognition, and maintain a reasonable label sequence. The sequence annotation optimization layer uses the formula to calculate the final score of each label:
[0093]
[0094] in, The score sequence S = (S1, S2, ..., S n ), is the transfer matrix between the labels predicted by the sequence labeling optimization layer, x represents the input sequence of the sequence labeling optimization layer, and y is the label sequence. The loss function is defined as:
[0095]
[0096] When the score sequence S = (S1, S2, ..., S n) is input to the sequence labeling optimization layer, and the sequence labeling optimization layer calculates the water withdrawal permit label sequence with the highest score according to formula (7) as the final output of the model.
[0097] The fourth step is named entity recognition of water abstraction permit.
[0098] The water abstraction permit data to be identified is input into the named entity recognition model, which will automatically mark out various entities that may exist in the data.
[0099] The key technical points of the named entity recognition method for water withdrawal permit management data provided by the present invention are mainly reflected in the following two aspects:
[0100] (1) Hybrid data augmentation method based on dictionary and pre-trained model
[0101] When simple data enhancement commonly used in data enhancement technology is applied to the Yangtze River water permit data, the water permit entity will be destroyed, affecting the validity of the data set. The problem of water permit entity destruction can be effectively avoided by building a dictionary for each type of water permit entity. The data enhancement strategy combined with the pre-trained model enhances the variability and scalability of the data set, and shows strong data enhancement capabilities in the field of water permits with few available data sets.
[0102] (2) Named Entity Recognition Model for Water Permit Management
[0103] In response to the problems of weak entity recognition and poor generalization ability of the Yangtze River water abstraction permit data by constructing rule template methods and machine learning methods, the named entity recognition model for water abstraction permit management data has stronger generalization ability, can extract long-sequence water abstraction permit text features, and obtain more accurate results based on the contextual features of the entity when facing water abstraction permit professional entities.
[0104] On the other hand, the present invention also provides a method for identifying named entities in water withdrawal permit management information, comprising:
[0105] The first processing module is used to perform text preprocessing on the water withdrawal permit management data, and use the BIO annotation method to perform entity annotation on the preprocessed text to form an initial data set;
[0106] The second processing module is used to perform data enhancement on the initial data set to expand samples and construct a professional data set in the field of water abstraction permit;
[0107] The third processing module is used to train a named entity recognition model for water permit management using a professional data set in the field of water permit; the named entity recognition model for water permit management is composed of a cascade of a feature representation layer, a context feature extraction layer, a bidirectional long short-term memory neural network layer, and a conditional random field layer;
[0108] The fourth processing module is used to input the water withdrawal permit management data to be identified into the trained water withdrawal permit management named entity recognition model to determine the entity recognition result.
[0109] It should be noted that the device for identifying named entities in water abstraction permit management data provided in the embodiment of the present invention can execute the method for identifying named entities in water abstraction permit management data described in any of the above embodiments during specific operation, which will not be elaborated in this embodiment.
[0110] Figure 3 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the method for identifying named entities of water withdrawal permit management data, the method comprising: performing text preprocessing on the water withdrawal permit management data, and using the BIO annotation method to perform entity annotation on the preprocessed text to form an initial data set; performing data enhancement on the initial data set to expand the sample to construct a professional data set in the field of water withdrawal permit; using the professional data set in the field of water withdrawal permit, training a water withdrawal permit management named entity recognition model; the water withdrawal permit management named entity recognition model is formed by cascading a feature representation layer, a context feature extraction layer, a bidirectional long short-term memory neural network layer and a conditional random field layer; inputting the water withdrawal permit management data to be identified into the trained water withdrawal permit management named entity recognition model to determine the entity recognition result.
[0111] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the named entity recognition method for water withdrawal permit management information provided in the above-mentioned embodiments, and the method includes: performing text preprocessing on the water withdrawal permit management information, and using the BIO annotation method to perform entity annotation on the preprocessed text to form an initial data set; performing data enhancement on the initial data set to expand the samples and construct a professional data set in the field of water withdrawal permits; using the professional data set in the field of water withdrawal permits to train a named entity recognition model for water withdrawal permit management; the named entity recognition model for water withdrawal permit management is composed of a cascade of a feature representation layer, a context feature extraction layer, a bidirectional long short-term memory neural network layer, and a conditional random field layer; the water withdrawal permit management information to be identified is input into the trained named entity recognition model for water withdrawal permit management to determine the entity recognition result.
[0112] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the method for named entity recognition of water withdrawal permit management data provided in the above-mentioned embodiments, the method comprising: performing text preprocessing on the water withdrawal permit management data, and using the BIO annotation method to perform entity annotation on the preprocessed text to form an initial data set; performing data enhancement on the initial data set to expand the samples and construct a professional data set in the field of water withdrawal permits; using the professional data set in the field of water withdrawal permits to train a named entity recognition model for water withdrawal permit management; the named entity recognition model for water withdrawal permit management is composed of a cascade of a feature representation layer, a context feature extraction layer, a bidirectional long short-term memory neural network layer, and a conditional random field layer; the water withdrawal permit management data to be identified is input into the trained named entity recognition model for water withdrawal permit management to determine the entity recognition result.
[0113] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for identifying named entities in water abstraction permit management data, characterized in that: include: The water abstraction permit management data was preprocessed, and the preprocessed text was annotated using the BIO annotation method to form an initial data set. Performing data enhancement on the initial data set to expand samples and construct a professional data set in the field of water abstraction permit; Using a professional data set in the field of water abstraction permit, a named entity recognition model for water abstraction permit management is trained; the named entity recognition model for water abstraction permit management is composed of a cascade of a feature representation layer, a context feature extraction layer, a bidirectional long short-term memory neural network layer, and a conditional random field layer; The water withdrawal permit management data to be identified is input into the trained water withdrawal permit management named entity recognition model to determine the entity recognition result.
2. The method for identifying named entities in water abstraction permit management data according to claim 1, characterized in that: Performing data enhancement on the initial data set including an operation of synonym replacement; The synonym replacement operation includes: compiling a corresponding dictionary for each type of entity in the water abstraction permit management data, and directly replacing non-water abstraction permit management entities with synonyms; for water abstraction permit management entities, randomly selecting an entity from the dictionary of the corresponding category of the entity to replace the original entity according to the entity category.
3. The method for identifying named entities in water abstraction permit management data according to claim 1, characterized in that: Performing data enhancement on the initial data set includes random insertion, random deletion, and random exchange operations; The random insertion operation includes: randomly selecting a synonym of a non-entity in a sentence and randomly inserting it into a position other than the entity in the sentence; The random deletion operation includes: randomly selecting a word in the sentence, if it is an entity, reselecting it, if it is a non-entity, deleting it; The random exchange operation includes: randomly selecting two non-entity words in a sentence for exchange.
4. The method for identifying named entities in water abstraction permit management data according to claim 1, characterized in that: The data enhancement of the initial data set also includes operations of sentence order exchange, style transfer, and context enhancement; The operation of sentence order exchange includes: randomly exchanging the order of two sentences; The style transfer operation includes: changing the writing style of the text through the pre-trained language model without changing the basic semantics and entities; Context-enhanced operations: Expand the context of the text through a pre-trained language model to extend the sentence length.
5. The method for identifying named entities in water abstraction permit management data according to claim 1, characterized in that: Also includes: The feature representation layer is used to convert the water withdrawal permit text sequence W=(W1, W2, …, W n ) is converted into a water withdrawal permission vector sequence E = (E1, E2, ..., E n ), wherein n represents the number of words in the water permit text corresponding to the water permit text sequence; The context feature extraction layer is a RoBERTa pre-trained language model for receiving a water withdrawal permission vector sequence E=(E1, E2, ..., E n ), outputs a vector representation sequence T=(T1, T2, …, T n ); The bidirectional long short-term memory neural network layer is used to receive the vector representation sequence T = (T1, T2, ..., T n ), output the named entity prediction result; The conditional random field layer is used to optimize the named entity prediction results based on the constraint relationship between BIO tags and output the final entity prediction results.
6. The method for identifying named entities in water abstraction permit management data according to claim 5, characterized in that: The constraints between BIO tags include: A B tag cannot be followed by a B tag, but must be followed by an I tag.
7. A named entity recognition device for water withdrawal permit management data, characterized in that: include: The first processing module is used to perform text preprocessing on the water withdrawal permit management data, and use the BIO annotation method to perform entity annotation on the preprocessed text to form an initial data set; The second processing module is used to perform data enhancement on the initial data set to expand samples and construct a professional data set in the field of water abstraction permit; The third processing module is used to train a named entity recognition model for water permit management using a professional data set in the field of water permit; the named entity recognition model for water permit management is composed of a cascade of a feature representation layer, a context feature extraction layer, a bidirectional long short-term memory neural network layer, and a conditional random field layer; The fourth processing module is used to input the water withdrawal permit management data to be identified into the trained water withdrawal permit management named entity recognition model to determine the entity recognition result.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method for named entity recognition of water withdrawal permit management data as described in any one of claims 1 to 6 are implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for named entity recognition of water withdrawal permit management data as described in any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for named entity recognition of water withdrawal permit management data as described in any one of claims 1 to 6 are implemented.