A named entity recognition method based on prompt learning and label perception

Abstract

The application belongs to the technical field of natural language processing, and particularly relates to a named entity recognition method based on prompt learning and label perception; comprising: splicing a plurality of prompt templates in front of each sentence in preprocessed named entity recognition data; adding prompt masks to the processed sentences and inputting the processed sentences into a BERT model to obtain semantic representations of the prompt slots and the sentences; inputting entity labels into the BERT model to obtain label semantic representations; inputting the semantic representations of the sentences into a bidirectional LSTM and processing the semantic representations of the prompt slots using a self-attention mechanism to further obtain a sentence context semantic matrix and a prompt slot deep semantic matrix; calculating a classification slot probability distribution according to the prompt slot deep semantic matrix and the label semantic representations to realize entity classification; calculating a positioning slot probability distribution according to the sentence context semantic matrix and the prompt slot deep semantic matrix to realize entity positioning; and the application can significantly improve the entity extraction capability of the model while reducing time and space overhead.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, specifically relating to a named entity recognition method based on prompting learning and label awareness. Background Technology

[0002] Named Entity Recognition (NER) is one of the most fundamental tasks in NLP. This task aims to identify and classify nouns with special meanings from text of predefined semantic types, such as names of people, places, and organizations.

[0003] Early Neural Error Reduction (NER) primarily relied on linguists manually constructing rule templates based on linguistic knowledge. Supervised learning, the most widely used method in statistical machine learning, mainly involves selecting appropriate features from textual information as the basis for entity classification. In recent years, deep learning-based NER models have become mainstream. The release of models like Transformer and BERT ushered in a new era of pre-trained language models, with pre-training plus fine-tuning achieving outstanding performance. Recently, cue learning, as the fourth paradigm of NLP, has also attracted widespread research interest, with the pre-training + cue + prediction approach gaining traction. A classic approach to applying cue learning to NER iteratively obtains all candidate entity fragments using an N-gram method, then concatenates them with a hand-designed template, and uses the BART model to score each fragment to predict the entity category. This method significantly outperforms traditional sequence labeling methods and distance-based few-shot NER methods in cross-domain and few-shot scenarios. A corresponding approach is to construct cue for each entity type and then guide the model to locate specific types of entities. This approach avoids the drawback of excessive iterations in TemplateNER and is suitable for nested NER. One approach uses soft templates to transform the original sequence labeling task into a sequence-to-sequence generation task. By incorporating cue information into the self-attention mechanism, the model achieves better performance. During training, the parameters of the pre-trained model are fixed, and only cue-related parameters are optimized, making the model more flexible and lightweight. The PromptNER method uses a dual-slot multi-cue template and a dynamic template filling mechanism to fill all cue templates in parallel, achieving full entity output in a single round of cueing, thus improving efficiency while maintaining performance.

[0004] However, most of the aforementioned prompt learning models suffer from problems such as high time overhead, template sensitivity, independent prompts, poor performance in low-sample environments, and limited task applicability. For example, TemplateNER, which iteratively generates spans, requires N(N+1) / 2 rounds of prompts (where N is the total number of tokens in a sentence). Each round of prompts in these models is independent, ignoring the potential relationships between prompts. For instance, the PromptNER method reduces time complexity by providing prompts before matching, but its localization and classification slots are functionally isolated, lacking cross-task interaction, easily losing much useful prior information, and its entity repetition mechanism is rather crude, easily causing template confusion. The use of dual slots also implicitly increases the limitation on text length. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a named entity recognition method based on cue learning and label awareness. The method includes: acquiring the text to be recognized and preprocessing it; inputting the preprocessed text into a trained named entity recognition model for processing; and obtaining the named entity recognition result.

[0006] The training process of a named entity recognition model includes:

[0007] S1: Obtain the named entity recognition dataset and preprocess it to obtain preprocessed named entity recognition data;

[0008] S2: Concatenate multiple prompt templates before each sentence in the named entity recognition data, and unify the length of the sentence after concatenating the prompt templates;

[0009] S3: After adding a cue mask to the sentence following the concatenated cue template, input it into the BERT model to obtain the cue slot semantic representation and the sentence semantic representation;

[0010] S4: Input the entity labels into the BERT model to obtain the semantic representation of the labels;

[0011] S5: Input the sentence semantic representation into a bidirectional LSTM network to obtain the sentence context semantic matrix; process the cue slot semantic representation using a self-attention mechanism based on the sentence context semantic matrix to obtain the cue slot deep semantic matrix;

[0012] S6: Map the deep semantic matrix of the cue slot to the classification slot coding matrix through a linear layer, and calculate the classification slot probability distribution based on the classification slot coding matrix and the label semantic representation;

[0013] S7: Calculate the probability distribution of the location slots based on the deep semantic matrix of the cue slots and the semantic matrix of the sentence context;

[0014] S8: Calculate the minimum matching loss between the prompt slot and the entity based on the probability distribution of the classification slot and the probability distribution of the positioning slot, and obtain the optimal matching scheme;

[0015] S9: Calculate the total loss of the model under the optimal matching scheme, adjust the model parameters according to the total loss of the model, and obtain the trained named entity recognition model.

[0016] Preferably, the semantic representation of the cue slot is processed using a self-attention mechanism as follows:

[0017]

[0018] Among them, H E H represents the original semantic representation of the cue slot. X H represents the semantic matrix of sentence context. S H represents the semantic representation of the prompt slot after cross-slot interaction. D This indicates that the semantic representation of the cue slots takes into account both inter-slot interactions and interactions between slots and sentences, i.e., the deep semantic representation of cue slots. and W represents the key, query, and value matrix of the suggestion slot, respectively. k W q and W v These represent the first, second, and third weight matrices, respectively. and L represents the key-value matrix of the sentence; h represents the slot identifier embedding matrix; h represents the dimension of the hidden vector; and Softmax represents the activation function.

[0019] Preferably, the process of calculating the probability distribution of the slot includes: mapping the deep semantic matrix of the cue slot to the slot encoding matrix through linear mapping; performing a dot product operation between the slot encoding matrix and the label semantic representation to obtain the matching degree matrix between the cue slot and the entity category; and decoding the matching degree matrix between the cue slot and the entity category through a linear layer to obtain the probability distribution of the slot.

[0020] Preferably, the process of calculating the probability distribution of the positioning slot includes:

[0021] The deep semantic matrix of the cue slot is linearly mapped to obtain the localization slot encoding matrix;

[0022] The first matrix is ​​obtained by adding the local slot encoding matrix to the sentence context semantic matrix after linear mapping.

[0023] The second matrix is ​​obtained by adding the class slot encoding matrix to the sentence context semantic matrix after linear mapping;

[0024] The first matrix is ​​added to the second matrix, and the location slot probability distribution is obtained by decoding through a linear layer.

[0025] Preferably, the process of calculating the matching loss between the cue slot and the entity includes:

[0026] The sampling weight of each entity is calculated based on the probability distribution of the classification slot and the probability distribution of the location slot;

[0027] Based on the sampling weight, the entity is repeatedly sampled to fill all the cue slots, and the matching schemes of all cue slots and entities are obtained.

[0028] Calculate the minimum matching loss among all matching schemes between the prompt slot and the entity, and select the matching scheme with the minimum matching loss as the optimal matching scheme.

[0029] Preferably, the formula for calculating the sampling weight of each entity is:

[0030]

[0031] Among them, MC j Weight represents the total number of suggestion slots in the sentence that match the j-th real entity. j The sampling weight corresponding to the j-th real entity in the sentence is represented by M, where M is the number of cue slots and A is the total number of entities in the original sentence. This indicates the matching category, left boundary, and right boundary values ​​obtained when slot k matches the j-th real entity; max k This indicates the index of the real entity that best matches slot k, and equal is a flag function.

[0032] Preferably, the formula for calculating the minimum matching loss among all matching schemes between the cue slot and the entity is:

[0033]

[0034] Where, σ * This represents the optimal matching scheme, where M represents the number of hint slots. Indicates the index function, This represents the confidence probability value corresponding to the true category output by slot i under the σ-th permutation. Let represent the confidence probability values ​​corresponding to the true left and right boundaries of the prompt slot i under the σ-th permutation, and argmin represents the function for finding the minimum value.

[0035] Preferably, the total model loss is the weighted sum of the localization loss and the classification loss, where:

[0036] Classification loss L type Represented as:

[0037]

[0038] Positioning loss L pOsitionRepresented as:

[0039]

[0040] in, The permutation σ represents the optimal matching. * The confidence probability value corresponding to the true category output by the lower suggestion slot i. This represents the confidence probability values ​​corresponding to the true left and right boundaries output by slot i under the optimal matching permutation. This represents the indicator function, and M represents the number of prompt slots.

[0041] The beneficial effects of this invention are as follows: This invention uses a single-slot dual-function template as a framework, avoiding the high time overhead common in conventional cue learning, and reducing the size of the cue template in the input. These two optimizations improve the model's inference speed. This invention achieves interaction between different cue slots and within the same cue slot through an interaction layer and cross-task representation fusion, integrating the semantics of localization and classification tasks. Prior knowledge is used to achieve mutual assistance between localization and classification, improving the model's entity extraction capability. This invention proposes an encouraging nearest-neighbor template filling mechanism, which alleviates template confusion while increasing the number of entities extracted by the model, thus mitigating the problem of misjudgment in the later stages of training. This invention introduces the natural semantics of labels, implicitly incorporating external knowledge, enabling the model to learn deeper semantics even in low-sample scenarios. Furthermore, contrastive learning allows the distributed semantic representation generated by the model to be as far away from non-entity categories as possible. In low-sample environments, external knowledge improves the model's accuracy, mitigating the problem of insufficient learning in low-sample environments. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating the training process of the named entity recognition model in this invention.

[0043] Figure 2 This is a structural diagram of the named entity recognition model in this invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] This invention proposes a named entity recognition method based on cue learning and label awareness, the method comprising the following:

[0046] The text to be recognized is obtained and preprocessed. The preprocessed text is then input into the trained named entity recognition model for further processing to obtain the named entity recognition result.

[0047] like Figure 1 , Figure 2 As shown, the training process of the named entity recognition model includes:

[0048] S1: Obtain the named entity recognition dataset and preprocess it to obtain preprocessed named entity recognition data.

[0049] The data preprocessing process includes: removing stop words from the original dataset, modifying the traditional sequence labeling dataset format to span representation, and representing a sentence and its contained entities as an object, with the entire dataset represented as an array of objects. Sentences are represented by the "tokens" field, whose value is an array of tokens obtained by dividing the sentence by characters. The entity field "entities" is an array consisting of all entity objects contained in the sentence. Each object has three attributes: start, end, and type. start and end represent the indices of the entity's left and right boundaries, respectively, and type represents the entity's category.

[0050] S2: Concatenate multiple prompt templates before each sentence in the named entity recognition data, and unify the length of the sentence after concatenating the prompt templates.

[0051] Multiple prompt templates are added in front, each consisting of a prompt slot and a prompt context. The entire input is converted into a dictionary ID representation of the BERT model. If the input exceeds the maximum input length of the model, the excess part is truncated, and the insufficient part is padded with the special character [PAD]. This ensures that each sentence in a training batch is processed to the same length.

[0052] Cue templates can use hard templates, where the cue context uses manually designed specific lexical units. For example, using the hard template "Entity is[E]" means that "Entity is" is the cue context, and "[E]" represents the cue slot. Alternatively, soft templates can be used to allow the model to learn the cue context during training. <soft>[E]”, for <soft>Random initialization is performed, and this semantic vector is updated during training. The cue slots [E] are eventually decoded into the entity's location and class probability distribution. Using hard templates, the final input to the model is:

[0053] Input(X)=Entity is[E1]Entity is[E2]…Entity is[E n [SEP]x1x2…x n [PAD]…

[0054] Where X represents the sentence, Entity is provides contextual clues, and E n This represents the nth suggestion slot, x n [SEP] represents the nth word in the sentence. [SEP] is a special symbol in the vocabulary used to separate the prompt template from the sentence. [PAD] represents the filler character.

[0055] S3: After adding a cue mask to the sentence following the concatenated cue template, input it into the BERT model to obtain the cue slot semantic representation and the sentence semantic representation.

[0056] Because the self-attention mechanism inherent in the BERT model causes sentence X to focus on randomly initialized cue templates during the initial encoding stage, thus compromising the original semantics of the sentence, adjustments are necessary. Here, a cue mask is used to prevent the sentence from focusing on the cue template information. Then, BERT encoding is performed on the input based on this mask. The formulas involved in the above steps are:

[0057] H = vector(Input(X))

[0058]

[0059] PA(H)=αHW v

[0060] H′=BERT PA (H)

[0061] Where X represents the sentence, Vector(Input(X)) represents the word embedding representation of the input, and W q ∈R h×h W represents the query vector parameter matrix. k ∈R h×h W represents the key vector parameter matrix. v ∈R h×h Represents a value vector parameter matrix, I∈R (N+M)×(N+M) The cue mask matrix I is defined as follows: elements that retain attention are set to 0, while elements that are masked (i.e., elements at the corresponding positions of the cue template) are set to negative infinity (M is the number of cue slots, and N is the length of the original sentence X). Softmax is an activation function that maps values ​​to 0 to 1 such that the sum is 1. PA(H) represents a cue-independent unidirectional self-attention mechanism, T represents the matrix transpose, h and D represent the dimensions of the vectors, and H′ represents the semantic representation H of the cue slots. E And the semantic representation of the sentence obtained by BERT encoding the input based on the cue mask.

[0062] S4: Input the entity labels into the BERT model to obtain the semantic representation of the labels.

[0063] The entity category set, i.e. entity labels, needs to be mapped to natural language descriptions. For example, if the entity category set = [organization, person, location], the natural language description would be: set = [organization, person, location].

[0064] If the label description is abbreviated, it needs to be manually labeled. The processed result is then input into BERT for label semantic encoding to obtain the label semantic representation.

[0065] S5: Input the sentence semantic representation into a bidirectional LSTM network to obtain the sentence context semantic matrix; process the cue slot semantic representation using a self-attention mechanism based on the sentence context semantic matrix to obtain the cue slot deep semantic matrix.

[0066] Sentence encoding requires a bidirectional LSTM module to enhance context awareness, thus serving subsequent boundary determination. After passing through the LSTM module, the sentence encoding X changes. A bidirectional LSTM is used here; the formula for the forward LSTM is:

[0067] i t =σ(W i x t +W i′ h t-1 +b i )

[0068] f t =σ(W f x t +W f′ h t-1 +b f )

[0069] o t =σ(W o x t +W o′ h t-1 +b o )

[0070] g t =tanh(W g x t +W g′ h t-1 +b g )

[0071] c t =f t ·c t-1 +i t ·g t

[0072] h t =o t ·tanh(c t )

[0073] Where, x t It is the input of the input sequence at time t, h t-1 It is the hidden state of the previous time step, i t f t o t It is the output of the input gate, forget gate, and output gate. t It represents the cell state at the current time step, h. t It is the hidden state at the current time step, W i W i′ W f W f′ W o W o′ W g W g′ Let b be the weight matrix. i b f b o b g σ is the bias vector, σ is the sigmoid function, and tanh is the hyperbolic tangent function.

[0074] The formula for backward LSTM is:

[0075] i′ t =σ(W′) i x t +W′i ′ h t+1 +b′ i )

[0076] f′ t =σ(W′) f x t +W′ f′ h t+1 +b′ f )

[0077] o′ t =σ(W′) o x t +W′ o′ h t+1 +b′ o )

[0078] g′ t =tanh(W′) g x t +W′ g′ h t+1 +b′ g )

[0079] c′ t =f′ t ·c′ t-1 +i′ t ·g′ t

[0080] h′ t =o′ t ·tanh(c′ c )

[0081] Where, x t It is the input of the input sequence at time t, h t+1 It is the hidden state of the next time step, i t f t o t It is the output of the input gate, forget gate, and output gate. t It represents the cell state at the current time step, h. t It is the hidden state at the current time step, W′ i 、W′ i′ 、W′ f 、W′ f′ 、W′ o 、W′ o′ 、W′ g 、W′ g′ Let b′ be the weight matrix. i b′ f b′ o b′ g σ is the bias vector, σ is the sigmoid function, and tanh is the hyperbolic tangent function.

[0082] After the above steps, the sentence context semantic matrix H is obtained. 3 H X With H E Through the interaction layer, the self-attention mechanism of the Transformer encoder is used to achieve self-attention between cue slots and cross-attention between cue slots and sentences, thereby obtaining a deep semantic representation of the cue slots, represented as:

[0083]

[0084] Among them, H E H represents the original semantic representation of the cue slot. X H represents the semantic matrix of sentence context. s H represents the semantic representation of the prompt slot after cross-slot interaction. D This indicates that the semantic representation of the cue slots takes into account both inter-slot interactions and interactions between slots and sentences, i.e., the deep semantic representation of cue slots. and W represents the key, query, and value matrix of the suggestion slot, respectively. k W q and W v These represent the first, second, and third weight matrices, respectively. and L represents the key-value matrix of the sentence; L represents the slot identifier embedding matrix; and Softmax represents the activation function.

[0085] S6: Map the deep semantic matrix of the cue slot to the classification slot coding matrix through a linear layer, and calculate the classification slot probability distribution based on the classification slot coding matrix and the label semantic representation.

[0086] After the above encoding process is completed, the next step is decoding, which involves obtaining the probability output based on the features extracted from the representation vector of the cue slot. First, classification is performed, using a linear mapping to transform the deep semantic matrix H of the cue slot. D The mapping is transformed into a slot encoding matrix; the slot encoding matrix is ​​then multiplied by the label semantic representation to obtain the matching degree matrix between the cue slot and the entity category; this matching degree matrix is ​​then decoded through a linear layer to obtain the slot probability distribution, thus achieving entity classification. The formula is as follows:

[0087] cls result =Softmax(W1(W2H) D +b1)·H l )+b2)

[0088]

[0089] Among them, cls result ∈R M×K Let W1∈R be the probability distribution related to entity classification. h×h W2∈R K×K Here is a trainable weight matrix, b1 and b2 are biases, and H... E ∈R M×h For the semantic representation of the cue slot, H l ∈R K×h For the semantic representation of tags, Let represent the probability that the entity extracted by the i-th slot belongs to the j-th category, K is the number of entity categories, and h represents the dimension of the hidden vector.

[0090] S7: Calculate the probability distribution of the location slots based on the deep semantic matrix of the cue slots and the semantic matrix of the sentence context.

[0091] The deep semantic matrix of the cue slot is linearly mapped to obtain the localization slot encoding matrix;

[0092] The first matrix is ​​obtained by adding the local slot encoding matrix to the sentence context semantic matrix after linear mapping.

[0093] The second matrix is ​​obtained by adding the class slot encoding matrix to the sentence context semantic matrix after linear mapping;

[0094] The first matrix is ​​added to the second matrix, and then decoded again through a linear layer to obtain the probability distribution of the location slot, thus achieving entity localization. The above operation can be represented as:

[0095]

[0096] Among them, W l1 W l2 W l3 W l4 W l5 ∈R h×h W is the trainable left boundary parameter matrix. r1 W r2 W r3 W r4 W r5 ∈R h×h W is the trainable right boundary parameter matrix. p ∈R h×h and W t ∈R h×h These are the mapping matrices that map the deep semantic representation of the cue slots to the localization slot encoding and the classification slot encoding, respectively, where h represents the dimension of the hidden vector, and b... p and b t For bias, This is the deep semantic representation of the cue slot corresponding to cue slot i; and For the probability distribution of the positioning slot, This indicates the probability that the entity extracted from the prompt slot uses j as its left boundary index. This indicates the probability that the entity extracted from slot i uses j as its right boundary index value.

[0097] S8: Calculate the minimum matching loss between the prompt slot and the entity based on the probability distribution of the classification slot and the probability distribution of the positioning slot, and obtain the optimal matching scheme.

[0098] Since the correspondence between the prompt template and the entity is unknown, it is impossible to assign labels to the slots in advance. Therefore, slot filling is regarded as a linear assignment problem, in which any entity can be filled into any prompt template. This will generate a loss value, and then the optimal match can be calculated based on the loss value.

[0099] Because the number of prompt templates exceeds the number of actual entities, many prompts will match... Training efficiency is reduced. To improve the utilization of the cue template, one-to-many matching is achieved by repeatedly sampling real entities, i.e., a dynamic template filling mechanism. Nearest neighbor template filling is used here. First, the best-matching entity is calculated for each cue slot, and this is used as the sampling weight to repeatedly sample entities. The goal is to maximize recall while ensuring that the template prioritizes matching with the nearest neighbor entity, avoiding template confusion. The sampling weight formula is:

[0100]

[0101] Among them, MC j Weight represents the total number of suggestion slots that match the j-th real entity in a sentence. j This represents the sampling weight corresponding to the j-th real entity in a sentence, M is the number of cue slots, and A is the original total number of entities in a sentence. This represents the confidence score obtained by matching the j-th real entity with the suggestion slot k in a sentence, which is related to the entity classification, left boundary, and right boundary. k This indicates the index of the most matching real entity in the hint slot k. `equal` is a flag function; it returns 1 if the two parameters within `equal` are equal, and 0 otherwise. That is, if the most matching real entity in the hint slot k is the j-th real entity, then `equal` returns 1; otherwise, it returns 0. If MC... j If the value is 0, meaning there is no cue slot matching the j-th entity, then the numerator of the sampling weight is 1 to ensure that each real entity always has a small probability of being sampled repeatedly.

[0102] Based on the sampling weight, entities are repeatedly sampled until the number of real entities becomes 0.9*M, meaning that 90% of the prompt slots can be matched one-to-one with real entities, and only 10% of the prompt slots will match empty values. These repeated real entities are used to fill the prompt slots to obtain all the matching schemes between prompt slots and entities.

[0103] Calculate the minimum matching loss among all matching schemes between the cue slot and the entity, and select the matching scheme with the minimum matching loss as the optimal matching scheme. The formula is:

[0104]

[0105] Where, σ * This represents the optimal matching scheme. This represents the index function, which takes the value 1 when the i-th slot has a matching real entity, and 0 otherwise. Let represent the confidence probability value corresponding to the true category output by slot i under the σ-th permutation, and argmin represent the function for finding the minimum value; This represents the confidence probability value corresponding to the true left and right boundaries output by slot i under the σ-th permutation.

[0106] S9: Calculate the total loss of the model under the optimal matching scheme; adjust the model parameters according to the total loss of the model to obtain the trained named entity recognition model.

[0107] Total model loss L total The weighted sum of localization loss and classification loss:

[0108] L total =αL type +βL position

[0109] Where α represents the classification weight, β represents the localization weight, and L type L represents the classification loss. position This indicates positioning loss.

[0110] The classification loss is expressed as:

[0111]

[0112] The positioning loss is expressed as:

[0113]

[0114] in, The permutation σ represents the optimal matching. * The confidence probability value corresponding to the true category output by the lower suggestion slot i. This represents the confidence probability value corresponding to the true left and right boundaries output by slot i under the optimal matching permutation.

[0115] Based on the total model loss, backpropagation updates the model parameters, and training is iterated continuously. Training stops when the loss function converges or reaches the maximum preset number of iterations, and the optimal model parameters are saved. The Adam optimizer is used to dynamically adjust the model's learning rate, and the BERT model parameters are frozen for the first 5 epochs to prevent catastrophic forgetting. The formula for the Adam algorithm is:

[0116]

[0117] in, This represents the corrected first-order moment estimate. Let represent the corrected second-order moment estimate, and ∈ and η be the parameters that need to be adjusted during training.

[0118] The Micro-F1 score evaluation model was used during testing. The Micro-F1 formula is as follows:

[0119]

[0120] Among them, TP i FP represents the true number of instances in the i-th class. i FN represents the number of false positives in the i-th class. i Let K represent the number of false negatives in the i-th category, and K be the total number of categories.

[0121] After the model is trained, the text to be recognized is obtained and preprocessed. The preprocessed text is then input into the trained named entity recognition model for processing. The model outputs the category and left and right boundaries corresponding to the maximum values ​​in the classification slot probability distribution and the localization slot probability distribution, thus obtaining the named entity recognition result.

[0122] In summary, this invention prepends a cue template before the sentence and obtains semantic representation through BERT encoding; it uses a self-attention mechanism to achieve interaction between cue slots and between cue slots and the sentence; it decodes entity boundaries by combining location information with classification information and textual semantics, and classifies entities based on the semantic similarity between classification information and natural label; based on the cue learning framework, this invention designs a single-slot multi-task cue template to extract multiple entities in parallel, solving the problem of high time overhead and computational cost in existing cue learning methods. Through the interaction layer and cross-task representation fusion, it achieves interaction between different cue slots and tasks, significantly improving the model's entity extraction capability.

[0123] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.< / soft> < / soft>

Claims

1. A named entity recognition method based on cue learning and label awareness, characterized in that, include: The text to be recognized is obtained and preprocessed. The preprocessed text is then input into the trained named entity recognition model for further processing to obtain the named entity recognition result. The training process of a named entity recognition model includes: S1: Obtain the named entity recognition dataset and preprocess it to obtain preprocessed named entity recognition data; S2: Concatenate multiple prompt templates before each sentence in the named entity recognition data, and unify the length of the sentence after concatenating the prompt templates; S3: After adding a cue mask to the sentence following the concatenated cue template, input it into the BERT model to obtain the cue slot semantic representation and the sentence semantic representation; S4: Input the entity labels into the BERT model to obtain the semantic representation of the labels; S5: Input the sentence semantic representation into a bidirectional LSTM network to obtain the sentence context semantic matrix; process the cue slot semantic representation using a self-attention mechanism based on the sentence context semantic matrix to obtain the cue slot deep semantic matrix; S6: The deep semantic matrix of the cue slot is mapped to the classification slot encoding matrix through a linear layer. The classification slot probability distribution is calculated based on the classification slot encoding matrix and the label semantic representation. The process of calculating the classification slot probability distribution includes: mapping the deep semantic matrix of the cue slot to the classification slot encoding matrix through a linear mapping; performing a dot product operation between the classification slot encoding matrix and the label semantic representation to obtain the matching degree matrix between the cue slot and the entity category; and decoding the matching degree matrix between the cue slot and the entity category through a linear layer to obtain the classification slot probability distribution. S7: Calculate the probability distribution of the location slots based on the deep semantic matrix of the cue slots and the semantic matrix of the sentence context; the process of calculating the probability distribution of the location slots includes: The deep semantic matrix of the cue slot is linearly mapped to obtain the localization slot encoding matrix; The first matrix is ​​obtained by adding the local slot encoding matrix to the sentence context semantic matrix after linear mapping. The second matrix is ​​obtained by adding the class slot encoding matrix to the sentence context semantic matrix after linear mapping; The first matrix is ​​added to the second matrix, and the location slot probability distribution is obtained by decoding through a linear layer again. S8: Calculate the minimum matching loss between the cue slot and the entity based on the probability distribution of the classification slot and the probability distribution of the positioning slot to obtain the optimal matching scheme; the process of calculating the matching loss between the cue slot and the entity includes: The sampling weight of each entity is calculated based on the probability distribution of the classification slot and the probability distribution of the location slot; Based on the sampling weight, the entity is repeatedly sampled to fill all the cue slots, and the matching schemes of all cue slots and entities are obtained. Calculate the minimum matching loss among all matching schemes between the prompt slot and the entity, and select the matching scheme with the minimum matching loss as the optimal matching scheme; S9: Calculate the total model loss under the optimal matching scheme, adjust the model parameters according to the total model loss, and obtain the trained named entity recognition model; the total model loss is the weighted sum of the localization loss and the classification loss, where: Classification loss Represented as: ； Location loss Represented as: ； in, Represents the permutation of the optimal matching The confidence probability value corresponding to the true category output by the lower suggestion slot i. , This represents the confidence probability values ​​corresponding to the true left and right boundaries output by slot i under the optimal matching permutation. This represents the indicator function, and M represents the number of prompt slots.

2. The named entity recognition method based on cue learning and label awareness according to claim 1, characterized in that, The semantic representation of the cue slot is processed using a self-attention mechanism as follows: ； ； ； ； ； in, This represents the original semantic representation of the cue slot. Represents the semantic matrix of sentence context. This represents the semantic representation of the prompt slot after cross-slot interaction. This indicates that the semantic representation of the cue slots takes into account both inter-slot interactions and interactions between slots and sentences, i.e., the deep semantic representation of cue slots. , and These represent the key, query, and value matrices for the suggestion slot, respectively. , and These represent the first, second, and third weight matrices, respectively. and A key-value matrix representing a sentence; Let h represent the slot identifier embedding matrix, and h represent the dimension of the hidden vector. This represents the activation function.

3. The named entity recognition method based on cue learning and label awareness according to claim 1, characterized in that, The formula for calculating the sampling weight of each entity is: ； ； in, This represents the total number of suggestion slots in the sentence that match the j-th real entity. The sampling weight corresponding to the j-th real entity in the sentence is represented by M, where M is the number of cue slots and A is the total number of entities in the original sentence. , , This indicates the matching category, left boundary, and right boundary values ​​obtained when slot k matches the j-th real entity. This indicates the index of the real entity that best matches slot k. This is a flag function.

4. The named entity recognition method based on cue learning and label awareness according to claim 1, characterized in that, The formula for calculating the minimum matching loss among all matching schemes between cue slots and entities is: ； in, This represents the optimal matching scheme, where M represents the number of hint slots. Indicates the index function, Indicates the first The confidence probability value corresponding to the true category output by slot i under the given permutation. , Indicates the first Let argmin represent the confidence probability values ​​corresponding to the true left and right boundaries output by slot i under the given permutation, and let argmin represent the minimum value function.

Images