A small sample entity recognition method and system of label enhancement and non-entity clustering

Abstract

The application discloses a kind of label enhancement and non-entity clustering small sample entity identification method and system, comprising: converting label into semantic word by label mapping, generate extended input;The extended input is encoded by BERT, and output sentence representation and class representation;Class representation is input to prompt pool, and class enhancement representation is dynamically generated;Construct positive sample pair and negative sample pair, optimize the similarity distribution of token enhancement representation and class enhancement representation by contrast loss;The semantic center of non-entity token is generated trainable, and the semantic center distribution of non-entity token is optimized by clustering loss;Based on the similarity of token enhancement representation and class enhancement representation, output entity identification result.Label semantic enhancement method is used to improve the accuracy of label understanding, and the semantic relevance of labels in different fields is enhanced;Through non-entity clustering optimization, the accuracy of non-entity classification is improved, and the false alarm rate of entity identification is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, in particular to a small sample entity recognition method and system based on label enhancement and non-entity clustering. BACKGROUND

[0002] Traditional information extraction methods mostly rely on manually constructed rules, templates or keyword dictionaries, etc. Such methods have certain practicality in processing fixed format, clear semantic text scenarios, but when facing unstructured, free description, and professional customs declaration texts, the recognition accuracy is significantly reduced. In recent years, with the development of natural language processing technology, the named entity recognition (NER) model based on deep learning is widely used in text structuring tasks, which can automatically extract specific types of entity information from complex texts, and shows strong generalization ability and semantic understanding ability. However, there are the following problems for small sample named entity recognition:

[0003] The existing prompt learning method guided by label name provides semantic guidance for the named entity recognition task to a large extent, but the label semantic information is limited, which limits the upper limit of named entity recognition, and it is difficult to distinguish cross-label categories; although the cross-domain scene is flexible, the model is difficult to adapt to new fields only by relying on label semantic changes, and the cross-domain entity recognition effect is reduced;

[0004] The existing named entity recognition method usually incorrectly divides entities into "entity" and "non-entity", and the semantic information in "non-entity" is chaotic, but the current named entity recognition method usually classifies "non-entity" as a category "O" and forcibly clusters, which interferes with normal entity recognition and requires additional processing of "non-entity". SUMMARY

[0005] The purpose of the present application is to provide a small sample entity recognition method and system based on label enhancement and non-entity clustering, which combines label semantic prompts and non-entity clustering to solve the problems of label semantic sparsity and non-entity noise interference in small sample NER proposed in the background technology.

[0006] To achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0007] According to one aspect of the present application, a small sample entity recognition method based on label enhancement and non-entity clustering is provided, which comprises:

[0008] Input the original sentence and the pre-defined entity label set, convert the label to a semantic word through label mapping, and generate an extended input;

[0009] encoding the extended input by BERT, outputting a sentence representation and a category representation;

[0010] inputting the category representation into a prompt pool for label semantic enhancement, and dynamically generating a category enhanced representation;

[0011] constructing a positive sample pair and a negative sample pair for contrastive learning, and optimizing a similarity distribution of the token enhanced representation and the category enhanced representation through a contrastive loss;

[0012] generating a trainable semantic center for a non-entity token, and optimizing a semantic center distribution of the non-entity token through a clustering loss;

[0013] outputting an entity recognition result based on the similarity of the token enhanced representation and the category enhanced representation.

[0014] According to the foregoing scheme, the label mapping includes mapping each entity category to a unique corresponding semantic word and mapping a non-entity category to 'none'.

[0015] According to the foregoing scheme, the label semantic enhancement includes initializing a prompt pool, the prompt pool including trainable keys and values, calculating a similarity weight of a category embedding and the keys, generating a prompt weighted and spliced with the category embedding, and outputting the category enhanced representation through BERT self-attention fusion and mean value taking.

[0016] According to the foregoing scheme, the similarity weight is:

[0017] ;

[0018] wherein, the similarity weight w i is composed of T components ;

[0019] ;

[0020] The component represents a normalized weight of the similarity of the category embedding c i and the jth key k j .

[0021] According to the foregoing scheme, the sentence representation is input into the last layer of BERT to obtain the token enhanced representation, a positive sample pair and a negative sample pair are constructed for the token enhanced representation, the contrastive loss is based on a temperature parameter to scale the distance, and a normalized similarity distribution is generated; the contrastive loss minimizes the distance of the positive sample pair and maximizes the distance of the negative sample pair.

[0022] According to the foregoing scheme, the contrastive loss is represented as:

[0023] ;

[0024] wherein, is the enhanced representation of the pth token in the sentence, τ is a temperature parameter, d is the Euclidean distance, is the positive sample pair, is the negative sample pair, is the class enhanced representation of the positive class, is the class enhanced representation of the negative class.

[0025] Based on the foregoing scheme, each of the non-entity tokens is assigned to B semantic centers {r j} by optimal transport theory, and the clustering loss is used to minimize the distance between the non-entity tokens and the corresponding semantic centers.

[0026] Based on the foregoing scheme, the clustering loss is:

[0027] ;

[0028] wherein, is the optimal transport matrix between the non-entity token distribution and the semantic center distribution calculated by the Sinkhorn algorithm; is the non-entity token, A is the number of non-entity tokens; r j is the semantic center, B is the number of semantic centers.

[0029] Based on the foregoing scheme, the total loss including the contrastive loss and the clustering loss is jointly optimized, represented as: ;L con is the contrastive learning loss, L ECR is the clustering loss, λ is the balance coefficient; gradient backpropagation is performed to update the BERT parameters, the key-value parameters of the prompt pool, and the semantic centers.

[0030] According to another aspect of the present application, a small sample entity recognition system with label enhancement and non-entity clustering is provided, which comprises: an input processing module, a BERT encoding module, a label semantic enhancement module, a contrastive learning module, a non-entity clustering module, and a prediction module.

[0031] The input processing module receives an original sentence and a set of predefined entity labels, performs label semantic mapping, and generates an expanded input.

[0032] The BERT encoding module encodes the expanded input using the first k layers of BERT to generate a sentence representation and a class representation, and encodes the sentence representation using the last layer of BERT to generate a token enhanced representation.

[0033] The prompt pool module initializes trainable keys and values in the prompt pool, calculates the similarity weight of the category embedding and the keys, generates the prompt after weighting, splices the category embedding, generates the extended prompt sequence, and inputs the BERT encoding module;

[0034] The contrast learning module constructs positive sample pairs and negative sample pairs, calculates the contrast loss, and optimizes the similarity distribution of the token enhanced representation and the category enhanced representation;

[0035] The non-entity clustering module manages the semantic center, and generates a clustering loss to optimize the semantic center distribution of the non-entity token.

[0036] The prediction module outputs the entity recognition result based on the similarity of the token enhanced representation and the category enhanced representation.

[0037] According to the above technical solutions, compared with the prior art, the present application has at least the following advantages and positive effects: through the label semantic enhancement method of the prompt pool, the label understanding accuracy is improved, and the semantic correlation of labels in different fields is enhanced; through non-entity clustering optimization, the subdivision accuracy of non-entities is improved, and the false positive rate of entity recognition is reduced.

[0038] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings incorporated in and forming a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application. It is apparent that the accompanying drawings in the following description are only some embodiments of the present application, and other drawings can be obtained from these drawings without creative labor for those skilled in the art. In the drawings:

[0040] Figure 1 A label enhancement and non-entity clustering small sample entity recognition method according to the present application is shown in the figure;

[0041] Figure 2 A label enhancement and non-entity clustering small sample entity recognition system according to the present application is shown in the figure. DETAILED DESCRIPTION

[0042] To more clearly illustrate the purpose, technical solutions, and advantages of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein. On the contrary, these embodiments are provided so that the present invention will be more comprehensive and complete, and fully convey the concept of the exemplary embodiments to those skilled in the art.

[0043] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a full understanding of embodiments of the invention. However, those skilled in the art will recognize that the technical solutions of the invention can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of the invention.

[0044] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0045] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0046] The present invention will now be described in detail with reference to specific embodiments:

[0047] Example 1, such as Figure 1 As shown, this embodiment provides a few-shot entity recognition method based on label enhancement and non-entity clustering. First, steps S1–S5 are performed using a few-shot training set to train the model, obtaining optimized BERT parameters, cue pool parameters, and semantic centers. Then, steps S1–S3 and S6 are performed during inference to perform entity recognition. The specific steps of this method are as follows:

[0048] S1: Input the original sentence and a predefined set of entity labels, convert the labels into semantic words through label mapping, and generate extended input.

[0049] The label mapping includes mapping each entity category to a unique corresponding semantic word, and mapping a non-entity category to 'none'.

[0050] In the embodiment, an original sentence X = {x1, x2, …, x m} is input, m is the number of tokens, a pre-defined entity label set E = {e1, e2, …, e N} is input, and N is the number of entity categories; a label mapping M is constructed, each entity label e i in the pre-defined label set E is converted into a unique corresponding semantic word, and the entity category name is used as EW, for example: M('location-GPE') = 'nation', M('location-park') = 'parks'; the non-entity label M(non-entity) = 'none'; it should be noted that each label can only correspond to one word and cannot be repeated to avoid ambiguity.

[0051] Further, an extended input X ext is generated, the mapped labels are arranged in sequence, and the sequence is spliced to the end of the original sentence to generate the extended input: ; and

[0052] ;

[0053] It should be noted that M(non-entity) = 'none'; the semantic sparsity of small samples is solved, the pre-training knowledge of BERT is activated by replacing the symbolic label with a natural language word, the model quickly understands new categories; the input format is unified, the sentence and the label share the same encoding space, and the semantic association between the token and the label is automatically learned through the self-attention mechanism.

[0054] S2: encoding the extended input by BERT to output the sentence representation and the category representation.

[0055] The first k layers (k = 11) of BERT are used to encode the extended input X ext , and the vector representation of the entire sequence is obtained as:

[0056] ;

[0057] The output H can be divided into a sentence representation , which corresponds to the vector representation of the original sentence token, and is used for subsequent entity recognition prediction; and a category representation , which corresponds to the vector representation of the label token p i , wherein c i encodes the label p​i semantic.

[0058] Specifically, the first 11 layers of BERT output contain rich semantic information, preserving semantic integrity; the class representation H c The input prompt pool is semantically enhanced, and the sentence representation H t The last layer of the input is refined.

[0059] S3: input the class representation into the prompt pool for label semantic enhancement, and dynamically generate a class enhanced representation.

[0060] Label semantic enhancement includes: initializing the prompt pool, the prompt pool including trainable keys and values, calculating the similarity weight of the class embedding and the key, generating the prompt weighted and splicing with the class embedding, and outputting the class enhanced representation by BERT self-attention fusion and taking the mean value.

[0061] Specifically, the prompt pool is a trainable key-value memory bank, the key is used to match the class semantics, and the value stores the transferable prompt knowledge. First, initialize the prompt pool:

[0062] ;

[0063] ;

[0064] Where T is the capacity of the prompt pool, L is the length of the value, d h is the dimension of the BERT hidden layer.

[0065] Further, the class representation H c is the query vector of the prompt pool, and the class enhanced representation is dynamically generated; for each class embedding c , the similarity weight with all keys k j is calculated:

[0066] ;

[0067] ;

[0068] Where, the weight w i is composed of T components, each component represents the normalized weight of the similarity between the class embedding c i and the jth key k j , that is, the importance of the jth prompt key k j to the class, reflecting the semantic relevance between the class embedding c i and the key k j ;

[0069] Further, the semantic enhanced prompt is obtained by weighted aggregation of the value vector V j :

[0070] ;

[0071] The T prompt vectors V in the prompt pool are prompted j Weighted fusion to generate semantic enhanced prompts v adapted to the current category c , ; semantic enhanced prompts v c The dynamic prompt vector changes in real time with the input category, and the same prompt pool generates different prompt combinations in real time when facing different categories or different contexts. It is generated in real time according to the category during forward calculation, and is adjusted in real time with the loss during back propagation.

[0072] Further, the original category embedding c i is concatenated with the L semantic enhanced prompts v generated by the prompt pool to form an extended prompt sequence of (L+1) vectors: c

[0073] ;

[0074] Further, the extended prompt sequence is input into the last layer of BERT for self-attention calculation and averaging to output:

[0075] ;

[0076] After one self-attention, each vector has integrated the information of each other, and the semantics are more abundant. Then, the output vectors are averaged to obtain the category enhanced embedding , which retains the original category signal and integrates the additional semantics provided by the prompt pool; further, the category enhanced representation is .

[0077] The label semantic enhancement method based on the prompt pool obtains semantic enhanced label names using the prompt pool, and the prompt pool can learn category-independent knowledge for cross-domain knowledge transfer.

[0078] S4: Construct positive and negative sample pairs for contrastive learning, and optimize the similarity distribution of token enhanced representation and the category enhanced representation through contrastive loss.

[0079] Input the sentence representation H t obtained in step S2 into the last layer of BERT to obtain the token enhanced representation:

[0080] ;

[0081] Construct positive and negative sample pairs for contrastive learning based on the token enhanced representation ; the positive sample pair is represented as ,​ It is the augmented representation of the p-th token in the sentence. The category augmentation representation for the positive category (the true category of the token); the negative sample pair is represented as... , A category-enhanced representation for negative categories (any category other than the true category).

[0082] Further, the contrast loss is calculated. The contrast loss can be expressed as:

[0083] ;

[0084] Where τ is the temperature parameter and d is the Euclidean distance; the token representation is calculated. With category representation The Euclidean distance d; based on temperature parameters The distance is scaled to generate a normalized similarity distribution; the negative logarithm of the true class probability is minimized; this loss function makes the similarity (negative distance) of positive sample pairs (token and correct class) much greater than the similarity of negative sample pairs, with the denominator being the sum of the similarities of all classes; the distance of positive sample pairs is minimized and the distance of negative sample pairs is maximized by contrastive loss.

[0085] Furthermore, the contrastive losses of the m tokens in the sentence are averaged to obtain the total contrastive learning loss L for the entire sentence. con :

[0086] ;

[0087] By minimizing L con The model learns to pull tokens of the same class to the vicinity of the corresponding class vector and push tokens of different classes apart, thereby forming a clear and separable entity recognition space.

[0088] It should be noted that this embodiment constructs positive and negative sample pairs for comparative learning. The similarity distribution between token representations and category label representations is optimized using a contrastive loss function, reducing the distance between token and category label representations of the same category and increasing the distance between representations of different categories. Contrastive learning is used to directly align the representation spaces of tokens and category labels, enabling the model to quickly adapt to new categories with small sample sizes. The embodiment also incorporates category representations enhanced with semantic labeling. This makes the category representation more discriminative; using category labels as category metrics for named entity recognition, the role of contrastive learning is mainly to bring the category labels closer to tokens of the same class and to widen the distance between them and tokens of different classes, thereby forming a classification space for entity recognition.

[0089] S5: Generating trainable semantic centers for non-entity tokens, and optimizing semantic center distribution of the non-entity tokens through clustering loss.

[0090] Since non-entity ("O" type) tokens usually have messy semantics and contain a large amount of background information irrelevant to entities, which interferes with entity boundary judgment, in this embodiment, a small number of "semantic centers" (trainable vectors) are automatically generated, and each non-entity token is assigned to the nearest semantic center using optimal transport. Through clustering loss, the token vector is pulled towards the center, and the center and token representation are updated through backpropagation, reducing non-entity noise while not excessively sacrificing entity recognition accuracy.

[0091] Specifically, the semantic center is defined as , B is the number of clustering centers, d h is the BERT hidden layer dimension, and the semantic center is a trainable parameter that is adaptively adjusted with gradient descent; embedding clustering regularization is used to force the O-type token X O to be transmitted to the semantic center. The core idea of embedding clustering regularization (ECR) is to strengthen the minimum transmission of embedding clustering characteristic distance. Specifically, the enhanced representation of all tokens is input , from which tokens with label "O" are filtered out, , A is the number of O-type samples; through optimal transport theory, non-entity tokens are assigned to B trainable semantic centers, and the distance between non-entity tokens and corresponding semantic centers is minimized.

[0092] Further, the clustering loss is calculated as:

[0093] ;

[0094] is the optimal transport matrix between token distribution and semantic center distribution calculated by Sinkhorn algorithm, and the clustering loss is the weighted distance sum of non-entity tokens to semantic centers under the optimal transport scheme, and the smaller the value represents the more compact the clustering; when backpropagation, the gradient flows to the semantic center r j and the non-entity token vector , the semantic center r j is pulled towards the weighted centroid of the corresponding token, and the non-entity token vector is pulled towards the nearest center, achieving clustering compactness.

[0095] Further, the total loss function is jointly optimized as:

[0096] ;

[0097] Total loss includes comparative loss L con Non-physical clustering loss L ECR λ is a balancing coefficient used to balance the two losses. Further, gradient backpropagation is performed to update the semantic center r. j To optimize non-entity clustering, BERT parameters are updated to optimize token representation, and key-value parameters of the hint pool are updated to optimize category representation;

[0098] Based on embedding clustering regularization, non-entities are decomposed and subdivided to form multiple clusters, thereby improving the accuracy of named entity recognition tasks by enhancing the recognition of non-entities.

[0099] S6: Output entity recognition results based on the similarity between the token-enhanced representation and the category-enhanced representation.

[0100] In this embodiment, the input during reasoning is the original sentence to be recognized, X={x1,x2,…,x...}. m}, and the predefined entity label set E={e1,e2,…,e N Construct extended input: This extended sequence is fed into the trained model, and after being encoded by the first 11 layers of BERT, a sentence representation H is generated. t and category representation H c Category-enhanced representations are generated after the cue pool. The token is enhanced by generating the final layer of BERT. ; Calculate the token augmentation representation for each token Enhanced representation for each category The similarity is used to determine the predicted label of the token, and the category with the highest similarity is selected. The predicted label sequence is then output in the order of the original sentence tokens.

[0101] This embodiment injects new category labels in real time by expanding the input, without the need for retraining; it has zero-sample transfer capability and can still predict on untrained categories.

[0102] Example 2, as Figure 2 As shown in the figure, this embodiment exemplarily presents a few-sample entity recognition system with label enhancement and non-entity clustering, including an input processing module, a BERT encoding module, a cue pool module, a contrastive learning module, a non-entity clustering module, and a prediction module;

[0103] The input processing module receives the original sentence and a predefined set of entity labels, performs label semantic mapping, and generates extended input.

[0104] BERT encoding module, using the first k layers (k = 11) of BERT to encode the extended input, generating sentence representation and category representation, using the last layer of BERT to encode the sentence representation, generating token enhanced representation (such as Figure 2 In SEQ ID NO: 1); using the last layer of BERT to perform self-attention calculation on the extended prompt sequence output by the prompt pool module and taking the average, outputting the category enhanced representation (such as Figure 2 In SEQ ID NO: 2);

[0105] Prompt pool module, initializing the trainable keys and values in the prompt pool, calculating the similarity weight between the category embedding and the key, weighting the prompt and concatenating with the category embedding, generating an extended prompt sequence, inputting the BERT encoding module;

[0106] Contrastive learning module, constructing positive sample pairs and negative sample pairs, calculating contrastive loss and optimizing the similarity distribution of token enhanced representation and category enhanced representation; Non-entity clustering module, managing semantic centers, generating clustering loss to optimize the semantic center distribution of non-entity tokens; Prediction module, calculating the similarity of token enhanced representation and category enhanced representation, outputting category prediction.

[0107] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice in the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims. It is understood that the application is not limited to the exact details shown and described herein, and that many variations and modifications can be made within the scope of the application. The scope of the application is limited only by the claims that follow.

Claims

1. A method of few-shot entity recognition with label augmentation and non-entity clustering, the method comprising: The method comprises: inputting an original sentence and a predefined entity label set, converting labels to semantic words through label mapping, generating an extended input; encoding the extended input through BERT, outputting a sentence representation and a category representation; inputting the category representation into a prompt pool for label semantic enhancement, dynamically generating a category enhanced representation; the label semantic enhancement comprises: initializing the prompt pool, the prompt pool comprising trainable keys and values, calculating the similarity weight of the category embedding and the keys, generating prompts by weighting and splicing with the category embedding, and outputting the category enhanced representation through BERT self-attention fusion and averaging; inputting the sentence representation into the last layer of BERT to obtain a token enhanced representation, constructing a positive sample pair and a negative sample pair for contrastive learning, and optimizing the similarity distribution of the token enhanced representation and the category enhanced representation through a contrastive loss; generating a trainable semantic center for non-entity tokens, and optimizing the semantic center distribution of the non-entity tokens through a clustering loss; outputting an entity recognition result based on the similarity of the token enhanced representation and the category enhanced representation.

2. The method of claim 1, wherein, The label mapping comprises mapping each entity category to a unique corresponding semantic word, and mapping a non-entity category to 'none'.

3. The method of claim 1, wherein, The similarity weight is: ； wherein the similarity weight w i consists of T components ; ； The components representing category embeddings c i similarity to the jth key k j normalized weight.

4. The method of claim 1, wherein, inputting the sentence representation into the last layer of BERT to obtain a token enhanced representation, constructing a positive sample pair and a negative sample pair for the token enhanced representation, the contrastive loss being based on a temperature parameter to scale the distance, and generating a normalized similarity distribution; minimizing the distance of the positive sample pair and maximizing the distance of the negative sample pair through the contrastive loss.

5. The method of claim 4, wherein, The contrastive loss is represented as: ； wherein, is an enhanced representation of the p-th token in the sentence, is a temperature parameter, d is the Euclidean distance, is the positive sample pair, is the negative sample pair, is a class-enhanced representation of the positive class, is a class-enhanced representation of the negative class.

6. The method of claim 1, wherein, Each of the non-entity tokens is assigned to one of B semantic centers {r j} by optimal transport theory, and the distance between the non-entity tokens and the corresponding semantic centers is minimized using a clustering loss.

7. The method of claim 6, wherein, The clustering loss is: ； wherein, is the optimal transport matrix between the non-entity token distribution and the semantic center distribution calculated by Sinkhorn algorithm; is the non-entity token, A is the number of non-entity tokens; r j is the semantic center, B is the number of semantic centers.

8. The method of claim 1, wherein, The method further comprises jointly optimizing a total loss comprising the contrast loss and the clustering loss, denoted as: ; L con is a contrast learning loss, L ECR is a clustering loss, and λ is a balance coefficient; gradient back propagation is performed to update the BERT parameters, the key-value parameters of the prompt pool, and the semantic center.

9. A system for few-shot entity recognition enhanced by label and non-entity clustering, the system comprising: comprising an input processing module, a BERT encoding module, a prompt pool module, a contrastive learning module, a non-entity clustering module, and a prediction module; The input processing module is configured to receive an original sentence and a predefined entity label set, perform label semantic mapping, and generate an extended input. The BERT encoding module is configured to encode the extended input using the first k layers of BERT to generate a sentence representation and a category representation, encode the sentence representation using the last layer of BERT to generate a token enhanced representation, and perform self-attention calculation on the extended prompt sequence output by the prompt pool module using the last layer of BERT and take the average to output a category enhanced representation. The prompt pool module initializes the trainable keys and values in the prompt pool, calculates the similarity weight of the category embedding and the keys, generates prompts by weighting and splices with the category embedding, generates the extended prompt sequence, and inputs the BERT encoding module. The contrastive learning module constructs a positive sample pair and a negative sample pair, calculates a contrastive loss, and optimizes the similarity distribution of the token enhanced representation and the category enhanced representation. The non-entity clustering module manages the semantic center, generates a clustering loss, and optimizes the semantic center distribution of the non-entity tokens. The prediction module outputs an entity recognition result based on the similarity of the token enhanced representation and the category enhanced representation.

Images