Named entity recognition method and device and storage medium

Abstract

The invention provides a named entity recognition method and device and a storage medium. According to the named entity recognition method and device provided by the embodiment of the invention, enhanced samples with similar features are constructed through data expansion, then a span-based comparative learning target is constructed by using a comparative learning algorithm, and vector representation of an entity and vector representation of a non-entity are mapped to the same vector space; and the span representation distances of the same category are as close as possible, and the span representation distances of different categories are as far as possible, so that the generalization ability of the recognition model is improved. In addition, according to the embodiment of the invention, the nested named entity identification can be supported by introducing spans with different lengths.

Description

Technical Field

[0001] This invention relates to the field of machine learning and natural language processing (NLP) technology, specifically to a named entity recognition (NER) method, apparatus, and storage medium. Background Technology

[0002] Named entity recognition (also known as entity identification, entity chunking, and entity extraction) is a fundamental task in Natural Language Processing (NLP), serving as a crucial tool for many NLP tasks such as information extraction, named entity recognition systems, syntactic analysis, and machine translation. Named entity recognition aims to locate and classify named entities in text into predefined entity types, such as person names (usually denoted by PER), organization names (usually denoted by ORG), place names (usually denoted by LOC), time expressions, quantities, currency values, percentages, etc. For ease of description, this article sometimes refers to "named entity" simply as "entity."

[0003] In recent years, NER has primarily been treated as sequence labeling or span classification, but these approaches have many limitations. For example, using sequence labeling to handle nested NERs is difficult, while learning and inference based on span classification are very complex and highly sensitive to noise in supervised data. Furthermore, existing techniques only learn semantic features and intermediate representations of specific classes from the source domain, which affects the ability to generalize to unseen target domains, resulting in suboptimal performance. Summary of the Invention

[0004] At least one embodiment of this application provides a named entity recognition method, apparatus, and storage medium to improve the recognition performance of named entities.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a named entity recognition method, including:

[0007] Determine at least one span in the sequence to be predicted, generate a vector representation of the sequence to be predicted using a pre-trained recognition model, and obtain a vector representation of the span in the sequence to be predicted, wherein the number of tokens in the span is not greater than a preset threshold.

[0008] The vector representation of the first span in the sequence to be predicted is compared with the final representation of various entity type identifiers obtained in advance to determine the entity type of the first span; the first span is any one of the at least one span.

[0009] The recognition model is obtained by fine-tuning a pre-trained language model based on a contrastive learning objective function. The objective function is a function constructed based on the vector representation of spans in the training samples, so that the distance between the vector representations of spans of the same category is close, and the distance between the vector representations of spans of different categories is far. The training samples include multiple original samples and multiple augmented samples. The original samples are samples with labeled entity types, and the augmented samples are obtained by replacing the entities in the original samples with the corresponding entity type identifiers. The final representation of each entity type identifier is obtained by average pooling the vector representations of the same entity type identifier in the vector representations of the multiple augmented samples generated by the recognition model.

[0010] Optionally, the span of the same category includes: the span corresponding to the first entity in the first original sample and the span corresponding to the first entity type identifier in the first enhanced sample, wherein the first original sample is any original sample among the plurality of original samples, the first entity is any entity in the first original sample; the first enhanced sample is the enhanced sample corresponding to the first original sample, and the first entity type identifier is the entity type identifier that replaces the first entity in the first enhanced sample;

[0011] The spans of the different categories include: the span corresponding to the first entity in the first original sample and the span corresponding to any non-entity in any training sample; the span corresponding to the first entity type identifier in the first augmented sample and the span corresponding to any non-entity in any training sample.

[0012] Optionally, the vector representation of the span is generated based on the vector representation of the starting token of the span, the vector representation of the ending token of the span, and the vector representation of the length of the span.

[0013] Optionally, the vector representation of the first span in the sequence to be predicted is compared with the final representation of various pre-obtained entity type identifiers to determine the entity type of the first span, including:

[0014] Calculate the similarity between the vector representation of the first span and the final representation of various entity type identifiers, and determine the entity type of the first span based on the entity type identifier with the highest similarity.

[0015] Optionally, the entity type of the first span is determined based on the entity type identifier with the highest similarity, including:

[0016] When the highest similarity is lower than the reference similarity, the first span is determined to be a non-entity;

[0017] When the highest similarity is not lower than the reference similarity, the entity type of the first span is determined to be the entity type corresponding to the entity type identifier of the highest similarity.

[0018] The reference similarity is the similarity between the vector representation of the first span and the vector representation of the preset token.

[0019] Optionally, the above methods also include:

[0020] The recognition model is obtained through pre-training, wherein the training of the recognition model specifically includes:

[0021] Obtain the training samples;

[0022] Determine the span in the training samples, generate a vector representation of the training samples using a pre-trained language model, and obtain the vector representation of the span in the training samples;

[0023] Based on the vector representation of the span in the training samples, the objective function of contrastive learning is constructed, and the pre-trained language model is fine-tuned based on the objective function to obtain the recognition model;

[0024] The recognition model is used to generate vector representations of the multiple enhanced samples. Average pooling is then performed on the vector representations of the same entity type identifiers in the multiple enhanced samples to generate the final representation of the corresponding entity type identifier.

[0025] Secondly, embodiments of this application provide a named entity recognition device, comprising:

[0026] The calling module is used to determine at least one span in the sequence to be predicted, generate a vector representation of the sequence to be predicted using a pre-trained recognition model, and obtain a vector representation of the span in the sequence to be predicted, wherein the number of tokens in the span is not greater than a preset threshold.

[0027] The comparison module is used to compare the vector representation of the first span in the sequence to be predicted with the final representation of various entity type identifiers obtained in advance to determine the entity type of the first span; the first span is any one of the at least one span;

[0028] The recognition model is obtained by fine-tuning a pre-trained language model based on a contrastive learning objective function. The objective function is a function constructed based on the vector representation of spans in the training samples, so that the distance between the vector representations of spans of the same category is close, and the distance between the vector representations of spans of different categories is far. The training samples include multiple original samples and multiple augmented samples. The original samples are samples with labeled entity types, and the augmented samples are obtained by replacing the entities in the original samples with the corresponding entity type identifiers. The final representation of each entity type identifier is obtained by average pooling the vector representations of the same entity type identifier in the vector representations of the multiple augmented samples generated by the recognition model.

[0029] Optionally, the span of the same category includes: the span corresponding to the first entity in the first original sample and the span corresponding to the first entity type identifier in the first enhanced sample, wherein the first original sample is any original sample among the plurality of original samples, the first entity is any entity in the first original sample; the first enhanced sample is the enhanced sample corresponding to the first original sample, and the first entity type identifier is the entity type identifier that replaces the first entity in the first enhanced sample;

[0030] The spans of the different categories include: the span corresponding to the first entity in the first original sample and the span corresponding to any non-entity in any training sample; the span corresponding to the first entity type identifier in the first augmented sample and the span corresponding to any non-entity in any training sample.

[0031] Optionally, the vector representation of the span is generated based on the vector representation of the starting token of the span, the vector representation of the ending token of the span, and the vector representation of the length of the span.

[0032] Optionally, the comparison module is further configured to calculate the similarity between the vector representation of the first span and the final representation of various entity type identifiers, and determine the entity type of the first span based on the entity type identifier with the highest similarity.

[0033] Optionally, the comparison module is further configured to determine that the first span is a non-entity when the highest similarity is lower than the reference similarity; and to determine that the entity type of the first span is the entity type corresponding to the entity type identifier of the highest similarity when the highest similarity is not lower than the reference similarity.

[0034] The reference similarity is the similarity between the vector representation of the first span and the vector representation of the preset token.

[0035] Optionally, the above apparatus further includes: a training module for pre-training the recognition model, wherein the training of the recognition model specifically includes:

[0036] Obtain the training samples;

[0037] Determine the span in the training samples, generate a vector representation of the training samples using a pre-trained language model, and obtain the vector representation of the span in the training samples;

[0038] Based on the vector representation of the span in the training samples, the objective function of contrastive learning is constructed, and the pre-trained language model is fine-tuned based on the objective function to obtain the recognition model;

[0039] The recognition model is used to generate vector representations of the multiple enhanced samples. Average pooling is then performed on the vector representations of the same entity type identifiers in the multiple enhanced samples to generate the final representation of the corresponding entity type identifier.

[0040] According to a third aspect of this application, at least one embodiment provides a named entity recognition device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method as described in any of the first aspects.

[0041] According to a fourth aspect of this application, at least one embodiment provides a computer-readable storage medium storing a program that, when executed by a processor, implements the steps of the method as described in any of the first aspects.

[0042] According to a fifth aspect of this application, at least one embodiment provides a computer program product including computer instructions that, when executed by a processor, implement the steps of the method as described in any of the first aspects.

[0043] Compared with existing technologies, the named entity recognition method and apparatus provided in this application construct enhanced samples with similar features through data augmentation, and then use a contrastive learning algorithm to construct a span-based comparative learning objective. This maps the vector representations of entities and non-entity entities to the same vector space, and makes the span representations of the same category as close as possible in distance, and the span representations of different categories as far apart as possible, which helps to improve the generalization ability of the recognition model. In addition, this application also supports nested named entity recognition by introducing spans of different lengths. Attached Figure Description

[0044] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0045] Figure 1 This is a flowchart of a named entity recognition method according to an embodiment of this application;

[0046] Figure 2 This is a schematic diagram of the training process of the recognition model in an embodiment of this application;

[0047] Figure 3 This is a schematic diagram of the structure of the named entity recognition device according to an embodiment of this application;

[0048] Figure 4 This is another structural schematic diagram of the named entity recognition device according to an embodiment of this application;

[0049] Figure 5 This is a schematic diagram of a named entity recognition system according to an embodiment of the present invention;

[0050] Figure 6 The computer described in this application is an exemplary hardware structure.

[0051] Figure 7 This is an exemplary hardware structure of a mobile terminal according to an embodiment of this application;

[0052] Figure 8 This is an example diagram illustrating the workflow of the named entity recognition system according to an embodiment of this application. Detailed Implementation

[0053] To make the technical problems, technical solutions, and advantages of this application clearer, a detailed description will be provided below in conjunction with the accompanying drawings and specific embodiments. In the following description, specific details such as particular configurations and components are provided merely to aid in a comprehensive understanding of the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Furthermore, for clarity and brevity, descriptions of known functions and structures have been omitted.

[0054] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The word "and / or" in the specification and claims indicates at least one of the connected objects.

[0055] In the various embodiments of this application, it should be understood that the sequence number of each process described below does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0056] The following description provides examples and is not intended to limit the scope, applicability, or configuration set forth in the claims. Changes may be made to the function and arrangement of the elements discussed without departing from the spirit and scope of this disclosure. Various procedures or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with reference to certain examples may be combined in other examples.

[0057] In this embodiment, a recognition model for named entity recognition is trained in advance using training samples. The recognition model is then used to recognize named entities in the sequence to be predicted, thereby improving the recognition performance of named entities.

[0058] In this embodiment, the training samples include multiple original samples and multiple augmented samples. The original samples are samples with labeled entity types, and the augmented samples are obtained by replacing the entities in the original samples with the corresponding entity type identifiers. After training the recognition model, final representations of various entity type identifiers are generated based on the recognition model. The final representation of each entity type identifier is obtained by average pooling the vector representations of the same entity type identifier in the vector representations of the multiple augmented samples generated by the recognition model.

[0059] The recognition model is obtained by fine-tuning a pre-trained language model based on a contrastive learning objective function. The pre-trained language model includes, but is not limited to, any of the following models: BERT model, RoBERTa model, Ernie model, etc. The objective function is a function constructed based on the vector representations of spans in the training samples, such that the distances between the vector representations of spans of the same category are close, while the distances between the vector representations of spans of different categories are far apart.

[0060] The span of the same category includes: the span corresponding to the first entity in the first original sample and the span corresponding to the first entity type identifier in the first enhanced sample, wherein the first original sample is any original sample among the multiple original samples, the first entity is any entity in the first original sample; the first enhanced sample is the enhanced sample corresponding to the first original sample, and the first entity type identifier is the entity type identifier that replaces the first entity in the first enhanced sample.

[0061] The spans of the different categories include: the span corresponding to the first entity in the first original sample and the span corresponding to any non-entity in any training sample; the span corresponding to the first entity type identifier in the first augmented sample and the span corresponding to any non-entity in any training sample. Non-entities refer to unnamed entities. Samples may typically contain both named and unnamed entities.

[0062] like Figure 1 As shown, the named entity recognition method provided in this application includes:

[0063] Step S11: Determine at least one span in the sequence to be predicted, generate a vector representation of the sequence to be predicted using a pre-trained recognition model, and obtain a vector representation of the span in the sequence to be predicted, wherein the number of tokens in the span is not greater than a preset threshold.

[0064] Here, the sequence to be predicted can be a sentence or a text segment comprising multiple sentences. The sequence to be predicted includes multiple tokens. In this embodiment, the token refers to the granularity of text processing by the pre-trained language model, specifically a single Chinese character, a word or subword in English, etc. The span in the sequence to be predicted is a token sequence composed of at least one consecutive token in the sequence, and the number of tokens it contains is no greater than a preset threshold L. The threshold L can be set based on the number of tokens contained in the longest named entity; for example, when the longest named entity contains 6 tokens, the threshold L can be set to 6.

[0065] When determining at least one span in the sequence to be predicted, it can be done by enumerating a continuous sequence of tokens in the sequence whose length is not greater than the threshold, thereby obtaining at least one span. For example, L = 6, and the sequence to be predicted is (z1, z2, ..., z...). n ), where zi represents the i-th token in the sequence to be predicted. By enumeration, the following span can be obtained:

[0066] Spans of length 1 include: z1; z2; ...; z n ;

[0067] A span of length 2 includes: z1z2; z2z3; ...; z n-1 z n ;

[0068] …

[0069] A span of length 6 includes: z1z2z3z4z5z6; z2z3z4z5z6z7; ...; z n-5 z n-4 z n-3 z n-2 z n-1 z n .

[0070] In step S11 above, the sequence to be predicted is input into the recognition model to obtain a vector representation of the sequence to be predicted generated by the recognition model. The vector representation of the sequence to be predicted includes the vector representation of each token in the sequence. In this embodiment, during the training process of the recognition model, the vector representation of each token in the training samples is generated by the pre-trained language model. Then, based on the vector representation of the sequence to be predicted, the vector representation of the span in the sequence is obtained.

[0071] In this embodiment, the vector representation of the span is generated based on the vector representations of the starting token, the ending token, and the length of the span. For example, the vector representation of the span can be obtained by concatenating the vector representations of the starting token, the ending token, and the length of the span, or by performing other vector calculations. The vector representation of the span length can be obtained by querying a pre-trained length representation matrix. Various lengths have corresponding vector representations, and this embodiment does not specifically limit this.

[0072] Step S12: Compare the vector representation of the first span in the sequence to be predicted with the final representation of various entity type identifiers obtained in advance to determine the entity type of the first span; the first span is any of the at least one span.

[0073] Here, the similarity between the vector representation of the first span and the final representation of various pre-obtained entity type identifiers is calculated, and the entity type of the first span is determined based on the entity type identifier with the highest similarity.

[0074] Through the above steps, enhanced samples with similar features are constructed through data augmentation. Then, a comparative learning algorithm is used to construct a span-based comparative learning objective, mapping the vector representations of entities and non-entity entities to the same vector space. This ensures that span representations of the same category are as close as possible in distance, while span representations of different categories are as far apart as possible, which helps improve the generalization ability of the recognition model. This application also supports nested named entity recognition by introducing spans of different lengths.

[0075] Considering that the embodiments of this application pre-generate the final representations of various entity type identifiers and do not generate vector representations of non-entity entities, in order to further improve the accuracy of named entity recognition, in step S12 above, when determining the entity type of the first span based on the entity type identifier with the highest similarity, the embodiments of this application can compare the highest similarity with a reference similarity. Specifically, if the highest similarity is lower than the reference similarity, the first span is determined to be a non-entity; and if the highest similarity is not lower than the reference similarity, the entity type of the first span is determined to be the entity type corresponding to the entity type identifier with the highest similarity. Here, the reference similarity is the similarity between the vector representation of the first span and the vector representation of a preset token. The preset token can be a token added by the pre-trained language model to the sequence to be predicted, such as the start identifier ([CLS]) or end identifier ([SEP]) of the sequence to be predicted. The vector representation of the preset token is generated by the recognition model, and the vector representation of the preset token may be different in different text sequences. Therefore, the reference similarity calculated based on the vector representation of the preset token is not fixed and is actually a dynamic threshold. Based on the above-mentioned reference similarity, the embodiments of this application can more accurately identify entities or non-entities in the sequence to be predicted.

[0076] Prior to step S11 above, a recognition model can be pre-trained in this embodiment of the application. The training of the recognition model in this embodiment of the application will be described below.

[0077] like Figure 2 As shown, the training of the recognition model includes the following steps:

[0078] Step 21: Obtain training samples.

[0079] Here, the training samples include multiple original samples and multiple augmented samples. The original samples are samples with labeled entity types, and the augmented samples are obtained by replacing the entities in the original samples with the corresponding entity type identifiers.

[0080] Specifically, identifiers for various entity types (i.e., entity type identifiers) can be predefined. For example, the entity type identifier for a person's name is: <per>The entity type identifier for the entity type of a place name is: <loc>The entity type identifier for the entity type of the organization name is: <org>Then, each entity in the original sample is replaced with its corresponding entity type identifier, thus generating an enhanced sample. For example, the original sample is "Xiaoming watched a game of the Chinese men's basketball team at Peking University's Yan Garden.", where the entity type of "Xiaoming" is "PER", the entity types of "Peking University" and "Chinese men's basketball team" are "ORG", and the entity type of "Yan Garden" is "LOC". After replacing the entities in the original sample with their corresponding entity type identifiers, the resulting enhanced sample is... <per>exist <org>of <loc>I watched <org>"A match." It can be seen that the enhanced sample has a similar feature distribution to the original sample. Through step 21, the embodiment of this application constructs an enhanced sample with similar features to the original sample.

[0081] Step 22: Determine the span in the training sample, generate a vector representation of the training sample using a pre-trained language model, and obtain the vector representation of the span in the training sample.

[0082] Here, the span is a sequence of tokens in the training sample whose length is no greater than a preset threshold. The threshold can be set based on the number of tokens contained in the longest named entity. In this embodiment, a continuous sequence of tokens in the training sample whose length is no greater than the threshold can be enumerated to obtain at least one span.

[0083] This application embodiment utilizes a pre-trained language model to generate the vector representation of the training samples. That is, the training samples are input into the pre-trained language model, and the pre-trained language model encodes the training samples to obtain the vector representation of the training samples (including the vector representation of each token). The pre-trained language model includes, but is not limited to, any of the following models: BERT model, RoBERTa model, Ernie model, etc.

[0084] Then, based on the vector representation of the training samples, the vector representation of each span in the training samples is obtained. Specifically, the method for generating the vector representation of a certain span in the training samples is the same as the method for generating the vector representation of the span in the sequence to be predicted mentioned above. For example, it can be generated based on the vector representation of the starting token of the span, the vector representation of the ending token of the span, and the vector representation of the length of the span. As one implementation, the vector representation of the starting token of the span, the vector representation of the ending token of the span, and the vector representation of the length of the span can be concatenated or other vector calculations can be performed to obtain the vector representation of the span. The vector representation of the length of the span can be obtained by querying a pre-trained length representation matrix. Various lengths have corresponding vector representations, and this application embodiment does not specifically limit this.

[0085] By using vector concatenation, the vector representation of the span can be expressed by the following formula:

[0086] A certain span S ij The representation of is Where h i h is the vector representation of the starting token of the span. j w is the vector representation of the end token of the span. l Let w be the vector representation of the span width l. l It can be obtained by querying the index using a pre-trained width representation matrix.

[0087] The vector representation of the span corresponding to the entity type identifier in the enhanced sample is also generated in the above manner.

[0088] Step 23: Based on the vector representation of the span in the training samples, construct the objective function for contrastive learning, and fine-tune the pre-trained language model based on the objective function to obtain the recognition model.

[0089] Here, by constructing a span-based comparative learning objective, the vector representations of entities and non-entity entities are mapped to the same vector space. During the fine-tuning training of the pre-trained language model, the model parameters are adjusted to make the span representations of the same category as close as possible and the span representations of different categories as far apart as possible, thereby training the recognition model.

[0090] The span of the same category includes: the span corresponding to the first entity in the first original sample and the span corresponding to the first entity type identifier in the first enhanced sample, wherein the first original sample is any original sample among the plurality of original samples, the first entity is any entity in the first original sample; the first enhanced sample is the enhanced sample corresponding to the first original sample, and the first entity type identifier is the entity type identifier that replaces the first entity in the first enhanced sample. For example, suppose the first original sample is "Xiaoming watched a game of the Chinese men's basketball team at Yanyuan, Peking University.", and the first enhanced sample is " <per>exist <org>of <loc>I watched <org>A match. Then, the "Peking University" in the first original sample and the first [sample name] in the first augmented sample... <org>For the same category span, similarly, "Peking University" in the first original sample and the second in the first augmented sample <org>It is also the span of the same category.

[0091] The spans of the different categories include: the span corresponding to the first entity in the first original sample and the span corresponding to any non-entity in any training sample; the span corresponding to the first entity type identifier in the first augmented sample and the span corresponding to any non-entity in any training sample. Non-entities refer to unnamed entities. Samples may typically contain both named and unnamed entities.

[0092] For example, constructing a contrastive loss based on the infoNEC loss function. span Comparison of loss l span The calculation formula is: Where S i，j S represents the vector representation of the span corresponding to an entity of a certain entity type in the original sample. label S represents the vector representation of the entity type identifier mapped to the entity type in the enhanced sequence. no It is a set of vector representations of the spans of entities of other entity types in the original sample and the augmented sample, and vector representations of the spans of all non-entity entities.

[0093] Based on the constructed loss function, the pre-trained language model can be fine-tuned using the AdamW optimizer to obtain a new encoding model, i.e., the recognition model. Thus, through contrastive learning, this embodiment encourages the representation of entity types to be similar to the corresponding entity span, but different from other text spans, thereby improving the generalization ability of the trained recognition model.

[0094] Step 24: Use the recognition model to generate vector representations of the multiple enhanced samples, and perform average pooling on the vector representations of the same entity type identifiers in the multiple enhanced samples to generate the final representation of the corresponding entity type identifier.

[0095] After training the recognition model, final representations of various entity type identifiers are generated based on the recognition model. The final representation of each entity type identifier is obtained by average pooling the vector representations of the same entity type identifier from the vector representations of the multiple augmented samples generated by the recognition model. This can be expressed by the formula: S type =AVGPooling(S1,S2,...,S m ). Among them, S type The final representation of a certain entity type, S1~S m These are the vector representations of the same entity type in the multiple augmented samples, and AVGPooling represents the vector representation of S1 to S2. m Perform average pooling calculation.

[0096] Through the above steps, the recognition model can be trained, and the final vector representation of various entity types can be obtained.

[0097] Based on the above methods, this application also provides an apparatus for implementing the above methods. Please refer to [link / reference]. Figure 3 This application provides a named entity recognition device, including:

[0098] Module 31 is invoked to determine at least one span in the sequence to be predicted, generate a vector representation of the sequence to be predicted using a pre-trained recognition model, and obtain a vector representation of the span in the sequence to be predicted, wherein the number of tokens in the span is not greater than a preset threshold.

[0099] The comparison module 32 is used to compare the vector representation of the first span in the sequence to be predicted with the final representation of various entity type identifiers obtained in advance to determine the entity type of the first span; the first span is any one of the at least one span;

[0100] The recognition model is obtained by fine-tuning a pre-trained language model based on a contrastive learning objective function. The objective function is a function constructed based on the vector representation of spans in the training samples, so that the distance between the vector representations of spans of the same category is close, and the distance between the vector representations of spans of different categories is far. The training samples include multiple original samples and multiple augmented samples. The original samples are samples with labeled entity types, and the augmented samples are obtained by replacing the entities in the original samples with the corresponding entity type identifiers. The final representation of each entity type identifier is obtained by average pooling the vector representations of the same entity type identifier in the vector representations of the multiple augmented samples generated by the recognition model.

[0101] Through the above modules, the embodiments of this application can improve the recognition performance of named entities.

[0102] In this embodiment of the application, the span of the same category includes: the span corresponding to the first entity in the first original sample and the span corresponding to the first entity type identifier in the first enhanced sample, wherein the first original sample is any original sample among the plurality of original samples, the first entity is any entity in the first original sample; the first enhanced sample is the enhanced sample corresponding to the first original sample, and the first entity type identifier is the entity type identifier that replaces the first entity in the first enhanced sample;

[0103] The spans of the different categories include: the span corresponding to the first entity in the first original sample and the span corresponding to any non-entity in any training sample; the span corresponding to the first entity type identifier in the first augmented sample and the span corresponding to any non-entity in any training sample.

[0104] The vector representation of the span is generated based on the vector representation of the starting token of the span, the vector representation of the ending token of the span, and the vector representation of the length of the span.

[0105] Optionally, the comparison module 32 is further configured to calculate the similarity between the vector representation of the first span and the final representation of various entity type identifiers, and determine the entity type of the first span based on the entity type identifier with the highest similarity.

[0106] Optionally, the comparison module 32 is further configured to determine that the first span is a non-entity when the highest similarity is lower than the reference similarity; and to determine that the entity type of the first span is the entity type corresponding to the entity type identifier of the highest similarity when the highest similarity is not lower than the reference similarity.

[0107] The reference similarity is the similarity between the vector representation of the first span and the vector representation of the preset token.

[0108] Optionally, the apparatus described in this application embodiment further includes: a training module for pre-training the recognition model, wherein the training of the recognition model specifically includes:

[0109] Obtain the training samples;

[0110] Determine the span in the training samples, generate a vector representation of the training samples using a pre-trained language model, and obtain the vector representation of the span in the training samples;

[0111] Based on the vector representation of the span in the training samples, the objective function of contrastive learning is constructed, and the pre-trained language model is fine-tuned based on the objective function to obtain the recognition model;

[0112] The recognition model is used to generate vector representations of the multiple enhanced samples. Average pooling is then performed on the vector representations of the same entity type identifiers in the multiple enhanced samples to generate the final representation of the corresponding entity type identifier.

[0113] It should be noted that the various devices provided in the above embodiments are devices corresponding to the above-described named entity recognition method. The implementation methods in each of the above embodiments are applicable to the embodiments of this device and can achieve the same technical effect. The above-described device provided in this application embodiment can implement all the method steps implemented in the above method embodiments and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.

[0114] Please refer to Figure 4 This application also provides a hardware structure block diagram of a named entity recognition device, such as... Figure 4 As shown, the named entity recognition device 400 includes:

[0115] Processor 402; and

[0116] Memory 404, in which computer program instructions are stored.

[0117] When the computer program instructions are executed by the processor, the processor 402 performs the following steps:

[0118] Determine at least one span in the sequence to be predicted, generate a vector representation of the sequence to be predicted using a pre-trained recognition model, and obtain a vector representation of the span in the sequence to be predicted, wherein the number of tokens in the span is not greater than a preset threshold.

[0119] The vector representation of the first span in the sequence to be predicted is compared with the final representation of various entity type identifiers obtained in advance to determine the entity type of the first span; the first span is any one of the at least one span.

[0120] The recognition model is obtained by fine-tuning a pre-trained language model based on a contrastive learning objective function. The objective function is a function constructed based on the vector representation of spans in the training samples, so that the distance between the vector representations of spans of the same category is close, and the distance between the vector representations of spans of different categories is far. The training samples include multiple original samples and multiple augmented samples. The original samples are samples with labeled entity types, and the augmented samples are obtained by replacing the entities in the original samples with the corresponding entity type identifiers. The final representation of each entity type identifier is obtained by average pooling the vector representations of the same entity type identifier in the vector representations of the multiple augmented samples generated by the recognition model.

[0121] It should be noted that the systems provided in the above embodiments are devices corresponding to the named entity recognition method described above. The implementation methods in each of the above embodiments are applicable to the embodiments of this device and can achieve the same technical effect. The device provided in this application embodiment can implement all the method steps implemented in the above method embodiments and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.

[0122] Furthermore, such as Figure 4 As shown, the named entity recognition device 400 also includes a network interface 401, an input device 403, a hard disk 405, and a display device 406.

[0123] The various interfaces and devices described above can be interconnected via a bus architecture. The bus architecture can include any number of interconnecting buses and bridges. Specifically, various circuits representing one or more central processing units (CPUs) and / or graphics processing units (GPUs), as represented by processor 402, and one or more memories, as represented by memory 404, are connected together. The bus architecture can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. It is understood that the bus architecture is used to implement communication between these components. In addition to the data bus, the bus architecture also includes a power bus, a control bus, and a status signal bus, which are well known in the art and will not be described in detail herein.

[0124] The network interface 401 can be connected to a network (such as the Internet, local area network, etc.), receive data such as raw training samples from the network, and save the received data to the hard disk 405.

[0125] The input device 403 can receive various instructions input by the operator and send them to the processor 402 for execution. The input device 403 may include a keyboard or a clicking device (e.g., a mouse, trackball, touchpad, or touchscreen).

[0126] The display device 406 can display the results obtained by the processor 402 executing instructions, such as displaying the model training progress.

[0127] The memory 404 is used to store programs and data necessary for the operation of the operating system, as well as intermediate results and other data during the calculation process of the processor 402.

[0128] It is understood that the memory 404 in the embodiments of this application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. The memory 404 of the apparatus and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0129] In some implementations, memory 404 stores elements such as executable modules or data structures, or subsets thereof, or extended sets thereof: operating system 4041 and application program 4042.

[0130] The operating system 4041 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 4042 includes various applications, such as a browser, used to implement various application functions. Programs implementing the methods of this application embodiment can be included in application program 4042.

[0131] The methods disclosed in the above embodiments of this application can be applied to processor 402, or implemented by processor 402. Processor 402 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 402 or by instructions in the form of software. The processor 402 may be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 404, and processor 402 reads the information in memory 404 and completes the steps of the above method in combination with its hardware.

[0132] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0133] For software implementation, the techniques described herein can be achieved through modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented within the processor or externally.

[0134] The span of the same category includes: the span corresponding to the first entity in the first original sample and the span corresponding to the first entity type identifier in the first enhanced sample, wherein the first original sample is any original sample among the multiple original samples, the first entity is any entity in the first original sample; the first enhanced sample is the enhanced sample corresponding to the first original sample, and the first entity type identifier is the entity type identifier that replaces the first entity in the first enhanced sample;

[0135] The spans of the different categories include: the span corresponding to the first entity in the first original sample and the span corresponding to any non-entity in any training sample; the span corresponding to the first entity type identifier in the first augmented sample and the span corresponding to any non-entity in any training sample.

[0136] The vector representation of the span is generated based on the vector representation of the starting token of the span, the vector representation of the ending token of the span, and the vector representation of the length of the span.

[0137] Specifically, when the computer program is executed by the processor 402, it can also perform the following steps:

[0138] Calculate the similarity between the vector representation of the first span and the final representation of various entity type identifiers, and determine the entity type of the first span based on the entity type identifier with the highest similarity.

[0139] Specifically, when the computer program is executed by the processor 402, it can also perform the following steps:

[0140] When the highest similarity is lower than the reference similarity, the first span is determined to be a non-entity;

[0141] When the highest similarity is not lower than the reference similarity, the entity type of the first span is determined to be the entity type corresponding to the entity type identifier of the highest similarity.

[0142] The reference similarity is the similarity between the vector representation of the first span and the vector representation of the preset token.

[0143] Specifically, when the computer program is executed by the processor 402, it can also perform the following steps:

[0144] The recognition model is pre-trained, wherein the training of the recognition model specifically includes:

[0145] Obtain the training samples;

[0146] Determine the span in the training samples, generate a vector representation of the training samples using a pre-trained language model, and obtain the vector representation of the span in the training samples;

[0147] Based on the vector representation of the span in the training samples, the objective function of contrastive learning is constructed, and the pre-trained language model is fine-tuned based on the objective function to obtain the recognition model;

[0148] The recognition model is used to generate vector representations of the multiple enhanced samples. Average pooling is then performed on the vector representations of the same entity type identifiers in the multiple enhanced samples to generate the final representation of the corresponding entity type identifier.

[0149] It should be noted that the device provided in this application embodiment can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0150] Figure 5 A schematic diagram of a named entity recognition system 800 provided in an embodiment of the present invention includes: a client 801 and a named entity recognition device 802.

[0151] The client 801 can be a personal computer or a mobile terminal, or an application running on any of the aforementioned terminals. A mobile terminal is a user-operated terminal device. A mobile terminal can be a smartphone, a personal digital assistant (PDA) device, a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, an in-vehicle device, a wearable device, or a terminal device in a next-generation communication system, such as a terminal device in an NR network or a terminal device in a future evolved Public Land Mobile Network (PLMN) network.

[0152] The named entity recognition device 802 can be a server system composed of one or more computers. The named entity recognition device 802 runs a recognition model 803, a calling module 804, a training module 806, and a comparison module 805. The recognition model 803 can be trained by the training module 806, which can be obtained by fine-tuning a pre-trained language model.

[0153] In the aforementioned named entity recognition system 800, the client 801 can connect to the named entity recognition device 802 via a wired and / or wireless network.

[0154] The functions of the client 801 and the named entity recognition device 802 can both be distributed across multiple computers.

[0155] The following describes the hardware or software structure of the relevant equipment, devices, or functions.

[0156] <Hardware Structure>

[0157] computer:

[0158] The client 801 and the named entity recognition device 802 communicate via, for example, Figure 6 The hardware structure shown is then materialized into a computer. Figure 6 An exemplary hardware structure of a computer according to an embodiment of this application is shown.

[0159] refer to Figure 6 The computer 500 includes an input device 501, a display device 502, an external I / F 503, RAM 504, ROM 505, a CPU 506, a communication I / F 507, an HDD 508, etc., interconnected via a bus B. It is acceptable to form a structure in which the input device 501 and the display device 502 are connected when necessary.

[0160] Input device 501 includes a keyboard, mouse, touchpad, etc., through which users input various operation signals. Display device 502 includes a monitor, etc., to display the processing results obtained by computer 500.

[0161] Communication I / F 507 is an interface configured to enable computer 500 to connect to various networks. Therefore, computer 500 performs data communication via communication I / F 507.

[0162] HDD 508 is an exemplary non-volatile storage device for storing programs and data. The stored data includes the operating system (OS) that forms the basis of the software controlling the entire computer 500, application software (also referred to herein as "applications") that provides various functionalities within the OS, and so on. Computer 500 may use a drive device that uses flash memory (e.g., a solid-state drive (SSD)) as the storage medium instead of HDD 508.

[0163] External I / F 503 is an interface for external devices. These external devices include recording media 503a, etc. In this case, computer 500 reads information from and / or writes information to recording media 503a via external I / F 503. Recording media 503a may be a floppy disk, CD, DVD, SD memory card, USB storage device, etc.

[0164] ROM 505 is a non-volatile semiconductor memory (storage device) that retains programs and / or data even when the power is off. ROM 505 stores programs and data used to execute the Basic Input / Output System (BIOS), OS settings, network settings, etc., at the time of power-on of computer 500. RAM 504 is an example of a volatile semiconductor memory (storage device) for temporary storage of programs and / or data.

[0165] CPU 506 is an algorithmic device that reads programs and / or data from storage devices such as ROM 505 and HDD 508. The read program or read data performs a process, thereby materializing the control or functional capabilities of the entire computer 500.

[0166] The client 801 and the named entity recognition device 802 communicate via, for example Figure 6 The hardware structure of the computer 500 shown is materialized.

[0167] Mobile terminals:

[0168] For example, the client 801 accesses the client via, as follows: Figure 7 The hardware structure shown is then materialized. Figure 7 An exemplary hardware structure of the mobile terminal in this embodiment is shown. Figure 7 The mobile terminal 12 shown includes a CPU 601, ROM 602, RAM 603, EEPROM 604, CMOS sensor 605, acceleration and orientation sensor 606, and media driver 608.

[0169] The CPU 601 controls the entire operation of the mobile terminal 12. The ROM 602 stores basic input and output programs. The RAM 603 is used as the working area of ​​the CPU 601. The EEPROM 604 reads or writes data corresponding to the control of the CPU 601. The CMOS sensor 605 captures image data corresponding to the control of the CPU 601 to obtain image data. The acceleration and orientation sensor 606 is an electromagnetic compass, rotary compass, acceleration sensor, etc., that detects the Earth's magnetic field.

[0170] Media drive 608 controls the reading or writing (storage) of data from or sent to recordable medium 607, such as flash memory. Data already stored in recordable medium 607 is read out, or new data is written to recordable medium 607. Recordable medium 607 is freely attachable to or detachable from media drive 608.

[0171] EEPROM 604 stores the operating system (OS) executed by CPU 601, as well as related information necessary for network settings. Applications for performing various processes of the first embodiment are stored in EEPROM 604, recordable medium 607, etc.

[0172] The CMOS sensor 605 is a charge-coupled device that converts light into electrical charges and digitizes an image of an object. The CMOS sensor 605 can be materialized, for example, by a charge-coupled device (CCD) sensor, as long as it can capture an image of the object.

[0173] In addition, the mobile terminal 12 includes an audio input unit 609, an audio output unit 610, an antenna 611, a communication unit 612, a wireless LAN communication unit 613, a wireless communication antenna 614, a wireless communication unit 615, a display 616, a touchpad 617, and a bus 619.

[0174] Audio input unit 609 converts sound into audio signals. Audio output unit 610 converts audio signals into sound. Communication unit 612 uses antenna 611 to communicate with the nearest base station device via wireless communication signals. Wireless LAN communication unit 613 performs wireless LAN communication with the access point in accordance with the IEEE 80411 standard. Wireless communication unit 615 performs wireless communication using wireless communication antenna 614.

[0175] Display 616 is configured to display images, various icons, etc. Display 616 is made of liquid crystal, organic EL, etc. Touchpad 617 is mounted on display 616 and is formed of a pressure-sensitive plate or an electrostatic plate. Touch positions on display 616 are detected by touch with a finger or stylus. Bus 619 is an address bus, data bus, etc., electrically connecting the aforementioned units or components.

[0176] Client 801 includes a dedicated battery 618. Client 801 is powered by battery 618. Audio input unit 609 includes a microphone for inputting sound. Audio output unit 610 includes a loudspeaker for outputting sound.

[0177] For example, client 801 via, for example, Figure 7 The hardware structure shown is then materialized.

[0178] The following is combined Figure 8 This document provides an example diagram illustrating the workflow of the named entity recognition system 800 described above. In this workflow, the recognition model trained according to the embodiments of this application is used to recognize named entities, thereby improving recognition performance. The workflow specifically includes:

[0179] In S801, the user sends a sequence to be predicted to the named entity recognition device 802 via the client 801. The named entity recognition device 802 receives the sequence to be predicted. Specifically, the calling module may receive the sequence to be predicted.

[0180] In S802, module 804 is called to determine the span in the sequence to be predicted, and the recognition model 803 is called to generate a vector representation of the sequence to be predicted, thereby obtaining the vector representation of the span; comparison module 805 compares the vector representation of each span with the final representation of various entity type identifiers to determine the entity type of each span.

[0181] In S802, the named entity recognition device 802 sends the identified entities and their types from the training data to the client 801. The client 801 can display the entities and their types through its display device. Specifically, the client 801 can display the entities and their types through the interface of its display device.

[0182] In some embodiments of this application, a computer-readable storage medium is also provided, on which a program is stored, which, when executed by a processor, performs the following steps:

[0183] Determine at least one span in the sequence to be predicted, generate a vector representation of the sequence to be predicted using a pre-trained recognition model, and obtain a vector representation of the span in the sequence to be predicted, wherein the number of tokens in the span is not greater than a preset threshold.

[0184] The vector representation of the first span in the sequence to be predicted is compared with the final representation of various entity type identifiers obtained in advance to determine the entity type of the first span; the first span is any one of the at least one span.

[0185] The recognition model is obtained by fine-tuning a pre-trained language model based on a contrastive learning objective function. The objective function is a function constructed based on the vector representation of spans in the training samples, so that the distance between the vector representations of spans of the same category is close, and the distance between the vector representations of spans of different categories is far. The training samples include multiple original samples and multiple augmented samples. The original samples are samples with labeled entity types, and the augmented samples are obtained by replacing the entities in the original samples with the corresponding entity type identifiers. The final representation of each entity type identifier is obtained by average pooling the vector representations of the same entity type identifier in the vector representations of the multiple augmented samples generated by the recognition model.

[0186] When executed by the processor, this program can implement all the above-mentioned named entity recognition methods and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0187] This application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, they implement the various processes of the above-described named entity recognition method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0188] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0189] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0190] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0191] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0192] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0193] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0194] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.< / org> < / org> < / org> < / loc> < / org> < / per> < / org> < / loc> < / org> < / per> < / org> < / loc> < / per>

Claims

1. A named entity recognition method, characterized in that, include: Determine at least one span in the sequence to be predicted, generate a vector representation of the sequence to be predicted using a pre-trained recognition model, and obtain a vector representation of the span in the sequence to be predicted, wherein the number of tokens in the span is not greater than a preset threshold. The vector representation of the first span in the sequence to be predicted is compared with the final representation of various entity type identifiers obtained in advance to determine the entity type of the first span; the first span is any one of the at least one span. The recognition model is obtained by fine-tuning a pre-trained language model based on a contrastive learning objective function. The objective function is a function constructed based on the vector representation of spans in the training samples, so that the distance between the vector representations of spans of the same category is close, and the distance between the vector representations of spans of different categories is far. The training samples include multiple original samples and multiple augmented samples. The original samples are samples with labeled entity types, and the augmented samples are obtained by replacing the entities in the original samples with the corresponding entity type identifiers. The final representation of each entity type identifier is obtained by average pooling the vector representations of the same entity type identifier in the vector representations of the multiple augmented samples generated by the recognition model.

2. The method according to claim 1, characterized in that, The span of the same category includes: the span corresponding to the first entity in the first original sample and the span corresponding to the first entity type identifier in the first enhanced sample, wherein the first original sample is any original sample among the multiple original samples, the first entity is any entity in the first original sample; the first enhanced sample is the enhanced sample corresponding to the first original sample, and the first entity type identifier is the entity type identifier that replaces the first entity in the first enhanced sample; The spans of the different categories include: the span corresponding to the first entity in the first original sample and the span corresponding to any non-entity in any training sample; the span corresponding to the first entity type identifier in the first augmented sample and the span corresponding to any non-entity in any training sample.

3. The method according to claim 1, characterized in that, The vector representation of the span is generated based on the vector representation of the starting token of the span, the vector representation of the ending token of the span, and the vector representation of the length of the span.

4. The method according to claim 1, characterized in that, The vector representation of the first span in the sequence to be predicted is compared with the final representation of various pre-obtained entity type identifiers to determine the entity type of the first span, including: Calculate the similarity between the vector representation of the first span and the final representation of various entity type identifiers, and determine the entity type of the first span based on the entity type identifier with the highest similarity.

5. The method according to claim 4, characterized in that, Based on the entity type identifier with the highest similarity, the entity type of the first span is determined, including: When the highest similarity is lower than the reference similarity, the first span is determined to be a non-entity; When the highest similarity is not lower than the reference similarity, the entity type of the first span is determined to be the entity type corresponding to the entity type identifier of the highest similarity. The reference similarity is the similarity between the vector representation of the first span and the vector representation of the preset token.

6. The method according to claim 1, characterized in that, Also includes: The recognition model is obtained through pre-training, wherein the training of the recognition model specifically includes: Obtain the training samples; Determine the span in the training samples, generate a vector representation of the training samples using a pre-trained language model, and obtain the vector representation of the span in the training samples; Based on the vector representation of the span in the training samples, the objective function of contrastive learning is constructed, and the pre-trained language model is fine-tuned based on the objective function to obtain the recognition model; The recognition model is used to generate vector representations of the multiple enhanced samples. Average pooling is then performed on the vector representations of the same entity type identifiers in the multiple enhanced samples to generate the final representation of the corresponding entity type identifier.

7. A named entity recognition device, characterized in that, include: The calling module is used to determine at least one span in the sequence to be predicted, generate a vector representation of the sequence to be predicted using a pre-trained recognition model, and obtain a vector representation of the span in the sequence to be predicted, wherein the number of tokens in the span is not greater than a preset threshold. The comparison module is used to compare the vector representation of the first span in the sequence to be predicted with the final representation of various entity type identifiers obtained in advance to determine the entity type of the first span; the first span is any one of the at least one span; The recognition model is obtained by fine-tuning a pre-trained language model based on a contrastive learning objective function. The objective function is a function constructed based on the vector representation of spans in the training samples, so that the distance between the vector representations of spans of the same category is close, and the distance between the vector representations of spans of different categories is far. The training samples include multiple original samples and multiple augmented samples. The original samples are samples with labeled entity types, and the augmented samples are obtained by replacing the entities in the original samples with the corresponding entity type identifiers. The final representation of each entity type identifier is obtained by average pooling the vector representations of the same entity type identifier in the vector representations of the multiple augmented samples generated by the recognition model.

8. The apparatus according to claim 7, characterized in that, The span of the same category includes: the span corresponding to the first entity in the first original sample and the span corresponding to the first entity type identifier in the first enhanced sample, wherein the first original sample is any original sample among the multiple original samples, the first entity is any entity in the first original sample; the first enhanced sample is the enhanced sample corresponding to the first original sample, and the first entity type identifier is the entity type identifier that replaces the first entity in the first enhanced sample; The spans of the different categories include: the span corresponding to the first entity in the first original sample and the span corresponding to any non-entity in any training sample; the span corresponding to the first entity type identifier in the first augmented sample and the span corresponding to any non-entity in any training sample.

9. The apparatus according to claim 7, characterized in that, The vector representation of the span is generated based on the vector representation of the starting token of the span, the vector representation of the ending token of the span, and the vector representation of the length of the span.

10. The apparatus according to claim 7, characterized in that, The comparison module is further configured to calculate the similarity between the vector representation of the first span and the final representation of various entity type identifiers, and determine the entity type of the first span based on the entity type identifier with the highest similarity.

11. The apparatus according to claim 10, characterized in that, The comparison module is further configured to determine that the first span is a non-entity when the highest similarity is lower than the reference similarity; and to determine that the entity type of the first span is the entity type corresponding to the entity type identifier of the highest similarity when the highest similarity is not lower than the reference similarity. The reference similarity is the similarity between the vector representation of the first span and the vector representation of the preset token.

12. The apparatus according to claim 7, characterized in that, Also includes: A training module for pre-training the recognition model, wherein the training of the recognition model specifically includes: Obtain the training samples; Determine the span in the training samples, generate a vector representation of the training samples using a pre-trained language model, and obtain the vector representation of the span in the training samples; Based on the vector representation of the span in the training samples, the objective function of contrastive learning is constructed, and the pre-trained language model is fine-tuned based on the objective function to obtain the recognition model; The recognition model is used to generate vector representations of the multiple enhanced samples. Average pooling is then performed on the vector representations of the same entity type identifiers in the multiple enhanced samples to generate the final representation of the corresponding entity type identifier.

13. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the named entity recognition method as described in any one of claims 1 to 6.

14. A computer program product, characterized in that, Includes computer instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 6.

Images