Method for training entity recognition model, entity recognition method and corresponding apparatus

Abstract

Embodiments of the present application disclose a method for training an entity recognition model, an entity recognition method and corresponding devices. The method comprises: obtaining at least two training sample sets, wherein different training sample sets are based on different entity type label sets; and training an entity recognition model using the at least two training sample sets. The training comprises: taking a text sample in the at least two training sample sets as an input of the entity recognition model, and outputting a first vector of each element in the text sample by the entity recognition model; performing mask processing on probability information corresponding to part of entity type labels in the first vector of each element according to an entity type label set based on the text sample, to obtain a second vector; and the training target is to minimize the difference between the entity type determined according to the second vector and the label annotated for the text sample. The present application can obtain an entity recognition model at low cost and ensure the recognition effect.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and in particular to a method for training an entity recognition model, an entity recognition method, and a corresponding device. Background Technology

[0002] Entity recognition is one of the hot research directions in natural language processing. Its purpose is to identify entities in text and classify them into corresponding entity types. It is a fundamental tool for many applications such as information extraction, question answering systems, syntactic analysis, and machine translation.

[0003] In practical natural language processing tasks, the problem of incremental entity type recognition is frequently encountered. For example, initially 20 entity types are defined, and labels are generated on the sample set based on these 20 types. As business grows, it becomes apparent that the initial 20 entity types are insufficient, requiring the addition of 10 new entity types. In this situation, how to obtain an entity recognition model at low cost while achieving good entity recognition performance becomes a pressing issue. Summary of the Invention

[0004] In view of this, this application provides a method for training an entity recognition model, an entity recognition method, and a corresponding apparatus, so as to obtain an entity recognition model at low cost and ensure recognition effect.

[0005] This application provides the following solution:

[0006] Firstly, a method for training an entity recognition model is provided, the method comprising:

[0007] Obtain at least two training sample sets, each training sample set including text samples and labels annotating the text samples based on entity type label sets, wherein the entity type label sets on which different training sample sets are based are different;

[0008] The entity recognition model is trained using the at least two training sample sets.

[0009] The training includes: taking text samples from the at least two training sample sets as input to the entity recognition model; outputting a first vector for each token (element) in the text sample by the entity recognition model; the first vector including the probability distribution information of the token on a target label set, the target label set being the union of the entity type label sets on which the at least two training sample sets are based; masking the probability information corresponding to a portion of the entity type labels in the first vector of each token according to the entity type label set on which the text sample is based, to obtain a second vector; the portion of the entity type labels being labels that do not belong to the entity type label set on which the text sample is based; the training objective is to minimize the difference between the entity type determined based on the second vector and the label labeled on the text sample.

[0010] According to one achievable method in an embodiment of this application, obtaining at least two training sample sets includes:

[0011] The historical sample set is obtained and a new sample set is constructed. The historical sample set includes the text samples used to train the previous entity recognition model and their labeled tags. The new sample set includes the text samples and the labels labeled on the text samples based on the newly added entity type label set.

[0012] The step of merging the historical sample set and the newly added sample set to execute the training entity recognition model.

[0013] According to one achievable method in an embodiment of this application, obtaining at least two training sample sets further includes:

[0014] The labels in the historical sample set are edited, including changing the labels corresponding to the deleted entity types to non-entity types;

[0015] Merging the historical sample set and the newly added sample set includes merging the historical sample set after editing and the newly added sample set.

[0016] According to one achievable method in this application embodiment, the step of masking the probability information corresponding to a portion of the entity type tags in the first vector of each Token based on the entity type tag set on which the text sample is based, to obtain the second vector, includes:

[0017] The first vector of each token in the text sample is multiplied by the mask variable of the text sample to obtain the second vector of each token; wherein the mask variable includes N bits, where N is the number of tags contained in the target tag set, and the bits corresponding to the entity type tag set on which the text sample is based are set to 1, and the other bits are set to 0.

[0018] The training also includes: calculating the value of the loss function based on the second vector of each token and the label of each token, and updating the model parameters of the entity recognition model using the value of the loss function until the preset training termination condition is met.

[0019] According to one achievable method in an embodiment of this application, the entity recognition model includes:

[0020] A feature extraction network based on a pre-trained language model is used to extract features from each token in the input text sample to obtain the feature representation of each token.

[0021] A mapping network is used to map the feature representations of each Token to a target label set to obtain a first vector for each Token. The first vector includes the probability information of the Token on each entity type label contained in the target label set.

[0022] Secondly, a method for entity recognition is provided, the method comprising:

[0023] Obtain the text to be recognized;

[0024] The text to be identified is input into the entity recognition model, and the first vector of each token in the text to be identified is obtained from the output of the entity recognition model. The first vector includes the probability distribution information of the token on the target label set.

[0025] Based on the first vector of each token in the text to be identified, determine the entity type information corresponding to each token;

[0026] The entity recognition model is pre-trained using the method described in any one of the first aspects above.

[0027] According to one feasible method in the embodiments of this application, each text in a preset text set is taken as the text to be identified, and entity words are extracted from the text to be identified using entity type information determined in the text to be identified, in order to construct a knowledge graph; or,

[0028] The text entered by the user in the intelligent customer service system is used as the text to be identified. Entity words are extracted from the text using the entity type information determined in the text to be identified. The extracted entity words are matched with a preset keyword table. The attribute information or service items corresponding to the matched keywords are returned to the user.

[0029] Thirdly, an apparatus for training an entity recognition model is provided, the apparatus comprising:

[0030] The sample acquisition unit is configured to acquire at least two training sample sets, each training sample set including text samples and labels annotating the text samples based on entity type label sets, wherein the entity type label sets on which different training sample sets are based are different;

[0031] The model training unit is configured to train an entity recognition model using the at least two training sample sets. The training includes: taking text samples from the at least two training sample sets as input to the entity recognition model, and having the entity recognition model output a first vector for each element Token in the text sample. The first vector includes the probability distribution information of the Token on a target label set, where the target label set is the union of the entity type label sets on which the at least two training sample sets are based. Based on the entity type label set on which the text sample is based, the probability information corresponding to a portion of the entity type labels in the first vector of each Token is masked to obtain a second vector, where the portion of the entity type labels are labels that do not belong to the entity type label set on which the text sample is based. The training objective is to minimize the difference between the entity type determined based on the second vector and the label labeled on the text sample.

[0032] Fourthly, an entity recognition device is provided, characterized in that the device comprises:

[0033] The text acquisition unit is configured to acquire the text to be recognized.

[0034] The entity recognition unit is configured to input the text to be recognized into an entity recognition model, obtain the first vector of each token in the text to be recognized output by the entity recognition model, wherein the first vector includes the probability distribution information of the token on the target label set; and determine the entity type information corresponding to each token based on the first vector of each token in the text to be recognized.

[0035] The entity recognition model is pre-trained using the apparatus described in the third aspect above.

[0036] According to a fifth aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the first or second aspects above.

[0037] According to a sixth aspect, an electronic device is provided, comprising:

[0038] One or more processors; and

[0039] A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any one of the first and second aspects above.

[0040] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0041] 1) The entity recognition model used in this application maintains the entity type labels on which each training sample set is based during training, eliminating the need to re-label all training sample sets and greatly reducing manual costs. Furthermore, by masking the probability information corresponding to the labels in the entity type label set deployed on the text samples during training, the negative impact caused by the lack of entity type labels between different training sample sets is eliminated, ensuring the recognition performance of the entity recognition model.

[0042] 2) For the case of new entity types, this application only needs to annotate the text samples based on the new entity type label set to generate a new sample set. The new sample set is then combined with the historical sample set to train the entity recognition model. There is no need to re-annotate the historical sample set, which obviously reduces the cost of data annotation.

[0043] 3) For cases involving modification of entity types, this can be viewed as two separate processes: deleting and adding entity types. Therefore, the labels in the historical sample set can be simply edited by changing the labels corresponding to the deleted entity types to non-entity types. Then, the method for adding entity types can be executed according to the procedure described in this application to train the entity recognition model corresponding to the modified entity type. The entire process also eliminates the need for re-labeling all samples (including the historical and new sample sets) based on the modified entity types, resulting in low data labeling costs.

[0044] 4) During the training process, this application only needs to add the mask variable of each text sample, multiply the first vector of each token by the mask variable of the text sample to obtain the second vector, and calculate the loss function based on the second vector to update the model parameters. The model calculation is simple.

[0045] 5) The method adopted in this application still ensures that the model framework of the entity recognition model remains unchanged. Only a slight modification is needed on the basis of the general entity recognition model to adapt to the target label set, thus ensuring the effect of the entity recognition model on the basis of a simple model structure.

[0046] Of course, any product implementing this application does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is an exemplary system architecture diagram that can be applied to embodiments of this application;

[0049] Figure 2 A flowchart illustrating the method for training an entity recognition model provided in this application embodiment;

[0050] Figure 3 This is a schematic diagram of the structure of the training entity recognition model provided in the embodiments of this application;

[0051] Figure 4 A schematic block diagram of an apparatus for training an entity recognition model provided in an embodiment of this application;

[0052] Figure 5 A schematic block diagram of the entity recognition device provided in the embodiments of this application;

[0053] Figure 6 A schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0055] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0056] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0057] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0058] For newly added entity types, the existing processing methods mainly include the following three:

[0059] The first approach involves having annotators re-annotate the samples based on all the new entity types, and then retraining the entity recognition model. This method involves re-annotating the entire dataset. If new entity types need to be added multiple times at different stages due to business requirements, the annotation cost will increase linearly. Furthermore, deep learning itself requires a large number of samples; the number of annotated samples could be in the tens of thousands, or even millions or tens of millions. This would result in an enormous workload for annotators, leading to excessively high labor costs.

[0060] The second approach involves annotating the new sample set based on the newly added entity types, then merging the new sample set with the historical sample set to retrain the entity recognition model. However, in this approach, since the historical sample set was not annotated based on the new entity types, it becomes a negative sample for the new entity types, resulting in a poor performance of the trained entity recognition model.

[0061] The third approach involves using distillation learning, where a pre-trained entity recognition model is used as the teacher model to guide the student model in learning new entity types while retaining previously learned knowledge. However, this method involves very complex model design, and the entity recognition model obtained through distillation learning does not perform well.

[0062] In view of this, this application provides a novel approach, transforming the incremental entity recognition problem into a problem of training an entity recognition model by fusing datasets based on different entity type label sets. To facilitate understanding of this application, the system architecture to which this application applies is first briefly described. Figure 1 An exemplary system architecture applicable to embodiments of this application is shown. This system architecture includes a model training device for training an entity recognition model offline, and an entity recognition device for performing entity recognition on text to be recognized online.

[0063] The model training device can train the model using the method provided in the embodiments of this application each time a new entity type is added, so as to obtain an entity recognition model.

[0064] Entity recognition devices utilize established entity recognition models to perform entity recognition on the text to be recognized and obtain recognition results. That is, they identify the words containing entities from the text to be recognized and determine the entity type of each word.

[0065] The model training device and the entity recognition device can be configured as separate servers, or they can be configured on the same server or server cluster, or they can be configured on a separate or the same cloud server. A cloud server, also known as a cloud computing server or cloud host, is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Servers (VPs) services, such as high management difficulty and weak service scalability. The model training device and the entity recognition device can also be configured on computer terminals with strong computing capabilities.

[0066] It should be noted that, in addition to performing entity recognition online, the aforementioned entity recognition device can also perform entity recognition offline, such as performing entity recognition on batches of text to be recognized.

[0067] It should be understood that Figure 1 The number of model training devices, entity recognition devices, and entity recognition models shown in the diagram is merely illustrative. Depending on implementation needs, any number of model training devices, entity recognition devices, and entity recognition models can be included.

[0068] Figure 2 This is a flowchart of a method for training an entity recognition model provided in an embodiment of this application. The method can be described by… Figure 1 The model training device in the system shown is executed. For example... Figure 2 As shown, the method may include the following steps:

[0069] Step 202: Obtain at least two training sample sets, each training sample set including text samples and labels annotating the text samples based on entity type label sets, wherein the entity type label sets on which different training sample sets are based are different.

[0070] Step 204: Train the entity recognition model using the above-mentioned at least two training sample sets; wherein the training includes: taking text samples from the at least two training sample sets as input to the entity recognition model, and having the entity recognition model output the first vector of each token (element) in the text sample, the first vector including the probability distribution information of the token on the target label set, the target label set being the union of the entity type label sets on which the at least two training sample sets are based; based on the entity type label set on which the text sample is based, performing masking processing on the probability information corresponding to a portion of the entity type labels in the first vector of each token to obtain a second vector, the portion of the entity type labels being labels that do not belong to the entity type label set on which the text sample is based; the training objective is: to minimize the difference between the entity type determined based on the second vector and the label labeled on the text sample.

[0071] As can be seen from the above process, the entity recognition model adopted in this application maintains the entity type labels on which each training sample set is based during training, eliminating the need to re-label all training sample sets and greatly reducing manual costs. Furthermore, by masking the probability information corresponding to the labels in the entity type label set deployed on the text samples during training, the negative impact caused by the lack of entity type labels between different training sample sets is eliminated, ensuring the recognition performance of the entity recognition model.

[0072] The steps in the above process are described in detail below.

[0073] First, the above step 202, namely "obtaining at least two training sample sets", will be described in detail with reference to the embodiments.

[0074] When faced with the problem of incremental entity types, this embodiment of the application can retain the historical sample set unchanged, that is, the historical sample set used in the training process of the previous entity recognition model. This historical sample set is annotated based on the original entity type label set. The new sample set includes text samples and labels annotated on the text samples based on the new entity type label set.

[0075] For example, three entity type labels are initially defined: label1, label2, and label3. These three entity type labels form the first entity type label set {label1, label2, label3, O}, where O represents a non-entity type label. Based on this first entity type label set, text samples in sample set 1 are labeled to obtain a training sample set A. Entity recognition model 1 is then trained using training sample set A.

[0076] After a period of business development, two new entity type labels were added: label4 and label5. These two entity type labels form a second entity type label set {label4, label5, O}. Based on this second entity type label set, the text samples in sample set 2 are labeled to obtain a training sample set B. Since new entity type labels have been added, a new entity recognition model needs to be trained. In this stage, the historical sample set is used as training sample set A, and the newly added sample set is used as training sample set B. Using the method described in this embodiment, entity recognition model 2 is trained using training sample set A and training sample set B.

[0077] After a period of business development, two new entity type labels were added: label6 and label7. These two entity type labels form a third entity type label set {label6, label7, O}. Based on this third entity type label set, the text samples in sample set 3 are labeled to obtain a training sample set C. Since new entity type labels have been added, a new entity recognition model needs to be trained. In this stage, the historical sample sets are training sample sets A and B, and the newly added sample set is training sample set C. Using the method described in this embodiment, entity recognition model 3 is trained using training sample sets A, B, and C.

[0078] It should be noted that sample sets 1, 2, and 3 may contain some of the same text samples, completely different text samples, or completely identical text samples. However, the entity type label set used for labeling each sample set is different.

[0079] The first entity tag set, the second entity tag set, and the third entity tag set mentioned above can be completely different entity tag sets, or they can be entity tag sets with some tags in common.

[0080] The target label set used for each training iteration of the entity recognition model is the union of the existing entity type label set and the newly added entity type label set. Taking the third stage above as an example, the target label set used when training entity recognition model 3 is {label1, label2, label3, label4, label5, label6, label7, and O}. For new sample sets, only the text samples need to be labeled with the newly added entity type labels; there is no need to label the existing entity type labels. For example, in the third stage above, for sample set 3, only the tokens (elements) of label6, label7, and O need to be labeled. Of course, it is also possible to label sample set 3 based on all labels in the target label set, but the former is preferred.

[0081] As can be seen from the example above, the different training sample sets obtained in this step are based on different sets of entity type labels.

[0082] It should be noted that the terms "first" and "second" used in this disclosure do not have any restrictions on size, order, or quantity, but are only used to distinguish them by name. For example, "first entity type label set", "second entity type label set" and "third entity type label set" are used to distinguish the three entity type label sets.

[0083] The entity types involved in the embodiments of this application may include, but are not limited to, personal names, place names, time, date, organization names, currencies, movie titles, book titles, etc.

[0084] The following describes step 204, namely "training the entity recognition model using the above-mentioned at least two training sample sets", in detail with reference to the embodiments.

[0085] The training sample sets obtained in step 202 are merged, and the merged sample set is used to train the entity recognition model. For ease of understanding, the structure of the entity recognition model is first briefly described. For example... Figure 3 As shown, the entity recognition model mainly consists of two parts: a feature extraction network and a mapping network.

[0086] The feature extraction network is used to extract features from each token in the input text sample, obtaining the feature representation of each token. The feature extraction network can be implemented based on a pre-trained language model or based on LSTM (Long-Short Term Memory) network, etc.

[0087] The pre-trained language model can be an initial feature extraction model, such as BERT (Bidirectional Encoder Representation from Transformers), XLNet, or GPT (Generative Pre-Training), and then further trained on top of it.

[0088] The input text sample typically begins with a start character (usually denoted by "[CLS]"), followed by tokens that typically include characters (some languages ​​may use words) and delimiters (usually denoted by "[SEP]"). For example Figure 3As shown in the figure, assuming that the text sample contains M tokens, the above feature extraction network can first perform embedding processing on each token, and then use each Transformer layer in the pre-trained language model to encode the embedding of each token to obtain the feature representation of each token.

[0089] The embedding processing for each token includes at least two parts: word embedding and position embedding. Word embedding involves encoding each token into word vectors to obtain a word vector representation. Position embedding involves encoding the position of each token in the text sample to obtain a position representation. For example, the tokens are numbered sequentially as 0, 1, 2, 3, 4, 5, and 6 based on their positions in the text sample.

[0090] A mapping network is used to map the feature representations of each token to a set of target labels, resulting in the first vector of each token. Figure 3 The values ​​are denoted as T1, T2, ..., TM respectively. The first vector includes the probability information of the Token on each entity type label contained in the target label set.

[0091] The mapping network can be a CRF (conditional random field) or a network such as Softmax.

[0092] The mapping network outputs a first vector for each token, the length of which is the same as the number of tags N in the target tag set. Taking the third stage in the example above as an example, the target tag set is {label1, label2, label3, label4, label5, label6, label7, and O}, therefore, the length of the first vector is 8. The value of each bit in the first vector is the probability information of that token for each entity type.

[0093] In this application, to eliminate the mutual influence between text tags from different sources, i.e., the mutual influence between tags of text samples based on different entity type tag sets, during model training, the probability information corresponding to a portion of the entity type tags in the first vector of each token is masked to obtain the second vector. Figure 3 The terms L1, L2, ..., LM are used to represent the entity types. Some of these entity type labels are labels that do not belong to the entity type label set on which the text sample is based.

[0094] One possible approach is to multiply the first vector of each token in the text sample by the mask variable of the text sample to obtain the second vector of each token. The mask variable includes N bits, where N is the number of tags contained in the target tag set. The bits corresponding to the entity type tag set on which the text sample is based are set to 1, and the other bits are set to 0.

[0095] Taking the third stage in the example above, each text sample in training sample set A is based on the first entity type label set, and the mask variables for these text samples are [1,1,1,0,0,0,0,1]. Each text sample in training sample set B is based on the second entity type label set, and the mask variables for these text samples are [0,0,0,1,1,0,0,1]. Each text sample in training sample set C is based on the third entity type label set, and the mask variables for these text samples are [0,0,0,0,0,1,1,1].

[0096] Multiplying the first vector of the Token in the text sample by the mask variable is equivalent to masking the probability information on the unused entity type labels in the text sample, thereby eliminating the influence of the probability information on the unused entity type labels.

[0097] Taking training sample set A as an example, multiplying the first vector of each token in the text sample by the mask variable of that text sample is equivalent to eliminating the influence of unlabeled entities of labels 4, 5, 6, and 7. Therefore, for labels 4, 5, 6, and 7, training sample set A will not become a negative sample of the entities corresponding to these labels. Similarly, multiplying the first vector of each token in the text samples of training sample set B by the mask variable of that text sample is equivalent to eliminating the influence of unlabeled entities of labels 1, 2, 3, 6, and 7. Therefore, for labels 1, 2, 3, 6, and 7, training sample set B will not become a negative sample of the entities corresponding to these labels. Likewise, entities corresponding to label types not based on in other training sample sets will not influence each other.

[0098] The training objective of the entity recognition model is to minimize the difference between the entity type determined by the second vector and the label assigned to the text sample. A loss function can be designed based on this training objective, such as the cross-entropy loss function. That is, in each iteration, the loss function value is calculated using the second vector and the label assigned to each token. The model parameters are then updated using methods such as gradient descent based on the loss function value until a preset training termination condition is met. This termination condition may include, for example, the loss function value being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold.

[0099] In real-world business scenarios, you'll often encounter situations where you add new entity types. However, you may also encounter situations where you delete or modify entity types.

[0100] For cases where entity types are deleted, only the labels in the historical training sample set need to be edited. This editing process includes replacing all deleted entity type labels in the historical training sample set with "O" (i.e., non-entity types). Then, the entity recognition model is trained using the replaced training sample set.

[0101] For cases involving modification of entity types, the process can be viewed as first deleting the modified entity type and then adding the modified entity type. Specifically, the labels in the historical sample set are first edited, including changing the labels corresponding to the deleted entity type to "O". Then, the edited historical sample set and the newly added sample set are merged. The newly added sample set is obtained by labeling new training samples using the modified entity type. Finally, the entity recognition model is trained using the historical and newly added sample sets in step 204 above. The target type label set of this entity recognition model is the label type set after the entity type modification.

[0102] After training, when using the trained entity recognition model for entity recognition, the text to be recognized is input into the entity recognition model, and the first vector of each token in the text output by the entity recognition model is obtained. Specifically, the feature extraction network extracts features from the text to be recognized, obtaining the feature representation of each token in the text. The mapping network maps the feature representation of each token to the target label set, obtaining the first vector of each token. The first vector includes the probability information of each token on each entity type label contained in the target label set.

[0103] Then, based on the first vector of each token in the text to be identified, the entity type information corresponding to each token is determined. For example, the entity type with the highest probability value of the token on each entity type label is selected as the entity type to which the token belongs.

[0104] The entity recognition model trained in this application can be applied to a variety of application scenarios, of which only two are listed here:

[0105] Application Scenario 1: Building a knowledge graph.

[0106] Knowledge graphs are primarily used to describe various entities existing in the real world and the relationships between them. These entities can be mined from large amounts of text. During the construction of a knowledge graph, new things and corresponding new entity types will inevitably emerge. In such cases, instead of fully re-annotating the entire historical training sample set, annotation can be performed only on new text samples based on the newly added entity type labels. Then, the entity recognition model can be trained using the method provided in this application's embodiments. This approach results in low annotation costs while ensuring the recognition performance of the entity recognition model.

[0107] Then, an entity recognition model is used to identify text in a pre-defined text set, and entities are identified as nodes in the knowledge graph. The edges in the knowledge graph represent the relationships between entities.

[0108] Application Scenario 2: Customer Service.

[0109] More and more customers are now using online customer service to communicate when encountering service issues. Many current online customer service systems employ intelligent customer service systems, which use the text entered by the user as the text to be recognized. Entity words are extracted from the text based on the entity type information identified within the text, and these extracted entity words are matched against a pre-set keyword list. The attribute information or service items corresponding to the matched keywords are then returned to the user. Most of these keywords are entity words, but as services diversify and become more complex, new entity types will emerge. In such cases, instead of fully re-annotating the entire historical training sample set, annotation can be performed only on new text samples based on the newly added entity type labels. Then, the entity recognition model can be trained using the method provided in this application's embodiment. This approach reduces annotation costs while ensuring the recognition performance of the entity recognition model.

[0110] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0111] According to another embodiment, an apparatus for training an entity recognition model is provided. Figure 4A schematic block diagram of an apparatus for training an entity recognition model according to one embodiment is shown. Figure 4 As shown, the device 400 includes a sample acquisition unit 401 and a model training unit 402, wherein the main functions of each component unit are as follows:

[0112] The sample acquisition unit 401 is configured to acquire at least two training sample sets, each training sample set including text samples and labels annotating the text samples based on entity type label sets, wherein the entity type label sets on which different training sample sets are based are different.

[0113] The model training unit 402 is configured to train an entity recognition model using at least two training sample sets. The training includes: taking text samples from at least two training sample sets as input to the entity recognition model, and having the entity recognition model output a first vector for each element Token in the text sample. The first vector includes the probability distribution information of the Token on a target label set, where the target label set is the union of the entity type label sets on which the at least two training sample sets are based. Based on the entity type label set on which the text sample is based, the probability information corresponding to a portion of the entity type labels in the first vector of each Token is masked to obtain a second vector, where the portion of the entity type labels are labels that do not belong to the entity type label set on which the text sample is based. The training objective is to minimize the difference between the entity type determined based on the second vector and the label labeled on the text sample.

[0114] As one possible implementation method, the sample acquisition unit 401 can be specifically configured to: acquire a historical sample set and construct a new sample set. The historical sample set includes the text samples used to train the previous entity recognition model and their labeled tags. The new sample set includes the text samples and the labels labeled on the text samples based on the newly added entity type label set.

[0115] Accordingly, the model training unit 402 merges the historical sample set and the newly added sample set to perform the above-mentioned training entity recognition model processing.

[0116] As one possible approach, if the entity type needs to be modified, the sample acquisition unit 401 can edit the labels in the historical sample set. The editing process includes modifying the labels corresponding to the deleted entity type to non-entity types.

[0117] Accordingly, model training unit 402 merges the historical sample set after editing and processing, as well as the newly added sample set.

[0118] As one possible implementation, the model training unit 402 can be specifically configured to: multiply the first vector of each token in the text sample by the mask variable of the text sample to obtain the second vector of each token; wherein the mask variable includes N bits, where N is the number of tags contained in the target tag set, and each bit corresponding to the entity type tag set on which the text sample is based is set to 1, and the other bits are set to 0.

[0119] Accordingly, the model training unit 402 is configured to calculate the value of the loss function based on the second vector of each token and the label of each token, and update the model parameters of the entity recognition model using the value of the loss function until the preset training termination condition is met.

[0120] The principles and structure of the entity recognition model can be found in [reference needed]. Figure 3 It includes feature extraction networks and mapping networks.

[0121] The feature extraction network is used to extract features from each token in the input text sample to obtain the feature representation of each token.

[0122] A mapping network is used to map the feature representation of each token to a target label set, resulting in a first vector for each token. The first vector includes the probability information of the token on each entity type label contained in the target label set.

[0123] Feature extraction networks can be implemented based on pre-trained language models or LSTM, etc. Mapping networks can use CRF networks or networks such as Softmax.

[0124] Figure 5 A schematic block diagram of an entity recognition device according to one embodiment is shown. Figure 5 As shown, the device 500 includes a text acquisition unit 501 and an entity recognition unit 502, wherein the main functions of each component are as follows:

[0125] The text acquisition unit 501 is configured to acquire the text to be recognized.

[0126] Depending on the application scenario, the text to be recognized can be obtained in different ways. For example, in a customer service scenario, the text to be recognized can be the text entered by the user. In a knowledge graph construction scenario, the text to be recognized can be text crawled from a specific website, and so on.

[0127] Entity recognition unit 502 is configured to input the text to be recognized into an entity recognition model, obtain the first vector of each token in the text to be recognized output by the entity recognition model, the first vector including the probability distribution information of the token on the target label set; and determine the entity type information corresponding to each token based on the first vector of each token in the text to be recognized. The entity recognition model employs, for example... Figure 4 The device shown is pre-trained.

[0128] For example, the entity type with the highest probability value for the Token on each entity type label is selected as the entity type to which the Token belongs.

[0129] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0130] In addition, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0131] And an electronic device, comprising:

[0132] One or more processors; and

[0133] A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any of the foregoing method embodiments.

[0134] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0135] in, Figure 6 An exemplary architecture of an electronic device is shown, which may include a processor 610, a video display adapter 611, a disk drive 612, an input / output interface 613, a network interface 614, and a memory 620. The processor 610, video display adapter 611, disk drive 612, input / output interface 613, network interface 614, and memory 620 can communicate with each other via a communication bus 630.

[0136] The processor 610 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0137] The memory 620 can be implemented in the form of ROM (Read-Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 620 can store the operating system 621 for controlling the operation of the electronic device 600, and the basic input / output system (BIOS) 622 for controlling the low-level operations of the electronic device 600. Additionally, it can store a web browser 623, a data storage management system 624, and a model training device / entity recognition device 625, etc. The aforementioned model training device / entity recognition device 625 can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when the technical solution provided in this application is implemented through software or firmware, the relevant program code is stored in the memory 620 and is called and executed by the processor 610.

[0138] Input / output interface 613 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0139] Network interface 614 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0140] Bus 630 includes a pathway for transmitting information between various components of the device, such as processor 610, video display adapter 611, disk drive 612, input / output interface 613, network interface 614, and memory 620.

[0141] It should be noted that although the above-described device only shows the processor 610, video display adapter 611, disk drive 612, input / output interface 613, network interface 614, memory 620, bus 630, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0142] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer program product. This computer program product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0143] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0144] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for training an entity recognition model, characterized in that, The method includes: Obtain at least two training sample sets, each training sample set including text samples and labels annotating the text samples based on entity type label sets, wherein the entity type label sets on which different training sample sets are based are different; The entity recognition model is trained using the at least two training sample sets. The training includes: using text samples from the at least two training sample sets as input to the entity recognition model; outputting a first vector for each element in the text sample by the entity recognition model, the first vector including the probability distribution information of the element on a target label set, the target label set being the union of the entity type label sets on which the at least two training sample sets are based; masking the probability information corresponding to a portion of the entity type labels in the first vector of each element according to the entity type label set on which the text sample is based, to obtain a second vector, the portion of the entity type labels being labels that do not belong to the entity type label set on which the text sample is based; the training objective is to minimize the difference between the entity type determined based on the second vector and the label labeled on the text sample.

2. The method according to claim 1, characterized in that, The acquisition of at least two training sample sets includes: The historical sample set is obtained and a new sample set is constructed. The historical sample set includes the text samples used to train the previous entity recognition model and their labeled tags. The new sample set includes the text samples and the labels labeled on the text samples based on the newly added entity type label set. The step of merging the historical sample set and the newly added sample set to execute the training entity recognition model.

3. The method according to claim 2, characterized in that, The step of obtaining at least two training sample sets also includes: The labels in the historical sample set are edited, including changing the labels corresponding to the deleted entity types to non-entity types; Merging the historical sample set and the newly added sample set includes merging the historical sample set after editing and the newly added sample set.

4. The method according to claim 1, characterized in that, The second vector is obtained by masking the probability information corresponding to a portion of the entity type labels in the first vector of each element, based on the entity type label set on which the text sample is based, and performing masking on the first vector of each element. The first vector of each element in the text sample is multiplied by the mask variable of the text sample to obtain the second vector of each element; wherein the mask variable includes N bits, where N is the number of tags contained in the target tag set, and each bit corresponding to the entity type tag set on which the text sample is based is set to 1, and the other bits are set to 0. The training also includes: calculating the value of the loss function based on the second vector of each element and the label of each element, and updating the model parameters of the entity recognition model using the value of the loss function until the preset training termination condition is met.

5. The method according to any one of claims 1 to 4, characterized in that, The entity recognition model includes: A feature extraction network based on a pre-trained language model is used to extract features from each element in the input text sample to obtain the feature representation of each element. A mapping network is used to map the feature representations of each element to a target label set to obtain a first vector for each element. The first vector includes the probability information of the element on each entity type label contained in the target label set.

6. A method for entity recognition, characterized in that, The method includes: Obtain the text to be recognized; The text to be identified is input into the entity recognition model, and the first vector of each element in the text to be identified is obtained from the output of the entity recognition model. The first vector includes the probability distribution information of the element on the target label set. Based on the first vector of each element in the text to be identified, determine the entity type information corresponding to each element; The entity recognition model is pre-trained using the method described in any one of claims 1 to 5.

7. The method according to claim 6, characterized in that, Each text in the preset text set is taken as the text to be identified. Entity words are extracted from the text to be identified using the entity type information determined in the text to be identified, in order to construct a knowledge graph. or, The text entered by the user in the intelligent customer service system is used as the text to be identified. Entity words are extracted from the text using the entity type information determined in the text to be identified. The extracted entity words are matched with a preset keyword table. The attribute information or service items corresponding to the matched keywords are returned to the user.

8. An apparatus for training an entity recognition model, characterized in that, The device includes: The sample acquisition unit is configured to acquire at least two training sample sets, each training sample set including text samples and labels annotating the text samples based on entity type label sets, wherein the entity type label sets on which different training sample sets are based are different; The model training unit is configured to train an entity recognition model using the at least two training sample sets. The training includes: taking text samples from the at least two training sample sets as input to the entity recognition model, and having the entity recognition model output a first vector for each element in the text sample. The first vector includes the probability distribution information of the element on a target label set, where the target label set is the union of the entity type label sets on which the at least two training sample sets are based. Based on the entity type label set on which the text sample is based, the probability information corresponding to a portion of the entity type labels in the first vector of each element is masked to obtain a second vector. The portion of the entity type labels are labels that do not belong to the entity type label set on which the text sample is based. The training objective is to minimize the difference between the entity type determined based on the second vector and the label labeled on the text sample.

9. An entity recognition device, characterized in that, The device includes: The text acquisition unit is configured to acquire the text to be recognized. An entity recognition unit is configured to input the text to be recognized into an entity recognition model, obtain a first vector of each element in the text to be recognized output by the entity recognition model, wherein the first vector includes the probability distribution information of the element on the target label set; and determine the entity type information corresponding to each element based on the first vector of each element in the text to be recognized. The entity recognition model is pre-trained using the apparatus described in claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1 to 7.

11. An electronic device, characterized in that, include: One or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1 to 7.

Images