An entity recognition model training method and device, and an entity recognition method and device

Abstract

The application provides an entity recognition model training method and device and an entity recognition method and device, and is applied to the technical field of data processing. The entity recognition model training method comprises the following steps: obtaining a sample entity set corresponding to the current iteration; wherein the sample entity set corresponding to the current iteration is obtained by updating the sample entity set corresponding to the last iteration according to a first text representation and a second text representation; the first text representation is a text representation of each initial entity in an initial entity set corresponding to the current iteration; and the second text representation is a text representation of each sample entity in the sample entity set corresponding to the last iteration; and training an entity recognition model corresponding to the last iteration according to the sample entity set corresponding to the current iteration to obtain an entity recognition model corresponding to the current iteration. Therefore, in the embodiment of the application, the sample data used for training can be automatically obtained without manual annotation, thereby reducing the cost caused by manual annotation in the training process.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to an entity recognition model training method and apparatus, and an entity recognition method and apparatus. Background Technology

[0002] Named entity recognition (NER) is a hot research area in natural language processing. Its goal is to identify desired entities and types from unstructured text, and the results can be applied to downstream tasks such as entity relation extraction and knowledge graph construction. Current technologies generally employ supervised recognition methods. However, this approach requires a large amount of manually labeled, domain-specific corpora for training the entity recognition model, resulting in high training costs and limited application scenarios for the trained model. Summary of the Invention

[0003] The purpose of this application is to provide an entity recognition model training method and apparatus, and an entity recognition method and apparatus, to solve the technical problems in the prior art where the training cost of entity recognition models is high and the application scenarios of the trained entity recognition models are limited.

[0004] Firstly, embodiments of this application provide a method for training an entity recognition model, comprising: obtaining a sample entity set corresponding to the current iteration; wherein the sample entity set corresponding to the current iteration is obtained by updating the sample entity set corresponding to the previous iteration according to a first text representation and a second text representation, wherein the first text representation is the text representation of each initial entity in the initial entity set corresponding to the current iteration, and the second text representation is the text representation of each sample entity in the sample entity set corresponding to the previous iteration; and training the entity recognition model corresponding to the previous iteration according to the sample entity set corresponding to the current iteration to obtain the entity recognition model corresponding to the current iteration. In the above scheme, a sample entity set is obtained by filtering from the initial entity set through the text representations corresponding to each initial entity in the initial entity set, and the entity recognition model is trained using the filtered sample entity set. Therefore, in embodiments of this application, the sample data used for training can be automatically obtained without manual annotation, thereby reducing the cost caused by manual annotation during the training process. In addition, the trained entity recognition model obtained in embodiments of this application is not limited by labels specific to a particular domain and can be applied to various application scenarios.

[0005] In an optional implementation, obtaining the sample entity set corresponding to the current iteration includes: obtaining the initial entity set corresponding to the current iteration; inputting the initial entity set corresponding to the current iteration and the sample entity set corresponding to the previous iteration into the text representation model corresponding to the previous iteration, respectively obtaining the first text representation corresponding to each initial entity in the initial entity set corresponding to the current iteration and the second text representation corresponding to each sample entity in the sample entity set corresponding to the previous iteration; updating the sample entity set corresponding to the previous iteration according to the first text representation and the second text representation to obtain the sample entity set corresponding to the current iteration. In the above scheme, since the initial entity set may not all be correctly labeled results, a sample entity set with higher accuracy can be obtained from the initial entity set through the text representation model. Therefore, in this embodiment, the sample data used for training can be automatically obtained without manual annotation, thereby reducing the cost caused by manual annotation during training. In addition, the trained entity recognition model obtained in this embodiment is not limited by labels of a specific domain and can be applied to various application scenarios.

[0006] In an optional implementation, updating the sample entity set corresponding to the previous iteration based on the first text representation and the second text representation to obtain the sample entity set corresponding to the current iteration includes: for an initial entity in the initial entity set corresponding to the current iteration and a subset of sample entities in the sample entity set corresponding to the previous iteration that belong to the same category as the initial entity, calculating the entity confidence score corresponding to the initial entity based on the first text representation corresponding to the initial entity and the second text representation corresponding to the subset of sample entities; if the entity confidence score is greater than an entity threshold, adding the initial entity to the sample entity set corresponding to the previous iteration. In the above scheme, by calculating the entity confidence score of the initial entity and comparing the entity confidence score with an entity threshold, the sample entity set can be updated to obtain a more accurate sample entity set.

[0007] In an optional implementation, calculating the entity confidence score of the initial entity based on the first text representation corresponding to the initial entity and the second text representation corresponding to the subset of sample entities includes: calculating multiple cosine similarities between the first text representation corresponding to the initial entity and the second text representations corresponding to each sample entity in the subset of sample entities; and determining the maximum value among the multiple pre-similarity scores as the entity confidence score. In the above scheme, the entity confidence score can be obtained by calculating the local score of each initial entity and filtering noisy data in the entity set based on the entity confidence score, thus obtaining a more accurate sample entity set.

[0008] In an optional implementation, calculating the entity confidence score of the initial entity based on the first text representation corresponding to the initial entity and the second text representation corresponding to the subset of sample entities includes: calculating multiple cosine similarities between the first text representation corresponding to the initial entity and the second text representations corresponding to each sample entity in the subset of sample entities; determining the maximum value among the multiple cosine similarities as the local score of the initial entity; calculating the global score between the first text representation corresponding to the initial entity and the second text representations corresponding to each sample entity in the subset of sample entities; and calculating the entity confidence score based on the local score and the global score. In the above scheme, the entity confidence score can be obtained by calculating the local score and the global score of each initial entity, and filtering noisy data in the entity set based on the entity confidence score, thus obtaining a sample entity set with higher accuracy.

[0009] In an optional implementation, calculating the global score between the first text representation corresponding to the initial entity and the second text representations corresponding to each sample entity in the sample entity subset includes: calculating the average text representation of the second text representations corresponding to some sample entities in the sample entity subset; calculating the cosine similarity between the average text representation and the first text representation corresponding to the initial entity to obtain the global score. In the above scheme, since the local score is estimated based on a single instance in the sample entity set, a global score estimated based on the entire sample entity set can be further calculated to obtain a sample entity set with higher accuracy.

[0010] In an optional implementation, calculating the entity confidence score based on the local score and the global score includes: calculating the geometric mean of the local score and the global score to obtain the entity confidence score. In the above scheme, to ensure the exploration capability and reliability of new sample instances in the sample entity set, the entity confidence score of an initial entity can be obtained by calculating the geometric mean of the local score and the global score, thus obtaining a sample entity set with higher accuracy.

[0011] In an optional implementation, after adding the initial entity to the sample entity set corresponding to the previous iteration, the method further includes updating the entity threshold based on the entity confidence level of the sample entities in the sample entity set corresponding to the current iteration. In the above scheme, since the entity thresholds for different categories may differ in different iteration rounds, a dynamically adjustable entity threshold can be set to obtain a sample entity set with higher accuracy.

[0012] In an optional implementation, after updating the sample entity set corresponding to the previous iteration based on the first text representation and the second text representation to obtain the sample entity set corresponding to the current iteration, the method further includes: training the text representation model corresponding to the previous iteration based on the sample entity set corresponding to the current iteration to obtain the text representation model corresponding to the current iteration. In the above scheme, the updated sample entity set can be used to update the text representation model for the next round of sample entity set updates, thereby obtaining a more accurate sample entity set.

[0013] In an optional implementation, the loss function used to train the text representation model includes: a contrastive learning loss function, and / or, a masked language model loss function. In the above scheme, the text representation model can be updated using both the contrastive learning loss function and the masked language model loss function, thereby obtaining a sample entity set with higher accuracy.

[0014] In an optional implementation, the contrastive learning loss function is:

[0015]

[0016] Where N is the number of samples, k is the number of positive samples of the same class, and v i The text representation of sample i. For randomly selected negative samples, τ is the temperature coefficient, and I i≠j This indicates that the sample vector is not computed with itself. In the above scheme, contrastive learning loss can be introduced to enhance text representation, causing entities of the same category to cluster together and amplifying the differences between different categories, thus obtaining a text representation model with superior performance.

[0017] In an optional implementation, the masked language model loss function is:

[0018]

[0019] Where, p ij (x i ) represents the i-th word x i For the probability of the j-th label, y ij It is i words x i The true labels are given by M, where M represents the number of words in the sentence and L represents the number of labels. In the above scheme, a masked language model loss can be introduced to improve the text representation model's learning ability on the target data, thereby obtaining a text representation model with better performance.

[0020] In an optional implementation, obtaining the initial entity set corresponding to the current iteration includes: filtering the unlabeled data corresponding to the current iteration based on combinational logic rules to obtain a candidate entity set corresponding to the current iteration; and labeling the candidate entities in the candidate entity set corresponding to the current iteration based on the labeling rules corresponding to the current iteration to obtain the initial entity set corresponding to the current iteration. In the above scheme, combinational logic rules composed of simple rules are introduced to filter the unlabeled data, thereby improving the accuracy of boundary detection. Then, the labeling rules are used to label the candidate entities to obtain the initial entity set.

[0021] In an optional implementation, after training the entity recognition model corresponding to the previous iteration based on the sample entity set corresponding to the current iteration to obtain the entity recognition model corresponding to the current iteration, the method further includes: inputting the unlabeled data corresponding to the current iteration into the entity recognition model corresponding to the current iteration to obtain the entity output by the entity recognition model corresponding to the current iteration and the probability values ​​of each corresponding category; updating the labeling rules and the unlabeled data corresponding to the current iteration based on the probability values ​​to obtain the labeling rules and the unlabeled data corresponding to the next iteration. In the above scheme, the trained entity recognition model can be used to update the unlabeled data and labeling rules, thereby improving the performance of the entity recognition model obtained in the next training round.

[0022] In an optional implementation, after training the entity recognition model corresponding to the previous iteration based on the sample entity set corresponding to the current iteration to obtain the entity recognition model corresponding to the current iteration, the method further includes: performing model distillation on the trained entity recognition model to obtain a corresponding lightweight entity recognition model. In the above scheme, the trained entity recognition model can be used as a teacher model. Based on model distillation, the knowledge on the teacher model is transferred to another lightweight single model. The student model learns the generalization ability of the teacher model, thereby obtaining a lightweight entity recognition model that guarantees performance and has a simpler structure.

[0023] Secondly, embodiments of this application provide an entity recognition method, comprising: acquiring text to be recognized; inputting the text to be recognized into a pre-trained entity recognition model to obtain entities and corresponding entity categories output by the entity recognition model; wherein the entity recognition model is trained using the entity recognition model training method described in the first aspect. In the above scheme, a sample entity set is obtained by filtering the text representations corresponding to each initial entity in the initial entity set, and the entity recognition model is trained using the filtered sample entity set. Therefore, in embodiments of this application, the sample data used for training can be automatically obtained without manual annotation, thereby reducing the cost caused by manual annotation during the training process. In addition, the trained entity recognition model obtained in embodiments of this application is not limited by labels specific to a particular domain and can be applied to various application scenarios.

[0024] Thirdly, this application provides an entity recognition model training device, comprising: a first acquisition module, configured to acquire a sample entity set corresponding to the current iteration; wherein the sample entity set corresponding to the current iteration is obtained by updating the sample entity set corresponding to the previous iteration based on a first text representation and a second text representation, wherein the first text representation is the text representation of each initial entity in the initial entity set corresponding to the current iteration, and the second text representation is the text representation of each sample entity in the sample entity set corresponding to the previous iteration; and a first training module, configured to train the entity recognition model corresponding to the previous iteration based on the sample entity set corresponding to the current iteration, thereby obtaining the entity recognition model corresponding to the current iteration. In the above scheme, a sample entity set is obtained by filtering from the initial entity set through the text representations corresponding to each initial entity in the initial entity set, and the entity recognition model is trained using the filtered sample entity set. Therefore, in this application embodiment, the sample data used for training can be automatically obtained without manual annotation, thereby reducing the cost caused by manual annotation during the training process. In addition, the trained entity recognition model obtained in this application embodiment is not limited by labels specific to a particular domain and can be applied to various application scenarios.

[0025] In an optional implementation, the first acquisition module is specifically used to: acquire the initial entity set corresponding to the current iteration; input the initial entity set corresponding to the current iteration and the sample entity set corresponding to the previous iteration into the text representation model corresponding to the previous iteration, and obtain the first text representation corresponding to each initial entity in the initial entity set corresponding to the current iteration and the second text representation corresponding to each sample entity in the sample entity set corresponding to the previous iteration; update the sample entity set corresponding to the previous iteration according to the first text representation and the second text representation to obtain the sample entity set corresponding to the current iteration. In the above scheme, since the initial entity set is not necessarily all correctly labeled results, the text representation model can be used to filter out a sample entity set with higher accuracy from the initial entity set. Therefore, in this embodiment, the sample data used for training can be automatically obtained without manual labeling, thereby reducing the cost caused by manual labeling during training. In addition, the entity recognition model obtained after training in this embodiment is not limited by labels of a specific domain and can be applied to a variety of application scenarios.

[0026] In an optional implementation, the first acquisition module is further configured to: calculate the entity confidence score of an initial entity based on a first text representation of the initial entity and a second text representation of the sample entity subset corresponding to the initial entity in the current iteration and the sample entity set in the previous iteration, respectively; if the entity confidence score is greater than an entity threshold, add the initial entity to the sample entity set in the previous iteration. In the above scheme, by calculating the entity confidence score of the initial entity and comparing it with an entity threshold, the sample entity set can be updated to obtain a more accurate sample entity set.

[0027] In an optional implementation, the first acquisition module is further configured to: calculate multiple cosine similarities between the first text representation corresponding to the initial entity and the second text representations corresponding to each sample entity in the sample entity subset; and determine the maximum value among the multiple pre-similarity scores as the entity confidence score. In the above scheme, a more accurate sample entity set can be obtained by calculating the local score of each initial entity as the entity confidence score and filtering noisy data in the entity set based on the entity confidence score.

[0028] In an optional implementation, the first acquisition module is further configured to: calculate multiple cosine similarities between the first text representation corresponding to the initial entity and the second text representations corresponding to each sample entity in the sample entity subset; determine the maximum value among the multiple cosine similarities as the local score corresponding to the initial entity; calculate the global score between the first text representation corresponding to the initial entity and the second text representations corresponding to each sample entity in the sample entity subset; and calculate the entity confidence score based on the local score and the global score. In the above scheme, the entity confidence score can be obtained by calculating the local score and the global score of each initial entity, and filtering noisy data in the entity set based on the entity confidence score, thus obtaining a sample entity set with higher accuracy.

[0029] In an optional implementation, the first acquisition module is further configured to: calculate the average text representation of the second text representation corresponding to a subset of sample entities in the sample entity subset; calculate the cosine similarity between the average text representation and the first text representation corresponding to the initial entity, and obtain the global score. In the above scheme, since the local score is estimated based on a single instance in the sample entity subset, a global score estimated based on the entire sample entity set can be further calculated to obtain a sample entity set with higher accuracy.

[0030] In an optional implementation, the first acquisition module is further configured to: calculate the geometric mean of the local score and the global score to obtain the entity confidence score. In the above scheme, to ensure the exploration capability and reliability of new sample instances in the sample entity set, the entity confidence score of an initial entity can be obtained by calculating the geometric mean of the local score and the global score, thereby obtaining a sample entity set with higher accuracy.

[0031] In an optional implementation, the entity recognition model training device further includes: a first update module, used to update the entity threshold based on the entity confidence scores of the sample entities in the sample entity set corresponding to the current iteration. In the above scheme, since the entity thresholds for different categories may differ in different iterations, a dynamically adjustable entity threshold can be set to obtain a sample entity set with higher accuracy.

[0032] In an optional implementation, the entity recognition model training device further includes a second training module, used to train the text representation model corresponding to the previous iteration based on the sample entity set corresponding to the current iteration, to obtain the text representation model corresponding to the current iteration. In the above scheme, the updated sample entity set can be used to update the text representation model for the next round of sample entity set updates, thereby obtaining a sample entity set with higher accuracy.

[0033] In an optional implementation, the loss function used to train the text representation model includes: a contrastive learning loss function, and / or, a masked language model loss function. In the above scheme, the text representation model can be updated using both the contrastive learning loss function and the masked language model loss function, thereby obtaining a sample entity set with higher accuracy.

[0034] In an optional implementation, the contrastive learning loss function is:

[0035]

[0036] Where N is the number of samples, k is the number of positive samples of the same class, and v i The text representation of sample i. For randomly selected negative samples, τ is the temperature coefficient, and I i≠j This indicates that the sample vector is not computed with itself. In the above scheme, contrastive learning loss can be introduced to enhance text representation, causing entities of the same category to cluster together and amplifying the differences between different categories, thus obtaining a text representation model with superior performance.

[0037] In an optional implementation, the masked language model loss function is:

[0038]

[0039] Where, p ij (x i ) represents the i-th word x i For the probability of the j-th label, y ij It is i words x i The true labels are given by M, where M represents the number of words in the sentence and L represents the number of labels. In the above scheme, a masked language model loss can be introduced to improve the text representation model's learning ability on the target data, thereby obtaining a text representation model with better performance.

[0040] In an optional implementation, the first acquisition module is further configured to: filter the unlabeled data corresponding to the current iteration based on combinational logic rules to obtain a candidate entity set corresponding to the current iteration; and label the candidate entities in the candidate entity set corresponding to the current iteration based on the labeling rules corresponding to the current iteration to obtain an initial entity set corresponding to the current iteration. In the above scheme, combinational logic rules composed of simple rules are introduced to filter the unlabeled data, thereby improving the accuracy of boundary detection. Then, the labeling rules are used to label the candidate entities to obtain the initial entity set.

[0041] In an optional implementation, the entity recognition model training device further includes: a second recognition module, used to input the unlabeled data corresponding to the current iteration into the entity recognition model corresponding to the current iteration, to obtain the entity output by the entity recognition model corresponding to the current iteration and the probability values ​​of each corresponding category; and a second update module, used to update the labeling rules and the unlabeled data corresponding to the current iteration according to the probability values, to obtain the labeling rules and the unlabeled data corresponding to the next iteration. In the above scheme, the trained entity recognition model can be used to update the unlabeled data and labeling rules, thereby improving the performance of the entity recognition model trained in the next round.

[0042] In an optional embodiment, the entity recognition model training device further includes a distillation module for performing model distillation on the trained entity recognition model to obtain a corresponding lightweight entity recognition model. In the above scheme, the trained entity recognition model can be used as a teacher model. Based on model distillation, the knowledge from the teacher model is transferred to another lightweight single model. The student model learns the generalization ability of the teacher model, thereby obtaining a lightweight entity recognition model that guarantees performance and has a simpler structure.

[0043] Fourthly, embodiments of this application provide an entity recognition device, comprising: a second acquisition module for acquiring text to be recognized; and a first recognition module for inputting the text to be recognized into a pre-trained entity recognition model to obtain entities and corresponding entity categories output by the entity recognition model; wherein the entity recognition model is trained using the entity recognition model training method described in the first aspect. In the above scheme, a sample entity set is obtained by filtering the text representations corresponding to each initial entity in the initial entity set, and the entity recognition model is trained using the filtered sample entity set. Therefore, in embodiments of this application, the sample data used for training can be automatically obtained without manual annotation, thereby reducing the cost caused by manual annotation during the training process. Furthermore, the trained entity recognition model obtained in embodiments of this application is not limited by labels specific to a particular domain and can be applied to various application scenarios.

[0044] Fifthly, embodiments of this application provide a computer program product, including computer program instructions, which, when read and executed by a processor, perform the entity recognition model training method as described in the first aspect or the entity recognition method as described in the second aspect.

[0045] In a sixth aspect, embodiments of this application provide an electronic device, including: a processor, a memory, and a bus; the processor and the memory communicate with each other via the bus; the memory stores computer program instructions executable by the processor, and the processor can execute the entity recognition model training method as described in the first aspect or the entity recognition method as described in the second aspect by calling the computer program instructions.

[0046] In a seventh aspect, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a computer, cause the computer to perform the entity recognition model training method as described in the first aspect or the entity recognition method as described in the second aspect.

[0047] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, embodiments of this application are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 A flowchart illustrating an entity recognition model training method provided in this application embodiment;

[0050] Figure 2 A structural block diagram of an entity recognition model training device provided in an embodiment of this application;

[0051] Figure 3 A structural block diagram of an entity recognition device provided in an embodiment of this application;

[0052] Figure 4 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0053] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0054] Please refer to Figure 1 , Figure 1 A flowchart of an entity recognition model training method provided in this application embodiment, the entity recognition model training method may include the following steps:

[0055] Step S101: Obtain the sample entity set corresponding to the current iteration; wherein, the sample entity set corresponding to the current iteration is obtained by updating the sample entity set corresponding to the previous iteration according to the first text representation and the second text representation, the first text representation is the text representation of each initial entity in the initial entity set corresponding to the current iteration, and the second text representation is the text representation of each sample entity in the sample entity set corresponding to the previous iteration.

[0056] Step S102: Train the entity recognition model corresponding to the previous iteration based on the sample entity set corresponding to this iteration to obtain the entity recognition model corresponding to this iteration.

[0057] Specifically, the entity recognition model training method provided in this application embodiment is an iterative process, and each iteration can obtain a corresponding entity recognition model.

[0058] It is understood that the embodiments of this application do not impose specific limitations on the termination conditions of the iteration, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, iteration can be stopped when the number of iterations exceeds a threshold; or, iteration can be stopped when the iteration time exceeds a time threshold; or, iteration can be stopped when the performance of the entity recognition model meets the performance requirements, etc.

[0059] Furthermore, both the entity recognition model during the iteration process and the entity recognition model obtained after the iteration are used for entity recognition, and this application embodiment does not impose any specific limitations on this.

[0060] It should be noted that, for ease of description, only one round of iteration is described in this embodiment; it is understood that the iteration process of other rounds is similar to the iteration process described in the subsequent embodiments, and therefore will not be repeated.

[0061] In step S101 above, the electronic device can acquire the sample entity set corresponding to the current iteration. The sample entity set includes multiple sample entities and the labeling result corresponding to each sample entity; the labeling result represents the entity category to which each sample entity belongs, and the entity category includes person names, place names, organization names, proper nouns, etc.

[0062] It is understood that there are multiple ways for an electronic device to obtain the aforementioned sample entity set, and this application embodiment does not impose specific limitations on this. Those skilled in the art can make appropriate adjustments according to the actual situation. For example, the electronic device can receive a sample entity set sent by other devices; or, the electronic device can read a pre-stored sample entity set from the cloud or locally; or, the electronic device can generate its own sample entity set, etc.

[0063] Taking the sample entity set generated by the electronic device itself as an example, the sample entity set corresponding to the current iteration can be obtained by updating the sample entity set corresponding to the previous iteration based on the first text representation and the second text representation. The first text representation is the text representation of each initial entity in the initial entity set corresponding to the current iteration, while the second text representation is the text representation of each sample entity in the sample entity set corresponding to the previous iteration.

[0064] In step S102 above, the electronic device can train the entity recognition model corresponding to the previous iteration based on the sample entity set corresponding to the current iteration to obtain the entity recognition model corresponding to the current iteration. It is understood that the entity recognition model corresponding to the current iteration can be used for other purposes or can continue to be updated in the next iteration.

[0065] In the above scheme, a sample entity set is obtained by filtering the text representations corresponding to each initial entity in the initial entity set, and the entity recognition model is trained using the filtered sample entity set. Therefore, in this embodiment, the training sample data can be automatically obtained without manual annotation, thereby reducing the cost caused by manual annotation during the training process. Furthermore, the trained entity recognition model obtained in this embodiment is not limited by domain-specific labels and can be applied to various application scenarios.

[0066] Furthermore, based on the above embodiments, an entity recognition model provided by the embodiments of this application will be described below.

[0067] Specifically, entity recognition can be viewed as a span labeling problem, where each span is represented as a fixed-length embedding, and then the embeddings are used for label prediction.

[0068] In the entity recognition model provided in this application embodiment, given a span and its corresponding sentence, all words in the sentence can be initialized using a pre-trained BERT model; then, a Bi-LSTM and a self-attention layer are applied to obtain the context embedding of the sentence; the span embedding is then calculated by connecting two components: one is the content representation, which calculates the weighted average of all word embeddings in the span, and the other is the boundary representation, which connects the embedding representations of the start and end positions of the span; finally, a multilayer perceptron (MLP) is used to predict the label of the span.

[0069] For example, given a sentence x = [w1, w2, ..., w] containing n words... n A span Where b i ,e i These are the start and end index positions, respectively. Their content represents... Calculated as a weighted average of all word embeddings across the span, with boundary representation. Embedding of the start and end index positions.

[0070] c1,c2,...,c n =TokenRepr(w1,w2,...,w n )

[0071] u1,u2,...,u n =BiLSTM(c1,c2,...,c n )

[0072]

[0073]

[0074]

[0075] Where TokenRepr is the Bert initialization representation used, c1, c2, ..., c n BiLSTM is a bidirectional LSTM layer, u1, u2, ..., u, which is the data obtained after BERT initialization of the words in the sentence. n This represents the data obtained after processing the data obtained after BERT initialization using Bi-LSTM. SelfAttn is a self-attention layer, z i The content representation of the span and the boundary representation.

[0076] Next, a multilayer perceptron (MLP) is used to predict the labels for all words.

[0077] o i =softmax(MLP) span (z i ))

[0078] Among them, o i This is the prediction result for the span, while softmax is the probability distribution of the label obtained using the softmax function.

[0079] Furthermore, based on the above embodiments, the specific implementation method for generating the sample entity set is described below. Step S101 above may specifically include the following steps:

[0080] Step 1), obtain the initial entity set corresponding to this iteration.

[0081] Step 2) Input the initial entity set corresponding to this iteration and the sample entity set corresponding to the previous iteration into the text representation model corresponding to the previous iteration to obtain the first text representation of each initial entity in the initial entity set corresponding to this iteration and the second text representation of each sample entity in the sample entity set corresponding to the previous iteration.

[0082] Step 3) Update the sample entity set corresponding to the previous iteration based on the first text representation and the second text representation to obtain the sample entity set corresponding to the current iteration.

[0083] Specifically, in step 1) above, the electronic device can obtain the initial entity set corresponding to this iteration. The initial entity set includes multiple initial entities and the labeling result corresponding to each initial entity.

[0084] It is understood that, similar to obtaining the sample entity set corresponding to this iteration, there are multiple ways for an electronic device to obtain the aforementioned initial entity set. This application embodiment does not impose specific limitations on these methods, and those skilled in the art can make appropriate adjustments based on actual circumstances. For example, the electronic device can receive an initial entity set sent by other devices; or, the electronic device can read a pre-stored initial entity set from the cloud or locally; or, the electronic device can generate its own initial entity set, etc.

[0085] The specific implementation method for generating the initial entity set by the electronic device will be described in detail in subsequent embodiments, and will not be introduced here.

[0086] In step 2) above, the initial entity set corresponding to this iteration can be input into the text representation model corresponding to the previous iteration to obtain the first text representation of each initial entity in the initial entity set corresponding to this iteration. That is, based on the text representation model, each initial entity in the initial entity set can be represented by a vector, which is the first text representation mentioned above.

[0087] Similarly, the sample entity set corresponding to the previous iteration can be input into the text representation model corresponding to the previous iteration to obtain the second text representation of each sample entity in the sample entity set corresponding to the previous iteration. In other words, based on the text representation model, each sample entity in the sample entity set can be represented by a vector, which is the aforementioned second text representation.

[0088] In this embodiment, the text representation model can be implemented using the pre-trained BERT model. As one implementation, the output of the BERT model can be directly used as the first or second text representation; as another implementation, the average value of the first N layers of BERT embeddings can be used as the first or second text representation, where N is a natural number.

[0089] Furthermore, as one implementation method, step 2) above can be implemented using a pre-trained BERT model; that is, the text recognition model corresponding to the previous iteration and the text recognition model corresponding to the current iteration can be the same. As another implementation method, step 2) above can also be implemented using a real-time updated BERT model; that is, the text recognition model corresponding to the previous iteration and the text recognition model corresponding to the current iteration can be different.

[0090] The specific implementation methods for updating the text recognition model will be described in detail in subsequent embodiments, and will not be introduced here.

[0091] In step 3) above, the sample entity set corresponding to the previous iteration can be updated according to the first text representation and the second text representation, so as to obtain the sample entity set corresponding to the current iteration.

[0092] In the above scheme, since the initial entity set may not all contain correctly labeled results, a text representation model can be used to filter out a sample entity set with higher accuracy from the initial entity set. Therefore, in this embodiment, the sample data used for training can be obtained automatically without manual annotation, thereby reducing the cost caused by manual annotation during training. Furthermore, the trained entity recognition model obtained in this embodiment is not limited by domain-specific labels and can be applied to various application scenarios.

[0093] Furthermore, based on the above embodiments, the step of updating the sample entity set corresponding to the previous iteration according to the first text representation and the second text representation to obtain the sample entity set corresponding to the current iteration may specifically include the following steps:

[0094] Step 1) For an initial entity in the initial entity set corresponding to this iteration and a subset of sample entities in the sample entity set corresponding to the same category as the initial entity in the previous iteration, calculate the entity confidence corresponding to the initial entity based on the first text representation corresponding to the initial entity and the second text representation corresponding to the subset of sample entities.

[0095] Step 2): If the entity confidence is greater than the entity threshold, add the initial entity to the sample entity set corresponding to the previous iteration.

[0096] Specifically, the initial entity set includes multiple initial entities, and each initial entity can perform steps 1)-2) above.

[0097] Therefore, for each initial entity in the initial entity set, its entity confidence score can be calculated. If the entity confidence score of an initial entity is greater than the entity threshold, the initial entity is added to the sample entity set corresponding to the previous iteration. After calculating the entity confidence score of each initial entity, the update of the sample entity set for this round is completed; compared with the sample entity set corresponding to the previous iteration, the sample entity set corresponding to this iteration has added some initial entities.

[0098] For an initial entity, in step 1 above, based on the labeling results corresponding to the initial entity, a subset of sample entities of the same category as the initial entity can be selected from the sample entity set. This subset of sample entities can form a sample entity subset. Based on the first text representation corresponding to the initial entity and the second text representation corresponding to the sample entity subset, the entity confidence score corresponding to the initial entity can be calculated.

[0099] The specific implementation method for calculating the entity confidence level corresponding to the initial entity will be described in detail in subsequent embodiments, and will not be introduced here.

[0100] In step 2) above, as one implementation, the entity threshold can be predetermined and fixed; as another implementation, the entity threshold can also be updated in real time; the embodiments of this application do not impose specific limitations on this, and those skilled in the art can make appropriate adjustments according to the actual situation.

[0101] For initial entities whose entity confidence level is not greater than the entity threshold, no processing is performed on these initial entities in this embodiment.

[0102] In the above scheme, the entity confidence of the initial entity can be calculated and compared with the entity threshold. By updating the sample entity set, a more accurate sample entity set can be obtained.

[0103] Furthermore, based on the above embodiments, a specific implementation method for calculating the entity confidence score corresponding to the initial entity is described below. The steps for calculating the entity confidence score corresponding to the initial entity based on the first text representation corresponding to the initial entity and the second text representation corresponding to the subset of sample entities may specifically include the following steps:

[0104] Step 1) Calculate multiple cosine similarities between the first text representation corresponding to the initial entity and the second text representations corresponding to each sample entity in the sample entity subset.

[0105] Step 2) Determine the maximum value among multiple pre-similarity scores as the entity confidence score.

[0106] Specifically, in step 1) above, based on the initial entity eq The corresponding first text representation And a sample entity e in the subset of sample entities i (where i represents the i-th category) corresponding to the second text representation The cosine similarity between the initial entity and a sample entity can be calculated.

[0107] Therefore, the cosine similarity of each sample entity in the sample entity subset can be calculated.

[0108] It should be noted that the embodiments of this application do not specifically limit the specific implementation of calculating the cosine similarity between the first text representation and the second text representation, and those skilled in the art can make appropriate adjustments in combination with the prior art.

[0109] In step 2) above, the maximum value of the cosine similarity for each sample entity in the sample entity subset can be determined as the entity confidence score for that initial entity. Right now:

[0110]

[0111] In the above scheme, the local score of each initial entity can be calculated as the entity confidence score, and the noisy data in the entity set can be filtered based on the entity confidence score to obtain a sample entity set with higher accuracy.

[0112] Furthermore, based on the above embodiments, another specific implementation method for calculating the entity confidence score corresponding to the initial entity is described below. The steps for calculating the entity confidence score corresponding to the initial entity based on the first text representation corresponding to the initial entity and the second text representation corresponding to the subset of sample entities may specifically include the following steps:

[0113] Step 1) Calculate multiple cosine similarities between the first text representation corresponding to the initial entity and the second text representations corresponding to each sample entity in the sample entity subset.

[0114] Step 2) Determine the maximum value among multiple cosine similarities as the local score corresponding to the initial entity.

[0115] Step 3) Calculate the global score between the first text representation corresponding to the initial entity and the second text representation corresponding to each sample entity in the sample entity subset.

[0116] Step 4) Calculate the entity confidence score based on the local score and the global score.

[0117] Specifically, the specific implementation methods of steps 1)-2) above are similar to those of steps 1)-2) in the above embodiments, and will not be repeated here. The difference is that, since steps 1)-2) above are calculated based on a single sample entity, therefore, in this embodiment of the application, This is the local score corresponding to the initial entity.

[0118] In steps 3)-4) above, the global score between the initial entity and the sample entity subset can be further calculated. The entity confidence score corresponding to the initial entity is determined based on both the local score and the global score. i .

[0119] In the above scheme, the local score and global score of each initial entity can be calculated as the entity confidence score, and the noisy data in the entity set can be filtered based on the entity confidence score to obtain a sample entity set with higher accuracy.

[0120] Furthermore, based on the above embodiments, the specific implementation method for calculating the global score is described below. The step of calculating the global score between the first text representation corresponding to the initial entity and the second text representations corresponding to each sample entity in the subset of sample entities may specifically include the following steps:

[0121] Step 1) Calculate the average value of the second text representation of the sample entities in the subset of sample entities.

[0122] Step 2) Calculate the cosine similarity between the average text representation and the first text representation corresponding to the initial entity to obtain the global score.

[0123] Specifically, in step 1) above, a subset of sample entities can be randomly sampled to obtain a small set E consisting of some sample entities. S Then, calculate the smaller set E. S The average value of the second text representation corresponding to the sample entity in the text.

[0124] In step 2) above, the cosine similarity between the average value of the above text representation and the first text representation corresponding to the initial entity can be further calculated. At this point, the global score This refers to the cosine similarity mentioned above.

[0125] As another implementation method, steps 1)-2) above can be executed multiple times, that is, N random samplings are performed on the subset of sample entities. In this case, the global score... It can be determined in the following way:

[0126]

[0127] In the above scheme, since the local score is estimated based on a single instance in the sample entity set, a global score estimated based on the entire sample entity set can be further calculated to obtain a sample entity set with higher accuracy.

[0128] Furthermore, based on the above embodiments, the specific implementation method for calculating entity confidence is described below. The steps for calculating entity confidence based on local and global scores may specifically include the following steps:

[0129] The entity confidence score is obtained by calculating the geometric mean of the local and global scores.

[0130] Specifically, the entity confidence score can be calculated based on the following formula. i :

[0131]

[0132] In the above scheme, in order to ensure the exploration capability and reliability of new sample instances in the sample entity set, the entity confidence of an initial entity can be obtained by calculating the geometric mean of the local score and the global score, so as to obtain a sample entity set with higher accuracy.

[0133] Furthermore, based on the above embodiments, the entity threshold can also be updated in this application embodiment. That is, after the step of adding the initial entity to the sample entity set corresponding to the previous iteration, the entity recognition model training method provided in this application embodiment can further include the following steps:

[0134] The entity threshold is updated based on the entity confidence of the sample entities in the sample entity set corresponding to this iteration.

[0135] Specifically, as one implementation method, this applies to a sample entity e in the sample entity set. p The confidence score of an entity relative to other entities in the sample entity set can be calculated and repeated randomly T times, using the minimum value as the entity threshold. For the i-th category, the entity threshold can be calculated based on the following formula:

[0136]

[0137] Where τ2∈[0,1] is the temperature coefficient controlling the threshold.

[0138] In the above scheme, since the threshold values ​​for different categories of entities may differ in different iteration rounds, a dynamically adjustable entity threshold can be set to obtain a sample entity set with higher accuracy.

[0139] Furthermore, based on the above embodiments, the specific implementation method for updating the text recognition model is described below. The entity recognition model training method provided in this application embodiment may further include the following steps:

[0140] The text representation model corresponding to the previous iteration is trained based on the sample entity set corresponding to the current iteration to obtain the text representation model corresponding to the current iteration.

[0141] In the above scheme, the updated sample entity set can be used to update the text representation model for the next round of sample entity set updates, thereby obtaining a more accurate sample entity set.

[0142] Furthermore, based on the above embodiments, the loss function used to train the text representation model includes: a contrastive learning loss function, and / or, a masked language model loss function.

[0143] Specifically, there are three types of loss functions that can be used in the process of training a text representation model: the first is to use the contrastive learning loss function; the second is to use the masked language model loss function; and the third is to use a loss function composed of the contrastive learning loss function and the masked language model loss function.

[0144] Among them, the contrastive learning loss function L KGCL It can be represented as:

[0145]

[0146] Where N is the number of samples, k is the number of positive samples of the same class, and v i The text representation of sample u. For randomly selected negative samples, τ is the temperature coefficient, and I i≠j This indicates that the sample vector is not used in calculations with itself.

[0147] Masked language model loss function L MLM It can be represented as:

[0148]

[0149] Where, p ij (x i ) represents the i-th word x i For the probability of the j-th label, y ij It is i words x iThe true labels are M, which represents the number of words in the sentence, and L, which represents the number of labels.

[0150] The loss function LOSS, which is composed of the contrastive learning loss function and the masked language model loss function, can be expressed as:

[0151] LOSS=λ·L KGCL +(1-λ)·L MLM ;

[0152] Here, λ is the scaling parameter of the contrastive learning loss function, which can be determined in advance.

[0153] In the above scheme, the text representation model can be updated using a contrastive learning loss function and a masked language model loss function, resulting in a more accurate sample entity set. Specifically, introducing contrastive learning loss enhances text representation, causing entities of the same category to cluster together and amplifying the differences between different categories; introducing masked language model loss improves the text representation model's learning ability on target data. Therefore, a superior text representation model can be obtained.

[0154] Furthermore, based on the above embodiments, the specific implementation method for generating the initial entity set is described below. The steps for obtaining the initial entity set corresponding to the current iteration can specifically include the following steps:

[0155] Step 1) Filter the unlabeled data corresponding to this iteration based on combinational logic rules to obtain the candidate entity set corresponding to this iteration.

[0156] Step 2) Mark the candidate entities in the candidate entity set corresponding to the current iteration based on the marking rules corresponding to the current iteration to obtain the initial entity set corresponding to the current iteration.

[0157] Specifically, logical rules can include: words, preceding context, following context, parts of speech, and dependency relationships. Taking the unlabeled data "He moved to the United States in 1916" as an example, given a target candidate entity "United States", the filtering logical rules are as follows: For the word: United States; For the preceding context: moved to the (when the number of words in the preceding context is set to 3); For the following context: in 1916 (when the number of words in the following context is set to 3); For the part of speech: PROPN PROPN (part of speech is proper noun); For the dependency relationship: to-pobj (the relationship with to is a prepositional phrase).

[0158] Combinatorial logic rules can be rules obtained by combining the above five logic rules. In one implementation, a combinational logic rule can be obtained by combining any two of the above five logic rules; in another implementation, a combinational logic rule can be obtained by combining at least one of the above five logic rules. For example, a combinational logic rule can include the above five logic rules themselves, multiple rules obtained by combining any two of the above five logic rules, or a single rule obtained by combining the above five logic rules, etc.

[0159] In step 1) above, a set of candidate entities is obtained by filtering from the unlabeled data based on the above combinational logic rules.

[0160] In step 2) above, the candidate entity set obtained from the above filtering can be labeled using labeling rules to obtain an initial entity set. The labeling rules include multiple entities and their corresponding categories.

[0161] In one implementation, the marking rules can be predetermined and unchanging; in another implementation, the marking rules can be updated in real time. The specific implementation methods for updating the marking rules will be described in detail in subsequent embodiments and will not be described here.

[0162] In the above scheme, a combinational logic rule composed of simple rules is introduced to filter unlabeled data, thereby improving the accuracy of boundary detection. Then, the candidate entities are labeled using the labeling rule to obtain the initial entity set.

[0163] Furthermore, based on the above embodiments, the embodiments of this application can also update the unlabeled data and labeling rules. That is, after training the entity recognition model corresponding to the previous iteration based on the sample entity set corresponding to the current iteration to obtain the entity recognition model corresponding to the current iteration, the entity recognition model training method provided by the embodiments of this application can further include the following steps:

[0164] Step 1) Input the unlabeled data corresponding to this iteration into the entity recognition model corresponding to this iteration to obtain the entity output by the entity recognition model corresponding to this iteration and the probability values ​​of each category.

[0165] Step 2) Update the labeling rules and unlabeled data corresponding to the current iteration based on the probability values ​​to obtain the labeling rules and unlabeled data corresponding to the next iteration.

[0166] Specifically, in step 1) above, by inputting unlabeled data into the trained entity recognition model, multiple entities and the probability values ​​of each entity belonging to each category can be obtained.

[0167] In step 2) above, for each category, the top 70% (this embodiment does not limit the size of this value) of the span with the highest confidence can be sorted and filtered as weak labels for calculating the rule score, and then new rules are selected from the candidate rules based on their confidence scores.

[0168] The confidence level F(r) of rule r can be calculated based on the following formula:

[0169]

[0170] Among them, F i N represents the number of spans predicted using category i and matched with rule r. i This represents the total number of spans that match rule r. Represents the accuracy of the rule, log2(F) i ) indicates the ability of the rule to cover a wider range.

[0171] As one implementation, in each iteration, the top K rules for each category can be selected as the rules to be learned in the next iteration; as another implementation, in each iteration, the top K rules for each category can be selected as the rules to be learned in the next iteration, while K is increased by η in each iteration.

[0172] In the above scheme, the trained entity recognition model can be used to update the unlabeled data and labeling rules, thereby improving the performance of the entity recognition model trained in the next round.

[0173] Furthermore, based on the above embodiments, after obtaining the trained entity recognition model, as one implementation method, the entity recognition model can be directly used for entity recognition; as another implementation method, the entity recognition model can be distilled, and the lightweight entity recognition model obtained after model distillation can be used for entity recognition.

[0174] Following step S102 above, the entity recognition model training method provided in this application embodiment may further include the following steps:

[0175] The trained entity recognition model is distilled to obtain the corresponding lightweight entity recognition model.

[0176] Specifically, the BERT model can be divided into an embedding layer, a transformer layer, and a prediction layer, each representing a different form of knowledge. As one implementation method, different learning approaches can be applied to each layer, meaning each layer has a different objective function. Ultimately, the knowledge from each layer is integrated to obtain the student model, where the overall objective function of the model is:

[0177] Lmodel =∑ i∈{e,t,p} λ i L i (S i ,T i );

[0178] Where e, t, and p represent the embedding layer, transformer layer, and prediction layer, respectively, and λ i L represents the weight of the corresponding layer. i S represents the distillation loss of the corresponding layer. i ,T i These represent the student model and teacher model for the corresponding layer, respectively.

[0179] The following section will introduce the distillation methods for different layers.

[0180] For the embedding layer, the mean squared error formula can be used as the objective function for embedding layer distillation. The objective function for embedding layer distillation is L. e It can be determined based on the following formula:

[0181] L e =MSE(E S W e E T );

[0182] Among them, E S E T W represents the vector matrices of the student model and the teacher model, respectively. e This is a transformation matrix used to map the student vector matrix and the teacher vector matrix to the same dimension.

[0183] For the transformer layer, since it contributes the most to the model and contains the most information, the learning ability of the student model at this layer is also the most important. Therefore, in order to maximize the preservation of information in the small model, the transformer layer can be treated with a distillation method based on EMD.

[0184] The objective function of transformer layer distillation is L t It can be determined based on the following formula:

[0185] L t =L attn +L ffn ;

[0186]

[0187]

[0188] Among them, L attn The objective function is determined based on the multi-head attention mechanism. f is the EMD distance between attention matrices. ij Let L be the matrix of the teacher's i-th layer transformer and the matrix of the student's j-th layer transformer. ffn The objective function is determined based on a fully connected feed-forward neural network (FFN). denoted as EMD distance between FFN matrices.

[0189] For the prediction layer, distillation can be performed using a linear combination of soft and hard targets.

[0190] The objective function of prediction layer distillation is L p It can be determined based on the following formula:

[0191]

[0192] Among them, L ph L represents the cross-entropy loss between the student model and the true label. ps This represents the cross-entropy loss between the student model and the teacher model, where α represents the weight of each loss. This represents temperature and is used in the softmax function. When it approaches 0, the softmax output converges to a one-hot vector, and when it approaches infinity, the softmax output becomes even softer.

[0193] In the above scheme, the trained entity recognition model can be used as the teacher model. Based on model distillation, the knowledge on the teacher model can be transferred to another lightweight single model. The student model learns the generalization ability of the teacher model, thereby obtaining a lightweight entity recognition model that guarantees performance and has a simpler structure.

[0194] This application also provides a flowchart of an entity recognition method, which may include the following steps:

[0195] Step 1), obtain the text to be recognized.

[0196] Step 2) Input the text to be recognized into the pre-trained entity recognition model to obtain the entity and the corresponding entity category output by the entity recognition model; wherein, the entity recognition model is trained using the entity recognition model training method.

[0197] In the above scheme, a sample entity set is obtained by filtering the text representations corresponding to each initial entity in the initial entity set, and the entity recognition model is trained using the filtered sample entity set. Therefore, in this embodiment, the training sample data can be automatically obtained without manual annotation, thereby reducing the cost caused by manual annotation during the training process. Furthermore, the trained entity recognition model obtained in this embodiment is not limited by domain-specific labels and can be applied to various application scenarios.

[0198] Please refer to Figure 2 , Figure 2 This is a structural block diagram of an entity recognition model training device provided in an embodiment of this application. The entity recognition model training device 200 includes: a first acquisition module 201, used to acquire a sample entity set corresponding to the current iteration; wherein the sample entity set corresponding to the current iteration is obtained by updating the sample entity set corresponding to the previous iteration according to a first text representation and a second text representation, wherein the first text representation is the text representation of each initial entity in the initial entity set corresponding to the current iteration, and the second text representation is the text representation of each sample entity in the sample entity set corresponding to the previous iteration; and a first training module 202, used to train the entity recognition model corresponding to the previous iteration according to the sample entity set corresponding to the current iteration, to obtain the entity recognition model corresponding to the current iteration.

[0199] In the above scheme, a sample entity set is obtained by filtering the text representations corresponding to each initial entity in the initial entity set, and the entity recognition model is trained using the filtered sample entity set. Therefore, in this embodiment, the training sample data can be automatically obtained without manual annotation, thereby reducing the cost caused by manual annotation during the training process. Furthermore, the trained entity recognition model obtained in this embodiment is not limited by domain-specific labels and can be applied to various application scenarios.

[0200] Furthermore, the first acquisition module 201 is specifically used for: acquiring the initial entity set corresponding to the current iteration; inputting the initial entity set corresponding to the current iteration and the sample entity set corresponding to the previous iteration into the text representation model corresponding to the previous iteration, respectively obtaining the first text representation corresponding to each initial entity in the initial entity set corresponding to the current iteration and the second text representation corresponding to each sample entity in the sample entity set corresponding to the previous iteration; updating the sample entity set corresponding to the previous iteration according to the first text representation and the second text representation to obtain the sample entity set corresponding to the current iteration.

[0201] In the above scheme, since the initial entity set may not all contain correctly labeled results, a text representation model can be used to filter out a sample entity set with higher accuracy from the initial entity set. Therefore, in this embodiment, the sample data used for training can be obtained automatically without manual annotation, thereby reducing the cost caused by manual annotation during training. Furthermore, the trained entity recognition model obtained in this embodiment is not limited by domain-specific labels and can be applied to various application scenarios.

[0202] Furthermore, the first acquisition module 201 is also configured to: calculate the entity confidence of an initial entity based on a first text representation of the initial entity and a second text representation of the sample entity subset corresponding to the sample entity in the previous iteration, for an initial entity in the initial entity set corresponding to the current iteration and a sample entity subset in the sample entity set corresponding to the previous iteration; if the entity confidence is greater than the entity threshold, add the initial entity to the sample entity set corresponding to the previous iteration.

[0203] In the above scheme, the entity confidence of the initial entity can be calculated and compared with the entity threshold. By updating the sample entity set, a more accurate sample entity set can be obtained.

[0204] Furthermore, the first acquisition module 201 is also used to: calculate multiple cosine similarities between the first text representation corresponding to the initial entity and the second text representations corresponding to each sample entity in the sample entity subset; and determine the maximum value among the multiple pre-similarity values ​​as the entity confidence.

[0205] In the above scheme, the local score of each initial entity can be calculated as the entity confidence score, and the noisy data in the entity set can be filtered based on the entity confidence score to obtain a sample entity set with higher accuracy.

[0206] Furthermore, the first acquisition module 201 is also configured to: calculate multiple cosine similarities between the first text representation corresponding to the initial entity and the second text representations corresponding to each sample entity in the sample entity subset; determine the maximum value among the multiple cosine similarities as the local score corresponding to the initial entity; calculate the global score between the first text representation corresponding to the initial entity and the second text representations corresponding to each sample entity in the sample entity subset; and calculate the entity confidence based on the local score and the global score.

[0207] In the above scheme, the local score and global score of each initial entity can be calculated as the entity confidence score, and the noisy data in the entity set can be filtered based on the entity confidence score to obtain a sample entity set with higher accuracy.

[0208] Furthermore, the first acquisition module 201 is also used to: calculate the average text representation of the second text representation corresponding to some sample entities in the sample entity subset; calculate the cosine similarity between the average text representation and the first text representation corresponding to the initial entity, and obtain the global score.

[0209] In the above scheme, since the local score is estimated based on a single instance in the sample entity set, a global score estimated based on the entire sample entity set can be further calculated to obtain a sample entity set with higher accuracy.

[0210] Furthermore, the first acquisition module 201 is also used to: calculate the geometric mean of the local score and the global score to obtain the entity confidence level.

[0211] In the above scheme, in order to ensure the exploration capability and reliability of new sample instances in the sample entity set, the entity confidence of an initial entity can be obtained by calculating the geometric mean of the local score and the global score, so as to obtain a sample entity set with higher accuracy.

[0212] Furthermore, the entity recognition model training device 200 also includes: a first update module, used to update the entity threshold according to the entity confidence of the sample entities in the sample entity set corresponding to the current iteration.

[0213] In the above scheme, since the threshold values ​​for different categories of entities may differ in different iteration rounds, a dynamically adjustable entity threshold can be set to obtain a sample entity set with higher accuracy.

[0214] Furthermore, the entity recognition model training device 200 further includes: a second training module, used to train the text representation model corresponding to the previous iteration based on the sample entity set corresponding to the current iteration, so as to obtain the text representation model corresponding to the current iteration.

[0215] In the above scheme, the updated sample entity set can be used to update the text representation model for the next round of sample entity set updates, thereby obtaining a more accurate sample entity set.

[0216] Furthermore, the loss function used to train the text representation model includes: a contrastive learning loss function, and / or, a masked language model loss function.

[0217] In the above scheme, the text representation model can be updated by comparing the learning loss function and the masked language model loss function, thereby obtaining a sample entity set with higher accuracy.

[0218] Furthermore, the contrastive learning loss function is:

[0219]

[0220] Where N is the number of samples, k is the number of positive samples of the same class, and v i The text representation of sample i. For randomly selected negative samples, τ is the temperature coefficient, and I i≠j This indicates that the sample vector is not used in calculations with itself.

[0221] In the above scheme, contrastive learning loss can be introduced to enhance text representation, so that entities of the same category tend to cluster together and the differences between different categories are amplified, thereby obtaining a text representation model with better performance.

[0222] Furthermore, the loss function of the masked language model is:

[0223]

[0224] Where, p ij (x i ) represents the i-th word x i For the probability of the j-th label, y ij It is i words x i The true labels are M, which represents the number of words in the sentence, and L, which represents the number of labels.

[0225] In the above scheme, a masked language model loss can be introduced to improve the learning ability of the text representation model on the target data, thereby obtaining a text representation model with better performance.

[0226] Furthermore, the first acquisition module 201 is also used to: filter the unlabeled data corresponding to the current iteration based on combinational logic rules to obtain the candidate entity set corresponding to the current iteration; and label the candidate entities in the candidate entity set corresponding to the current iteration based on the labeling rules corresponding to the current iteration to obtain the initial entity set corresponding to the current iteration.

[0227] In the above scheme, a combinational logic rule composed of simple rules is introduced to filter unlabeled data, thereby improving the accuracy of boundary detection. Then, the candidate entities are labeled using the labeling rule to obtain the initial entity set.

[0228] Furthermore, the entity recognition model training device 200 further includes: a second recognition module, used to input the unlabeled data corresponding to the current iteration into the entity recognition model corresponding to the current iteration, to obtain the entity output by the entity recognition model corresponding to the current iteration and the probability values ​​of each corresponding category; and a second update module, used to update the labeling rules and the unlabeled data corresponding to the current iteration according to the probability values, to obtain the labeling rules and the unlabeled data corresponding to the next iteration.

[0229] In the above scheme, the trained entity recognition model can be used to update the unlabeled data and labeling rules, thereby improving the performance of the entity recognition model trained in the next round.

[0230] Furthermore, the entity recognition model training device 200 also includes a distillation module, used to perform model distillation on the trained entity recognition model to obtain a corresponding lightweight entity recognition model.

[0231] In the above scheme, the trained entity recognition model can be used as the teacher model. Based on model distillation, the knowledge on the teacher model can be transferred to another lightweight single model. The student model learns the generalization ability of the teacher model, thereby obtaining a lightweight entity recognition model that guarantees performance and has a simpler structure.

[0232] Please refer to Figure 3 , Figure 3 The present application provides a structural block diagram of an entity recognition device 300, which includes: a second acquisition module 301 for acquiring text to be recognized; and a first recognition module 301 for inputting the text to be recognized into a pre-trained entity recognition model to obtain the entity and corresponding entity category output by the entity recognition model; wherein the entity recognition model is trained using the entity recognition model training method described in the above embodiment.

[0233] In the above scheme, a sample entity set is obtained by filtering the text representations corresponding to each initial entity in the initial entity set, and the entity recognition model is trained using the filtered sample entity set. Therefore, in this embodiment, the training sample data can be automatically obtained without manual annotation, thereby reducing the cost caused by manual annotation during the training process. Furthermore, the trained entity recognition model obtained in this embodiment is not limited by domain-specific labels and can be applied to various application scenarios.

[0234] Please refer to Figure 4 , Figure 4This application provides a structural block diagram of an electronic device 400, which includes at least one processor 401, at least one communication interface 402, at least one memory 403, and at least one communication bus 404. The communication bus 404 enables direct communication between these components, the communication interface 402 facilitates signaling or data communication with other node devices, and the memory 403 stores machine-readable instructions executable by the processor 401. When the electronic device 400 is running, the processor 401 communicates with the memory 403 via the communication bus 404. When the machine-readable instructions are invoked by the processor 401, the aforementioned entity recognition model training method or entity recognition method is executed.

[0235] For example, the processor 401 in this embodiment of the application can read a computer program from the memory 403 via the communication bus 404 and execute the computer program to implement the following method: Step S101: Obtain the sample entity set corresponding to the current iteration; wherein, the sample entity set corresponding to the current iteration is obtained by updating the sample entity set corresponding to the previous iteration according to the first text representation and the second text representation, the first text representation being the text representation of each initial entity in the initial entity set corresponding to the current iteration, and the second text representation being the text representation of each sample entity in the sample entity set corresponding to the previous iteration. Step S102: Train the entity recognition model corresponding to the previous iteration according to the sample entity set corresponding to the current iteration to obtain the entity recognition model corresponding to the current iteration.

[0236] The processor 401 comprises one or more, and can be an integrated circuit chip with signal processing capabilities. The processor 401 can be a general-purpose processor, including a Central Processing Unit (CPU), a Microcontroller Unit (MCU), a Network Processor (NP), or other conventional processors; it can also be a special-purpose processor, including a Neural-network Processing Unit (NPU), a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. Furthermore, when there are multiple processors 401, some can be general-purpose processors, and others can be special-purpose processors.

[0237] The memory 403 includes one or more, which may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0238] Understandable. Figure 4 The structure shown is for illustrative purposes only; the electronic device 400 may also include components that are more advanced than those shown. Figure 4 The more or fewer components shown, or having the same Figure 4 The different configurations shown. Figure 4The components shown can be implemented using hardware, software, or a combination thereof. In the embodiments of this application, electronic device 400 can be, but is not limited to, physical devices such as desktop computers, laptops, smartphones, smart wearable devices, and in-vehicle devices, or virtual devices such as virtual machines. Furthermore, electronic device 400 is not necessarily a single device; it can be a combination of multiple devices, such as a server cluster, etc.

[0239] This application also provides a computer program product, including a computer program stored on a computer-readable storage medium. The computer program includes computer program instructions. When the computer program instructions are executed by a computer, the computer can perform the steps of the entity recognition model training method or entity recognition method described in the above embodiments. For example, it includes: obtaining the sample entity set corresponding to the current iteration; wherein the sample entity set corresponding to the current iteration is obtained by updating the sample entity set corresponding to the previous iteration according to a first text representation and a second text representation, wherein the first text representation is the text representation of each initial entity in the initial entity set corresponding to the current iteration, and the second text representation is the text representation of each sample entity in the sample entity set corresponding to the previous iteration; training the entity recognition model corresponding to the previous iteration according to the sample entity set corresponding to the current iteration to obtain the entity recognition model corresponding to the current iteration.

[0240] This application also provides a computer-readable storage medium that stores computer program instructions. When the computer program instructions are executed by a computer, the computer performs the entity recognition model training method or entity recognition method described in the foregoing method embodiments.

[0241] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0242] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0243] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0244] It should be noted that if the function is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0245] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0246] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for training an entity recognition model, characterized in that, include: Obtain the sample entity set corresponding to the current iteration; wherein, the sample entity set corresponding to the current iteration is obtained by updating the sample entity set corresponding to the previous iteration according to the first text representation and the second text representation, wherein the first text representation is the text representation of each initial entity in the initial entity set corresponding to the current iteration, and the second text representation is the text representation of each sample entity in the sample entity set corresponding to the previous iteration; The entity recognition model corresponding to the previous iteration is trained based on the sample entity set corresponding to the current iteration to obtain the entity recognition model corresponding to the current iteration. The step of obtaining the sample entity set corresponding to the current iteration includes: obtaining the initial entity set corresponding to the current iteration; inputting the initial entity set corresponding to the current iteration and the sample entity set corresponding to the previous iteration into the text representation model corresponding to the previous iteration to obtain the first text representation corresponding to each initial entity in the initial entity set corresponding to the current iteration and the second text representation corresponding to each sample entity in the sample entity set corresponding to the previous iteration; updating the sample entity set corresponding to the previous iteration according to the first text representation and the second text representation to obtain the sample entity set corresponding to the current iteration. The step of updating the sample entity set corresponding to the previous iteration based on the first text representation and the second text representation to obtain the sample entity set corresponding to the current iteration includes: for an initial entity in the initial entity set corresponding to the current iteration and a subset of sample entities in the sample entity set corresponding to the previous iteration that have the same category as the initial entity, calculating the entity confidence of the initial entity based on the first text representation of the initial entity and the second text representation of the sample entity subset; if the entity confidence is greater than an entity threshold, adding the initial entity to the sample entity set corresponding to the previous iteration; The step of calculating the entity confidence score based on the first text representation corresponding to the initial entity and the second text representation corresponding to the subset of sample entities includes: calculating multiple cosine similarities between the first text representation corresponding to the initial entity and the second text representations corresponding to each sample entity in the subset of sample entities; determining the maximum value among the multiple cosine similarities as the local score corresponding to the initial entity; calculating the global score between the first text representation corresponding to the initial entity and the second text representations corresponding to each sample entity in the subset of sample entities; and calculating the entity confidence score based on the local score and the global score. The step of calculating the global score between the first text representation corresponding to the initial entity and the second text representations corresponding to each sample entity in the sample entity subset includes: calculating the average text representation of the second text representations corresponding to some sample entities in the sample entity subset; calculating the cosine similarity between the average text representation and the first text representation corresponding to the initial entity to obtain the global score.

2. The entity recognition model training method according to claim 1, characterized in that, Obtaining the initial entity set corresponding to the current iteration includes: The candidate entity set corresponding to this iteration is obtained by filtering from the unlabeled data corresponding to this iteration based on combinational logic rules; wherein, the combinational logic rules refer to a combination of multiple logic rules; Based on the marking rules corresponding to this iteration, the candidate entities in the candidate entity set corresponding to this iteration are marked to obtain the initial entity set corresponding to this iteration.

3. The entity recognition model training method according to claim 2, characterized in that, After training the entity recognition model corresponding to the previous iteration based on the sample entity set corresponding to the current iteration to obtain the entity recognition model corresponding to the current iteration, the method further includes: Input the unlabeled data corresponding to the current iteration into the entity recognition model corresponding to the current iteration to obtain the entity output by the entity recognition model corresponding to the current iteration and the probability values ​​of each category; The labeling rules and unlabeled data corresponding to the current iteration are updated based on the probability value to obtain the labeling rules and unlabeled data corresponding to the next iteration.

4. The entity recognition model training method according to claim 1, characterized in that, After training the entity recognition model corresponding to the previous iteration based on the sample entity set corresponding to the current iteration to obtain the entity recognition model corresponding to the current iteration, the method further includes: The trained entity recognition model is distilled to obtain the corresponding lightweight entity recognition model.

5. An entity recognition method, characterized in that, include: Obtain the text to be recognized; The text to be recognized is input into a pre-trained entity recognition model to obtain the entity and its corresponding entity category output by the entity recognition model; wherein the entity recognition model is trained using the entity recognition model training method as described in any one of claims 1-4.

6. A computer program product, characterized in that, It includes computer program instructions, which, when read and executed by a processor, perform the method as described in any one of claims 1-5.

7. An electronic device, characterized in that, include: Processor, memory, and bus; The processor and the memory communicate with each other via the bus; The memory stores computer program instructions that can be executed by the processor, and the processor can invoke the computer program instructions to perform the method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a computer, cause the computer to perform the method as described in any one of claims 1-5.

Images