A data processing method for obtaining a named entity recognition model

Abstract

The application relates to the technical field of electric digital data processing, in particular to a data processing method for obtaining a named entity recognition model. The method comprises the following steps: traversing data se If the data se is annotated with the mtth preset type of entity, the proportion of positive samples of the mtth preset type of entity in the data se is determined as the mtth weight qz se of the data se,mt ; otherwise, the qz se,mt is set to 0; and the optimal text quantity for training the trained named entity recognition model is determined from the newly added text set according to the qz se,mt、 cb and zb. The application can obtain a better new training sample quantity.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electric digital data processing, in particular to a data processing method for obtaining a named entity recognition model. BACKGROUND

[0002] In the prior art, a trained named entity recognition model can recognize entities of specific types in a text. For example, CN109190110B discloses a training method and system for a named entity recognition model and an electronic device. The patent first pre-labels unlabeled corpus data obtained from a corpus based on a category library, and then modifies the pre-labeled results to obtain training text. Then, the named entity recognition model is trained based on the training text, so that the trained named entity recognition model has the function of recognizing the above-mentioned category words from the text.

[0003] When training a named entity recognition model, if two categories of words in the training text are labeled, then the named entity recognition model obtained after training has the function of recognizing two categories of words in the text. When training a named entity recognition model, if three categories of words in the training text are labeled, then the named entity recognition model obtained after training has the function of recognizing three categories of words in the text. If the trained named entity recognition model is required to recognize more entities of specific types, the trained named entity recognition model needs to be trained again. For example, the trained named entity recognition model can recognize entities of a first specific type and a second specific type. If the trained named entity recognition model is required to recognize entities of a third specific type, the existing method is to use new training samples and the labeling results of entities of the first specific type, the second specific type and the third specific type to train the trained named entity recognition model again. However, there may be a problem: although the named entity recognition model after retraining has the function of recognizing entities of the third specific type, the accuracy of the named entity recognition model after retraining in recognizing entities of the first specific type and the second specific type does not improve, but rather decreases, and sometimes the decrease increases with the increase in the number of new training samples. How to determine the number of new training samples to balance the recognition accuracy of the named entity recognition model after retraining for original specific types of entities and the recognition accuracy of the named entity recognition model after retraining for new specific types of entities is a problem to be solved. SUMMARY

[0004] The purpose of this invention is to provide a data processing method for obtaining a named entity recognition model, so as to determine the number of new training samples, so as to balance the recognition accuracy of the retrained named entity recognition model for the original specific types of entities and the recognition accuracy for newly added specific types of entities.

[0005] According to the present invention, a data processing method for obtaining a named entity recognition model is provided, the data processing method comprising the following steps:

[0006] D100, obtain the training text set data of the trained named entity recognition model. data includes st training text sets, each training text set including several training texts; st is the number of training text sets of the trained named entity recognition model; the trained named entity recognition model is a model trained using data, and the trained named entity recognition model is used to identify entities of preset type ty in the input text and label them with corresponding tags; ty includes ht preset types, ht is the number of preset types.

[0007] D200, iterate through data, if using data se When training a named entity recognition model, the data se If the mt-th entity of the preset type is annotated, then the data will be... se The proportion of positive samples for the mt-th preset type entity in the dataset is determined as data. se The mt-th weight qz se,mt Otherwise, set qz se,mt The value is 0; data se Let be the se-th training text set of the trained named entity recognition model, where se ranges from 1 to st; and mt ranges from 1 to ht.

[0008] D300, obtain the percentage of positive samples cb of ht preset types of entities in the test text set.

[0009] D400, retrieves the percentage of positive samples for ht preset types of entities in the newly added text set.

[0010] D500, according to qz se,mt、 cb and zb determine the optimal number of texts to obtain from the newly added text set for training the trained named entity recognition model.

[0011] Compared with the prior art, the present invention has at least the following beneficial effects:

[0012] The trained named entity recognition model in the present application is trained according to the training text set data, and has the function of recognizing ht preset types of entities; the new text set is the text obtained by the present application in order to make the trained text set have the function of recognizing entities of new preset types in addition to the above ht preset types, and the number of texts selected from the new text set when the trained named entity recognition model is retrained using the new text set will not only affect the recognition ability of the retrained named entity recognition model for entities of new preset types, but also affect the recognition ability of the retrained named entity recognition model for entities of the above ht preset types; the present application determines the optimal number of texts obtained from the new text set for training the trained named entity recognition model according to the proportion of positive samples of ht preset types in the test text set, the proportion of positive samples of ht preset types in the new text set zb and the weight of each training text set corresponding to each preset type in data, wherein the proportion of positive samples of ht preset types in the new text set determines the degree of influence of training the trained named entity recognition model using the new text set on the recognition accuracy of the original ht preset types, and the proportion of positive samples of ht preset types in the test text set and the weight of each training text set corresponding to each preset type in data reflect the recognition accuracy of the trained named entity recognition model for the original ht preset types, so that the present application can achieve the purpose of considering the recognition accuracy of the retrained named entity recognition model for the original ht preset types and the recognition accuracy of the retrained named entity recognition model for new preset types, and the retrained named entity recognition model has stronger semantic recognition ability. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0014] Figure 1 The flowchart of the data processing method for obtaining the named entity recognition model provided by the embodiments of the present application. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0016] According to the present application, a data processing method for obtaining a named entity recognition model is provided, which comprises the following steps, as shown in Figure 1

[0017] D100, obtaining a training text set data of a trained named entity recognition model, the data comprising st training text sets, each training text set comprising a plurality of training texts; st is the number of training text sets of the trained named entity recognition model; the trained named entity recognition model is a model trained using the data, and is used to identify entities of a preset type ty in an input text and label corresponding labels; ty comprises ht preset types, and ht is the number of preset preset types.

[0018] In this embodiment, data={data1, data2, …, data se ,…,data st}, data se is the st training text set of the trained named entity recognition model, data se comprises a plurality of training texts, and se ranges from 1 to st. ty=(ty1, ty2, …, ty mt ,…,ty ht ), ty mt is the mt preset type, and mt ranges from 1 to ht.

[0019] In this embodiment, the trained named entity recognition model is obtained by training using the st training text sets included in the data. The purpose of training the named entity recognition model using the st training text sets included in the data is to enable the trained named entity recognition model to have the function of identifying entities of the ht preset types.

[0020] In this embodiment, when the named entity recognition model is trained using the st training text sets included in the data, at least one entity of a preset type is labeled for each training text set and no entity other than the entities of the ht preset types is labeled, and each entity of the ht preset types is labeled by at least one training text set. Thus, the trained named entity recognition model has and only has the function of identifying entities of the ht preset types.

[0021] D200, traversing the data, if data se is labeled with an entity of the mt preset type when the named entity recognition model is trained using the data se , data se ​The positive sample proportion of the mt-th preset type of entity in data is determined as data se The mt-th weight qz of data se,mt ; otherwise, qz is set to 0; data se,mt The mt-th weight qz of data se is the st-th training text set of the trained named entity recognition model, and the value range of se is 1 to st; and the value range of mt is 1 to ht.

[0022] In the embodiment, when the named entity recognition model is trained using the st training text sets included in data, not all the ht preset types of entities are labeled when each training text set is labeled, for example, st=2 and ht=4, when the named entity recognition model is trained using the first training text set and the second training text set, only the first preset type and the second preset type of entities are labeled when the first training text set is labeled, and only the second preset type, the third preset type and the fourth preset type of entities are labeled when the second training text set is labeled.

[0023] In the embodiment, the positive sample proportion of the mt-th preset type of entity in data se is data se The ratio of the number of entities labeled as the mt-th preset type of entity in data se to the number of entities in data se If the named entity recognition model is trained using data se , the mt-th preset type of entity is labeled in data se , then the value of the positive sample proportion of the mt-th preset type of entity in data se,mt is assigned to qz se ; if the named entity recognition model is trained using data se , the mt-th preset type of entity is not labeled in data se,mt , then qz is set to 0.

[0024] D300, obtaining the positive sample proportion cb of the ht preset types of entities in the test text set.

[0025] In the embodiment, cb=(cb1, cb2,..., cb mt ,..., cb ht ), and cb mt is the positive sample proportion of the mt-th preset type of entity in the test text set.

[0026] In this embodiment, the positive sample proportion of the entity of the mt-th preset type in the test text set is the ratio of the number of entities labeled as the entity of the mt-th preset type in the test text set to the number of entities in the test text set.

[0027] In this embodiment, the positive sample proportion of the entity of the mt-th preset type in the test text set can represent the positive sample proportion of the entity of the mt-th preset type in the text set to be subsequently recognized by the trained named entity recognition. Optionally, the test text set is a text extracted from the text set to be subsequently recognized by the trained named entity recognition, and the entities of the ht preset types in the test text set are labeled by a manual labeling method, and the positive sample proportions of the ht preset types of entities are obtained.

[0028] D400, obtaining the positive sample proportions zb of the ht preset types of entities in the new text set.

[0029] In this embodiment, zb= (zb1, zb2,..., zb mt ,…,zb ht ), and zb mt is the positive sample proportion of the entity of the mt-th preset type in the new text set.

[0030] In this embodiment, the new text set is a text obtained to enable the trained text set to have the function of recognizing the new preset type of entity, and the positive sample proportion of the entity of the mt-th preset type in the new text set is the ratio of the number of entities labeled as the entity of the mt-th preset type in the new text set to the number of entities in the new text set. As an optional specific embodiment, D400 includes:

[0031] D410, obtaining a first extracted text set cq from the new text set by extracting a preset proportion of texts.

[0032] Optionally, a random extraction method is used to obtain the first extracted text set cq from the new text set by extracting a preset proportion of texts, wherein the preset proportion is an empirical value.

[0033] D420, using the trained named entity recognition model to recognize the ht preset types of entities in each text in cq to obtain the labeling result of the ht preset types of entities corresponding to each text in cq.

[0034] In this embodiment, each text in cq is used as the input of the trained named entity recognition model, and the corresponding labeling result of the ht preset types of entities can be obtained.

[0035] D430, determining zb mt according to the labeling result of the entity of the mt-th preset type corresponding to each text in cq.

[0036] In this embodiment, zb mt =∑ fs fe=1 zh fe,mt / (∑ fs fe=1 qh fe ), zh fe,mt is the number of entities in the fe-th text in cq that are labeled as the mt-th preset type of entity, qh fe is the number of entities included in the fe-th text in cq, and fe ranges from 1 to fs, where fs is the number of texts included in cq.

[0037] D500, determines the optimal number of texts from the new text set for training the trained named entity recognition model according to qz se,mt、 cband zb.

[0038] The trained named entity recognition model in this embodiment is trained according to the training text set data, and already has the function of recognizing ht preset types of entities; the new text set is a text obtained by the embodiment in order to enable the trained text set to have the function of recognizing entities of a new preset type in addition to the above ht preset types, and when the trained named entity recognition model is retrained using the new text set, the number of texts selected from the new text set will not only affect the recognition ability of the retrained named entity recognition model for entities of the new preset type, but also affect the recognition ability of the retrained named entity recognition model for entities of the above ht preset types; the embodiment determines the optimal number of texts from the new text set for training the trained named entity recognition model according to the proportion of positive samples of ht preset types of entities in the test text set, the proportion of positive samples of ht preset types of entities in the new text set zb, and the weight of each training text set in data corresponding to each preset type, wherein the proportion of positive samples of ht preset types of entities in the new text set determines the degree of influence of training the trained named entity recognition model using the new text set on the recognition accuracy of the original ht preset types of entities, and the proportion of positive samples of ht preset types of entities in the test text set and the weight of each training text set in data corresponding to each preset type reflect the recognition accuracy of the trained named entity recognition model for the original ht preset types of entities, so that the embodiment can achieve the purpose of taking into account the recognition accuracy of the retrained named entity recognition model for the original ht preset types of entities and the recognition accuracy for entities of the new preset type.

[0039] As a preferred specific embodiment, D500 includes:

[0040] D510, obtain a loss dif of the added text, dif = (∑ ht mt=1 |cb mt -((∑ st se=1 qz se,mt ×mu se )+zb mt ×zq) / (su se + zq)|)+ke×zq, mu se is the number of texts included in data se is the number of texts included in data se is the number of texts included in data se,mt is the number of texts included in data; ke is a preset adjustment coefficient, ke > 0, zq is the number of texts to be obtained from the added text set for training the trained named entity recognition model; zb mt is the proportion of positive samples of the mtth preset type of entity in the added text set; cb mt is the proportion of positive samples of the mtth preset type of entity in the test text set.

[0041] In this embodiment, ke is an empirical value, and optionally, 0 < ke < 1.

[0042] D520, determine the value of zq when dif takes the minimum value as the optimal number of texts to be obtained from the added text set for training the trained named entity recognition model.

[0043] Based on the above preferred specific embodiment, the loss of the added text corresponding to the optimal number of texts is the smallest, and the proportion of positive samples of each preset type of entity of the trained named entity recognition model using the texts obtained from the added text set as training samples according to the optimal number of texts is close to the proportion of positive samples of the corresponding preset type of entity in the test sample, so that the recognition accuracy of the named entity recognition model trained by using the texts with the optimal number of texts in the added text set for the original preset type of entity is higher; using the texts obtained from the added text set as training samples according to the optimal number of texts can also make the number of texts obtained from the added text set larger, so that the recognition accuracy of the named entity recognition model trained by using the texts with the optimal number of texts in the added text set for the new preset type of entity is also higher; thus, the optimal number of texts obtained based on the preferred specific embodiment can achieve the purpose of taking into account the recognition accuracy of the named entity recognition model for the original preset type of entity and the new preset type of entity, so that the named entity recognition model after retraining has stronger semantic recognition capability.

[0044] As an optional implementation, after D500, the computer readable instructions, when executed by the processor, further implement the following steps:

[0045] D600, obtaining fk texts from the new text set to form a to-be-labeled text set, where fk is the optimal number of texts.

[0046] D700, obtaining an annotation result res corresponding to the to-be-labeled text set, where the annotation result res includes an annotation result of ty corresponding to the to-be-labeled text set and an annotation result of a new preset type corresponding to the to-be-labeled text set; the new preset type is different from any ty. mt

[0047] In this embodiment, the annotation result of ty corresponding to the to-be-labeled text set includes an annotation result of each ty mt corresponding to the to-be-labeled text set.

[0048] D800, training the trained named entity recognition model using the to-be-labeled text set and the res.

[0049] Based on D600-D800, the trained named entity recognition model obtained has not only the function of recognizing entities of the new preset type, but also has a smaller decrease or an increase in the recognition accuracy of entities of the original preset type, thereby improving the overall recognition function of the trained named entity recognition model.

[0050] Although some specific embodiments of the present application have been described in detail through examples, those skilled in the art should understand that the above examples are only for illustration, but not for limiting the scope of the present application. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of the present application. The scope of the present application is defined by the appended claims.​

Claims

1. A data processing method for obtaining a named entity recognition model, characterized in that, The data processing method comprises the following steps: D100, obtaining a training text set data of a trained named entity recognition model, the data comprising st training text sets, each training text set comprising a plurality of training texts; st is the number of training text sets of the trained named entity recognition model; the trained named entity recognition model is a model trained using the data, and the trained named entity recognition model is used to identify entities of a preset type ty in an input text and label corresponding labels; ty comprises ht preset types, and ht is the number of preset preset types; D200, traverse data, if data is used se data when training the named entity recognition model se is annotated, the positive sample proportion of the mtth preset type of entity in data se is determined as the mtth weight qz se of data se,mt ; otherwise, qz se,mt is set to 0; data se is the sth training text set of the trained named entity recognition model, and the value range of se is 1 to st; the value range of mt is 1 to ht. D300, obtaining a positive sample proportion cb of entities of the ht preset types in the test text set; D400, obtaining a positive sample proportion zb of entities of the ht preset types in the new text set; D500, according to qz se,mt、 cb and zb determine the optimal number of texts to obtain from the new set of texts for training the trained named entity recognition model; D500 comprises: D510, obtain a loss dif of the added text, dif = (∑ ht mt=1 |cb mt - ((∑ st se=1 qz se,mt × mu se )+zb mt × zq) / (su se + zq)|)+ke×zq, mu se is the number of texts included in the data se su se is the number of texts included in the data corresponding to qz se,mt The sum of the number of texts included in the training text set that is not 0; ke is a preset adjustment coefficient, ke > 0, zq is the number of texts in the added text set that are used to train the trained named entity recognition model; zb mt is the proportion of positive samples of the mtth preset type of entity in the added text set; cb mt is the proportion of positive samples of the mtth preset type of entity in the test text set; D520, determining the value of zq when dif is the minimum value as the optimal number fk of texts obtained from the new text set for training the trained named entity recognition model. 2.The data processing method of acquiring a named entity recognition model according to claim 1, characterized in that, D400 comprises: D410, obtaining a first extraction text set cq comprising texts of a preset proportion from the new text set; D420, using the trained named entity recognition model to identify entities of the ht preset types in each text in cq, to obtain a labeling result of entities of the ht preset types corresponding to each text in cq; D430, determining zb according to the labeling result of each text in cq corresponding to the entity of the mt preset type mt . 3.The data processing method of acquiring a named entity recognition model according to claim 1, characterized in that, After D500, when the computer readable instructions are executed by the processor, the following steps are further implemented: D600, obtaining a to-be-labeled text set comprising fk texts from the new text set, fk being the optimal number of texts; D700, obtaining the annotation result res corresponding to the text set to be annotated, res including the annotation result of ty corresponding to the text set to be annotated and the annotation result of the newly added preset type corresponding to the text set to be annotated; the newly added preset type is different from any ty mt are all different; D800, training the trained named entity recognition model using the to-be-labeled text set and res. 4.The data processing method of acquiring a named entity recognition model according to claim 2, characterized in that, zb mt =∑ fs fe=1 zh fe,mt / (∑ fs fe=1 qh fe ),zh fe,mt is the number of entities in the fe-th text in cq that are labeled as the mt-th preset type of entity, qh fe is the number of entities included in the fe-th text in cq, fe ranges from 1 to fs, and fs is the number of texts included in cq. 5.The data processing method of acquiring a named entity recognition model according to claim 1, characterized in that, 0<ke<1. 6.The data processing method of acquiring a named entity recognition model according to claim 3, characterized in that, The labeling result of ty corresponding to the to-be-labeled text set comprises a labeling result of each preset type corresponding to the to-be-labeled text set.

Images