Training method, device and computer equipment for named entity recognition model

By combining public and specified domain datasets with a semi-supervised training method, the predicted labels of the trained model are iteratively updated, which solves the problem of low accuracy of BiLSTM-CRF model in named entity recognition in resume text, and achieves higher recognition accuracy and better generalization ability.

CN111553164BActive Publication Date: 2024-11-19PING AN TECH (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010357577.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-04-29
Publication Date
2024-11-19
Estimated Expiration
2040-04-29

AI Technical Summary

Technical Problem

Existing BiLSTM-CRF models have low accuracy in named entity recognition in resume text, and the training process is time-consuming and the amount of data is limited.

Method used

A semi-supervised training method is adopted, which combines a public dataset and a named entity training set in a specified domain. Two models are randomly selected from three training models for prediction through iteration. If the predicted labels are the same, the unlabeled data is updated to the unselected training model. This process continues until the unlabeled dataset is no longer updated, resulting in the final named entity recognition model.

Benefits of technology

It improves the accuracy of named entity recognition, reduces training time, and increases the reliability and generalization ability of the model, making it suitable for named entity recognition of resume text.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111553164B_ABST
    Figure CN111553164B_ABST
Patent Text Reader

Abstract

The present application relates to the field of artificial intelligence, and provides a training method, apparatus, computer equipment and storage medium for a named entity recognition model, including: training a BiLSTM-CRF model based on preset training samples to obtain three training models; iteratively randomly selecting any two training models from the three training models, and selecting one unlabeled target data from an unlabeled data set in turn and inputting it into the two selected training models for prediction, to obtain prediction labels predicted by the two training models; if the prediction labels predicted by the two training models are the same, adding the prediction label to the unlabeled target data and updating it to the training samples of the unselected training models to train the unselected training models; using a semi-supervised method to make full use of the original data and overcome the defect of insufficient labeled data; using the voting consistency principle of the three learning models to implicitly express confidence, thereby increasing the reliability of the model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a training method, apparatus, computer equipment and storage medium for a named entity recognition model. Background Art

[0002] In the scenario of recognizing electronic resume text, it is usually necessary to identify named entities such as school names and place names in the resume text. The named entity recognition (NER) task is mainly to identify and classify proper names such as names of people, places, and institutions that appear in the corresponding text. It is the basis of various natural language processing tasks such as information extraction, information retrieval, and question-answering systems. The currently used named entity recognition model is usually the BiLSTM-CRF model.

[0003] At present, higher requirements are placed on the accuracy of identifying named entities in resume texts, and the current BiLSTM-CRF model is usually based on a general corpus, so its recognition accuracy for resume texts is not high; at the same time, the training process mostly uses supervised methods for training, labeling is very time-consuming and the amount of data is limited. Summary of the invention

[0004] The main purpose of this application is to provide a training method, device, computer equipment and storage medium for a named entity recognition model, aiming to overcome the defects of low accuracy of the named entity recognition model and small amount of labeled data when training the model.

[0005] To achieve the above objectives, the present application provides a method for training a named entity recognition model, comprising the following steps:

[0006] When a request for training a named entity recognition model is received, a designated domain in which a target text to be recognized by the named entity recognition model is located is obtained; and according to the designated domain, a designated domain named entity training set is obtained;

[0007] The BiLSTM-CRF model is trained based on the preset training samples to obtain three training models; wherein the preset training samples include two sets of labeled data sets, which are public data sets and the specified domain named entity training sets;

[0008] Iteratively randomly select any two training models from the three training models, and select an unlabeled target data from the unlabeled data set in turn and input it into the two selected training models for prediction, so as to obtain the predicted labels predicted by the two training models;

[0009] If the predicted labels predicted by the two training models are the same, the predicted labels are added to the unlabeled target data and updated to the training samples of the unselected training model to train the unselected training model; and the unlabeled target data is put back into the unlabeled data set until the samples in the unlabeled data set are no longer updated, and the iterative training is stopped, and the three trained models are obtained as the final named entity recognition models; wherein the named entity recognition model is used to perform named entity recognition on the target text in the specified field.

[0010] Furthermore, the step of training the BiLSTM-CRF model based on the preset training samples to obtain three training models includes:

[0011] Based on the public data set, training is performed based on the BiLSTM-CRF model to obtain a pre-trained model;

[0012] Perform replacement sampling on the named entity training set in the specified domain to obtain three sets of training data sets;

[0013] The pre-training models are trained respectively based on the three groups of training data sets to obtain three training models.

[0014] Furthermore, the step of training the pre-trained models based on the three sets of training data sets to obtain three training models includes:

[0015] The model parameters of the BiLSTM in the pre-trained model are retained, and the model parameters of the CRF layer in the pre-trained model are initialized to obtain a pre-processed training model;

[0016] The preprocessing training model is trained respectively using the three sets of training data sets to obtain three training models.

[0017] Furthermore, the iterative training is stopped until the samples in the unlabeled data set are no longer updated, and the step of obtaining three trained models as the final named entity recognition model includes:

[0018] When receiving a named entity recognition instruction for a text to be recognized, the text to be recognized is input into any one of the named entity recognition models for prediction to obtain a named entity recognition result for the text to be recognized; wherein the named entity recognition result is a label of a character in the text to be recognized;

[0019] The text to be recognized is added to the unlabeled data set, and after the text to be recognized is added to the named entity recognition result, it is updated to the named entity training set in the specified field.

[0020] Furthermore, the step of training the BiLSTM-CRF model based on the preset training samples to obtain three training models includes:

[0021] Randomly select target public data from the public data set;

[0022] Dividing the specified domain named entity training set into a specified training set and a specified test set;

[0023] The target public data and the designated training set constitute a model training set, and the model training set is input into the BiLSTM-CRF model for training to obtain a pre-trained model;

[0024] Input the specified test set into the trained pre-trained model for testing, and obtain the correct probability that the predicted label of the specified test set is the correct label;

[0025] Determine whether the correct probability is greater than a preset probability, and if so, combine the target public data and the specified domain named entity training set into a target training set;

[0026] Perform replacement sampling on the target training set to obtain three sets of training data sets;

[0027] The pre-training models are trained respectively based on the three groups of training data sets to obtain three training models.

[0028] The present application also provides a training device for a named entity recognition model, comprising:

[0029] A request acquisition unit, configured to, upon receiving a request for training a named entity recognition model, acquire a designated domain in which a target text to be recognized by the named entity recognition model is located; and acquire a designated domain named entity training set according to the designated domain;

[0030] The first training unit is used to train the BiLSTM-CRF model based on preset training samples to obtain three training models; wherein the preset training samples include two sets of labeled data sets, which are public data sets and the specified domain named entity training set;

[0031] A first prediction unit is used to iteratively randomly select any two training models from the three training models, and select an unlabeled target data from the unlabeled data set in turn and input it into the two selected training models for prediction, so as to obtain the predicted labels predicted by the two training models;

[0032] The second training unit is used to add the predicted label to the unlabeled target data and update it to the training samples of the unselected training model if the predicted labels predicted by the two training models are the same, so as to train the unselected training model; and put the unlabeled target data back into the unlabeled data set until the samples in the unlabeled data set are no longer updated, and then stop the iterative training, so that the three trained models are all used as the final named entity recognition models; wherein the named entity recognition model is used to perform named entity recognition on the target text in the specified field.

[0033] Furthermore, the first training unit includes:

[0034] A first training subunit is used to perform training based on the public data set and the BiLSTM-CRF model to obtain a pre-trained model;

[0035] The first sampling subunit is used to perform replacement sampling on the specified domain named entity training set to obtain three sets of training data sets;

[0036] The second training subunit is used to train the pre-training models based on the three groups of training data sets respectively to obtain three training models.

[0037] Furthermore, the second training subunit is specifically used for:

[0038] The model parameters of the BiLSTM in the pre-trained model are retained, and the model parameters of the CRF layer in the pre-trained model are initialized to obtain a pre-processed training model;

[0039] The preprocessing training model is trained respectively using the three sets of training data sets to obtain three training models.

[0040] Furthermore, the training device for the named entity recognition model further includes:

[0041] A second prediction unit is configured to, upon receiving a named entity recognition instruction for a text to be recognized, input the text to be recognized into any one of the named entity recognition models for prediction, and obtain a named entity recognition result for the text to be recognized; wherein the named entity recognition result is a label of a character in the text to be recognized;

[0042] The adding unit is used to add the text to be recognized to the unlabeled data set, and after adding the text to be recognized to the named entity recognition result, update it to the named entity training set in the specified field.

[0043] Furthermore, the first training unit includes:

[0044] A selection subunit, used for randomly selecting target public data from the public data set;

[0045] A classification subunit, used for dividing the specified domain named entity training set into a specified training set and a specified test set;

[0046] A third training subunit is used to form a model training set by combining the target public data and the specified training set, and input the model training set into the BiLSTM-CRF model for training to obtain a pre-trained model;

[0047] A testing subunit, used to input the specified test set into the trained pre-trained model for testing, and obtain the correct probability that the predicted label of the specified test set is the correct label;

[0048] A judging unit, used for judging whether the correct probability is greater than a preset probability, and if so, combining the target public data and the specified domain named entity training set into a target training set;

[0049] The second sampling subunit is used to perform replacement sampling on the target training set to obtain three sets of training data sets;

[0050] The fourth training subunit is used to train the pre-training models based on the three groups of training data sets respectively to obtain three training models.

[0051] The present application also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0052] The present application also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above-mentioned methods are implemented.

[0053] The training method, apparatus, computer equipment and storage medium of the named entity recognition model provided in the present application include: training a BiLSTM-CRF model based on preset training samples to obtain three training models; wherein the preset training samples include two groups of labeled data sets, the two groups of labeled data sets are public data sets and training sets of named entities in a specified field; iteratively randomly selecting any two training models from the three training models, and selecting one unlabeled target data from the unlabeled data set in turn and inputting it into the two selected training models for prediction, so as to obtain predicted labels predicted by the two training models; if the predicted labels predicted by the two training models are the same, then adding the predicted labels to the unlabeled target data and updating it to the training samples of the unselected training models to train the unselected training models; and putting the unlabeled target data back into the unlabeled data set until the samples in the unlabeled data set are no longer updated, stopping the iterative training, and obtaining the three trained models as the final named entity recognition models. A semi-supervised approach is used to replace the original training method that only uses labeled data, which makes full use of the original data and overcomes the defect of insufficient labeled data. The three learning models use the voting consistency principle to implicitly express confidence, reduce the time required for frequent cross-validation, increase the reliability of the model, make the model training more effective, better the named entity recognition effect of resume text, and improve the generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a schematic diagram of the steps of a training method for a named entity recognition model in one embodiment of the present application;

[0055] Figure 2 It is a structural block diagram of a training device for a named entity recognition model in one embodiment of the present application;

[0056] Figure 3 A schematic block diagram of the structure of a computer device according to an embodiment of the present application.

[0057] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] Reference Figure 1 In one embodiment of the present application, a method for training a named entity recognition model is provided, comprising the following steps:

[0060] Step S01, when receiving a request to train a named entity recognition model, obtaining a designated domain where a target text to be recognized by the named entity recognition model is located; and obtaining a designated domain named entity training set according to the designated domain;

[0061] Step S1, training a BiLSTM-CRF model based on preset training samples to obtain three training models; wherein the preset training samples include two sets of labeled data sets, the two sets of labeled data sets are public data sets and a training set of named entities in a specified field;

[0062] Step S2, iteratively randomly selecting any two training models from the three training models, and sequentially selecting an unlabeled target data from the unlabeled data set and inputting it into the two selected training models for prediction, to obtain predicted labels predicted by the two training models;

[0063] Step S3, if the predicted labels predicted by the two training models are the same, then the unlabeled target data is added with the predicted label and updated to the training samples of the unselected training model to train the unselected training model; and the unlabeled target data is put back into the unlabeled data set until the samples in the unlabeled data set are no longer updated, and the iterative training is stopped, and the three trained models are obtained as the final named entity recognition models; wherein the named entity recognition model is used to perform named entity recognition on the target text in the specified field.

[0064] In this embodiment, the named entity recognition model trained in the above method is used to automatically identify school names, place names and other named entities in batches from the resume text (i.e., the above-mentioned specified field) to generate structured data; usually in the resume content recognition process, it is necessary to identify the school names, place names, etc. in the resume text, which requires the application of named entity recognition (NER) technology, and the above-mentioned named entity recognition model is to achieve this requirement.

[0065] At present, the above-mentioned named entity recognition model usually adopts the BiLSTM-CRF model, and its training samples usually use a large number of Chinese named entity datasets publicly available on the existing network (that is, the above-mentioned public dataset, which is an existing resource and has a large amount of data). The above-mentioned public dataset is a labeled dataset.

[0066] In this embodiment, the application scenario of this model is to realize named entity recognition of resume text, so the model in this embodiment needs to be trained for this scenario data set before it can be used for this task. If the named entity training set of the specified field is directly used, the overfitting problem will occur due to the small amount of data.

[0067] Therefore, as described in step S01 above, before performing named entity recognition on the target text in the specified field, it is necessary to train the corresponding named entity recognition model. Therefore, the user can trigger a request instruction for training the model, and when the terminal receives a request to train the named entity recognition model, in order to better train the above-mentioned named entity recognition model and improve its recognition accuracy, it is necessary to obtain the field in which the named entity recognition model is specifically used to recognize the target text, so as to obtain the training set of the corresponding field for training. Specifically, when a training request is received, the specified field in which all recognized target texts are located is obtained; if the specified field is a resume text field, then according to the resume text field, the entity training set in the corresponding resume text is obtained for the subsequent training process.

[0068] As described in step S1 above, since the application scenario of this model is resume text named entity recognition, this model needs to be trained on a named entity dataset in the resume text field before it can be used for this task. If the named entity dataset is used directly, the overfitting problem will occur due to the small amount of data. Therefore, this scheme first pre-trains the BiLSTM-CRF model based on the public dataset to obtain the pre-trained model M0 to initialize the neural network parameters, and then uses the specified domain named entity dataset for training. This method can effectively improve the robustness of the algorithm. The above training samples use a specified domain named entity training set (although the training set has a small amount of data, it is a specific domain vocabulary with strong pertinence). The above-mentioned specified domain named entity training set refers to the training corpus in the specified domain, which is also a labeled dataset. Specifically, in one embodiment, after first pre-training the model based on the public data set to obtain the pre-trained model M0, the specified domain named entity training set is divided into three groups of training data sets, and the pre-trained model M0 is trained based on each group of training data sets to obtain a training model respectively; the three groups of training models are all trained based on the pre-trained model M0, the difference is that the training data sets they use are different, so the training models finally obtained are also different.

[0069] As described in step S2 above, the above-mentioned model training using labeled data is a supervised training method, which is very time-consuming, and the amount of data is usually very limited. Therefore, in order to make full use of existing data, in this embodiment, a semi-supervised training method (tri-training) is further adopted to train the model, that is, in addition to using the above-mentioned labeled data set, an unlabeled data set is also used, which not only increases the amount of training data, but also increases the reliability of the model.

[0070] Specifically, based on the three models trained in step S1 above, in each round of semi-supervised training, any two models are randomly selected from the three models, and one unlabeled target data is selected from the unlabeled data set in turn for label prediction, that is, the predicted label corresponding to the same unlabeled target data is predicted by two random models. If the predicted labels predicted by the two models are the same, it can be considered that the confidence of the two models is high; otherwise, the confidence is low. It can be understood that the above-mentioned predicted label is not just one label, but a group of labels corresponding to the unlabeled data set; the number of labels in the group of labels depends on the number of words in the unlabeled data set.

[0071] The BIOES annotation method is used to annotate the above unlabeled dataset. In different application scenarios, the annotations corresponding to the same word will be different. For example, in some scenarios, a word is the beginning of a place name, and it can be annotated with B in the place name. If it is the end of the place name, it is annotated with E in the place name. For example, the north in "Beijing" will be annotated with B, and Beijing will be annotated with E. In other scenarios, the north mentioned above is a word in the name "Gubei", and the north may be annotated with E in the name, that is, the same word will have different labels in different scenarios.

[0072] Therefore, as described in step S3 above, if the two training models predict the same predicted labels for the same unlabeled target data, the predicted labels can be added to the unlabeled target data to update the training samples of the unselected training model to iteratively train the other unselected model. At the same time, the unlabeled target data is put back into the unlabeled data set. In this embodiment, it is necessary to determine whether the unlabeled data is added to the training sample in combination with the model prediction results. In the prior art, when the unlabeled data training model is used, it is usually determined whether the unlabeled data is added to the training sample based on whether the probability of the model predicting the unlabeled data reaches a threshold. This embodiment is significantly different from the prior art in which the model predicts the probability to determine whether the unlabeled data is added to the training sample; the present application combines the principle of consistency of voting of multiple models to implicitly express the confidence, which increases the reliability of the model, makes the model training effect better, and is more accurate in recognition.

[0073] If the predicted labels predicted by the two selected training models are different, the confidence of the two selected training models is low and further training is required. Therefore, the unlabeled target data cannot be added with the predicted labels to the training samples.

[0074] Repeat the above step S3 in sequence to complete the retraining of the above three training models.

[0075] Repeat the above steps S2 and S3 in sequence, that is, replace the two selected models, until the samples recognized from the unlabeled data set remain unchanged, stop the iterative training, indicating that the model training has been completed at this time, and obtain the final named entity recognition model.

[0076] In this embodiment, after each round of all unlabeled data prediction is completed, the training data set of the corresponding model is updated, and then the next cycle is entered, and the above steps are repeated until the training sets of all models are no longer updated. In this way, unlabeled data can be effectively used to increase the data volume of training samples and improve the generalization of the model.

[0077] In this embodiment, on the basis of the original BiLSTM-CRF, firstly, domain-specific vocabulary features are added, so that the accuracy of word segmentation is higher in the professional field, thereby improving the accuracy of named entity recognition; the neural network algorithm is combined and applied to the semi-supervised training method, that is, tri-training is applied in CRF and BiLSTM-CRF to complete the NER task, and the semi-supervised method is used to replace the original training with only labeled data, making full use of the original data, and overcoming the defect of insufficient labeled data at present; the three learning models are used to implicitly express the confidence by voting consistency principle, reducing the time required for frequent cross-validation, increasing the reliability of the model, making the model training effect better, and better for the named entity recognition effect of resume text, and improving the generalization ability. At the same time, the model trained in this embodiment can be iteratively trained with specific resume text in practical applications, such as in resume recognition scenarios, to automatically update the model.

[0078] In this embodiment, in order to enhance the efficient transmission and expression of information in the construction of smart cities, the above solution can also be used in smart office scenarios to promote the construction of smart cities.

[0079] In one embodiment, the step S1 of training the BiLSTM-CRF model based on the preset training samples to obtain three training models includes:

[0080] Step S11, based on the public data set, training is performed based on the BiLSTM-CRF model to obtain a pre-trained model;

[0081] Step S12, performing replacement sampling on the specified domain named entity training set to obtain three sets of training data sets;

[0082] Step S13, training the pre-trained models based on the three groups of training data sets respectively to obtain three training models.

[0083] Specifically, the step S13 of training the pre-trained models based on the three sets of training data sets to obtain three training models includes:

[0084] The model parameters of the BiLSTM in the pre-trained model are retained, and the model parameters of the CRF layer in the pre-trained model are initialized to obtain a pre-processed training model;

[0085] The preprocessing training model is trained respectively using the three sets of training data sets to obtain three training models.

[0086] In this embodiment, the difference between the two sets of labeled data sets is that the training set of named entities in the specified domain is the named entities specially labeled for the current task, and the public data set comes from a large-scale Chinese labeled named entity data set publicly available on the Internet; the specified domain is the domain where the current named entity recognition task is located. For example, in this embodiment, the specified domain can be the resume text domain.

[0087] As described in step S11 above, based on the public data set, the BiLSTM-CRF model is used for training to obtain the pre-trained model M0 to initialize the neural network parameters in the model; then as described in step S13 above, the three sets of training data sets obtained after the above-mentioned named entity training set in the specified domain is put back and sampled, and training is performed based on the above-mentioned pre-trained model M0. In this embodiment, the training samples used are public data sets and named entity training sets in the specified domain, which not only ensures that the above-mentioned three models obtained by training can guarantee a high traditional named entity recognition rate, but also ensures the named entity recognition effect in a specific domain.

[0088] In this embodiment, the BiLSTM-CRF model is first trained using a public data set to obtain M0. The M0 model in this state has optimized the model parameters and obtained a certain predictive ability after pre-training. On the basis of this M0 model, the CRF layer is replaced (i.e., initialized) and the training model is trained again using the specified domain named entity training set, and the parameters are further optimized so that the trained model can be used for the current task. In this embodiment, based on a large public data set, it is only to better initialize the parameters of the model neural network part (i.e., the parameters of the BiLSTM part) and improve the model robustness without initializing the CRF parameters. The model is ultimately used for resume named entity recognition, so it is necessary to train the labeled data set for this field, and the CRF layer will be retrained, so it is necessary to initialize the CRF layer. The initialization process is to retain only the pre-training parameters of the BiLSTM part, and reset the CRF parameters, and then use the replacement sampling method to obtain the above three training data sets, so as to train the model separately, and obtain the above three training models M1, M2, and M3. In this embodiment, compared with the current single use of the corresponding task data set or the single use of the public data set, the training method of this scheme can enable the model to obtain a higher generalization ability.

[0089] In one embodiment, until the samples in the unlabeled data set are no longer updated, the iterative training is stopped, and the three trained models are obtained as the final named entity recognition model, after step S3, the following steps are included:

[0090] Step S4, when receiving a named entity recognition instruction for the text to be recognized, input the text to be recognized into any one of the named entity recognition models for prediction to obtain a named entity recognition result for the text to be recognized; wherein the named entity recognition result is the label of the characters in the text to be recognized; the above process trains three named entity recognition models, all of which can be used to recognize named entities for the text to be recognized.

[0091] Step S5, adding the text to be recognized to the unlabeled data set, and after adding the named entity recognition result to the text to be recognized, updating it to the named entity training set in the specified field. In order to continuously update the above-mentioned named entity recognition model, that is, to iteratively train the above-mentioned named entity recognition model, the above-mentioned text to be recognized can continue to be used as a training sample, and the above-mentioned named entity recognition model can be continuously optimized. In this process, there is no need to manually annotate data, which reduces the workload, and the data volume of the training sample can be continuously increased.

[0092] In another embodiment, the step S1 of training the BiLSTM-CRF model based on the preset training samples to obtain three training models includes:

[0093] Step S1a, randomly select target public data from the public data set; in this embodiment, since the data volume of the above-mentioned named entity training set in the specified field is limited, some high-quality data can be selected from the above-mentioned public data set for training to enhance the recognition accuracy of the above-mentioned named entity recognition model. Therefore, the target public data is randomly selected from the public data set. In this process, the agent model can be used for selection. When selecting data, the agent model will automatically optimize the selection according to the result output by the final model, so that the quality of the selected data is getting better and better.

[0094] Step S1b, dividing the specified domain named entity training set into a specified training set and a specified test set;

[0095] Step S1c, the target public data and the designated training set constitute a model training set, and the model training set is input into the BiLSTM-CRF model for training to obtain a pre-trained model; in this embodiment, when training the above-mentioned BiLSTM-CRF model, if only the designated training set is used, the accuracy of the model will be the highest, but its data volume is small and its generalization ability is poor; the quality of the above-mentioned target public data is lower than that of the above-mentioned designated training set, and the above-mentioned target public data and the designated training set are trained together,

[0096] This will affect the accuracy of the model, but the better the quality of the target public data, the smaller the impact. Therefore, the quality of the above pre-trained model is related to the quality of the above target public data.

[0097] Step S1d, inputting the specified test set into the trained pre-trained model for testing, and obtaining the correct probability that the predicted label of the specified test set is the correct label;

[0098] Step S1e, determine whether the correct probability is greater than the preset probability. If it is, combine the target public data and the specified domain named entity training set into a target training set; in this embodiment, the above-mentioned specified test set is used to test the above-mentioned pre-trained model. If the correct probability that the predicted label of the specified test set is the correct label is greater than the preset probability, it indicates that the prediction ability of the above-mentioned pre-trained model is less affected, that is, the quality of the above-mentioned target public data is high, and the above-mentioned target public data can also be used as a target training set for the pre-trained model for subsequent training. If the above-mentioned correct probability is less than the preset probability, it indicates that the prediction ability of the above-mentioned pre-trained model is greatly affected, that is, the quality of the above-mentioned target public data is low, and at this time, it is necessary to randomly select another part of the target public data from the above-mentioned public data set.

[0099] Step S1f, performing replacement sampling on the target training set to obtain three sets of training data sets;

[0100] Step S1g, training the pre-trained models based on the three sets of training data sets to obtain three training models. The specific implementation of the above steps S1f and S1g is consistent with the above steps S12 and S13, and will not be repeated here.

[0101] Reference Figure 2 In one embodiment of the present application, a training device for a named entity recognition model is also provided, comprising:

[0102] The request acquisition unit 100 is used to acquire the designated domain where the target text to be recognized by the named entity recognition model is located when receiving a request for training the named entity recognition model; and acquire a designated domain named entity training set according to the designated domain;

[0103] The first training unit 10 is used to train the BiLSTM-CRF model based on preset training samples to obtain three training models; wherein the preset training samples include two sets of labeled data sets, which are public data sets and named entity training sets in a specified field;

[0104] The first prediction unit 20 is used to iteratively randomly select any two training models from the three training models, and select an unlabeled target data from the unlabeled data set in turn and input it into the two selected training models for prediction, so as to obtain the predicted labels predicted by the two training models;

[0105] The second training unit 30 is used to add the predicted label to the unlabeled target data and update it to the training samples of the unselected training model if the predicted labels predicted by the two training models are the same, so as to train the unselected training model; and put the unlabeled target data back into the unlabeled data set until the samples in the unlabeled data set are no longer updated, and then stop the iterative training, so that the three trained models are all used as the final named entity recognition models; wherein the named entity recognition model is used to perform named entity recognition on the target text in the specified field.

[0106] In one embodiment, the first training unit 10 includes:

[0107] A first training subunit is used to perform training based on the public data set and the BiLSTM-CRF model to obtain a pre-trained model;

[0108] The first sampling subunit is used to perform replacement sampling on the specified domain named entity training set to obtain three sets of training data sets;

[0109] The second training subunit is used to train the pre-training models based on the three groups of training data sets respectively to obtain three training models.

[0110] In one embodiment, the second training subunit is specifically configured to:

[0111] The model parameters of the BiLSTM in the pre-trained model are retained, and the model parameters of the CRF layer in the pre-trained model are initialized to obtain a pre-processed training model;

[0112] The preprocessing training model is trained respectively using the three sets of training data sets to obtain three training models.

[0113] In one embodiment, the training device for the named entity recognition model further includes:

[0114] A second prediction unit is configured to, upon receiving a named entity recognition instruction for a text to be recognized, input the text to be recognized into any one of the named entity recognition models for prediction, and obtain a named entity recognition result for the text to be recognized; wherein the named entity recognition result is a label of a character in the text to be recognized;

[0115] The adding unit is used to add the text to be recognized to the unlabeled data set, and after adding the text to be recognized to the named entity recognition result, update it to the named entity training set in the specified field.

[0116] In one embodiment, the first training unit 10 includes:

[0117] A selection subunit, used for randomly selecting target public data from the public data set;

[0118] A classification subunit, used for dividing the specified domain named entity training set into a specified training set and a specified test set;

[0119] A third training subunit is used to form a model training set by combining the target public data and the specified training set, and input the model training set into the BiLSTM-CRF model for training to obtain a pre-trained model;

[0120] A testing subunit, used to input the specified test set into the trained pre-trained model for testing, and obtain the correct probability that the predicted label of the specified test set is the correct label;

[0121] A judging unit, used for judging whether the correct probability is greater than a preset probability, and if so, combining the target public data and the specified domain named entity training set into a target training set;

[0122] The second sampling subunit is used to perform replacement sampling on the target training set to obtain three sets of training data sets;

[0123] The fourth training subunit is used to train the pre-training models based on the three groups of training data sets respectively to obtain three training models.

[0124] In this embodiment, the specific implementation of each unit and sub-unit in the above-mentioned device embodiment may refer to the specific implementation in the above-mentioned method embodiment, and will not be described in detail here.

[0125] Reference Figure 3 In an embodiment of the present application, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store training data, etc. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a training method for a named entity recognition model is implemented.

[0126] Those skilled in the art will understand that Figure 3 The structure shown in is merely a block diagram of a portion of the structure related to the present application solution and does not constitute a limitation on the computer device to which the present application solution is applied.

[0127] An embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, a training method for a named entity recognition model is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0128] In summary, the training method, apparatus, computer equipment and storage medium of the named entity recognition model provided in the embodiments of the present application include training a BiLSTM-CRF model based on preset training samples to obtain three training models; wherein the preset training samples include two groups of labeled data sets, the two groups of labeled data sets are public data sets and named entity training sets in a specified field; iteratively randomly select any two training models from the three training models, and select one unlabeled target data from the unlabeled data set in turn and input it into the two selected training models for prediction, so as to obtain the predicted labels predicted by the two training models; if the predicted labels predicted by the two training models are the same, then the unlabeled target data is added with the predicted label and updated to the training samples of the unselected training models to train the unselected training models; and the unlabeled target data is put back into the unlabeled data set until the samples in the unlabeled data set are no longer updated, and the iterative training is stopped to obtain the three trained models as the final named entity recognition models. A semi-supervised approach is used to replace the original training method that only uses labeled data, which makes full use of the original data and overcomes the defect of insufficient labeled data. The three learning models use the voting consistency principle to implicitly express confidence, reduce the time required for frequent cross-validation, increase the reliability of the model, make the model training more effective, better the named entity recognition effect of resume text, and improve the generalization ability.

[0129] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0130] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0131] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A training method for a named entity recognition model, characterized in that: The following steps are involved: When a request for training a named entity recognition model is received, a designated domain in which a target text to be recognized by the named entity recognition model is located is obtained; and according to the designated domain, a designated domain named entity training set is obtained; The BiLSTM-CRF model is trained based on the preset training samples to obtain three training models; wherein the preset training samples include two sets of labeled data sets, which are public data sets and the specified domain named entity training sets; Iteratively randomly select any two training models from the three training models, and select an unlabeled target data from the unlabeled data set in turn and input it into the two selected training models for prediction, so as to obtain the predicted labels predicted by the two training models; If the predicted labels predicted by the two training models are the same, the predicted labels are added to the unlabeled target data and updated to the training samples of the unselected training model to train the unselected training model; and the unlabeled target data is put back into the unlabeled data set until the samples in the unlabeled data set are no longer updated, and the iterative training is stopped, and the three trained models are obtained as the final named entity recognition models; wherein the named entity recognition model is used to perform named entity recognition on the target text in the specified field.

2. The training method of the named entity recognition model according to claim 1, characterized in that: The step of training the BiLSTM-CRF model based on the preset training samples to obtain three training models includes: Based on the public data set, training is performed based on the BiLSTM-CRF model to obtain a pre-trained model; Perform replacement sampling on the named entity training set in the specified domain to obtain three sets of training data sets; The pre-training models are trained respectively based on the three groups of training data sets to obtain three training models.

3. The training method of the named entity recognition model according to claim 2, characterized in that: The step of respectively training the pre-trained models based on the three sets of training data sets to obtain three training models includes: The model parameters of the BiLSTM in the pre-trained model are retained, and the model parameters of the CRF layer in the pre-trained model are initialized to obtain a pre-processed training model; The preprocessing training model is trained respectively using the three sets of training data sets to obtain three training models.

4. The method for training a named entity recognition model according to claim 1, characterized in that: After the step of stopping iterative training until the samples in the unlabeled data set are no longer updated and obtaining three trained models as the final named entity recognition models, the method further comprises: When receiving a named entity recognition instruction for a text to be recognized, the text to be recognized is input into any one of the named entity recognition models for prediction to obtain a named entity recognition result for the text to be recognized; wherein the named entity recognition result is a label of a character in the text to be recognized; The text to be recognized is added to the unlabeled data set, and after the text to be recognized is added to the named entity recognition result, it is updated to the named entity training set in the specified field.

5. The method for training a named entity recognition model according to claim 1, characterized in that: The step of training the BiLSTM-CRF model based on the preset training samples to obtain three training models includes: Randomly select target public data from the public data set; Dividing the specified domain named entity training set into a specified training set and a specified test set; The target public data and the designated training set constitute a model training set, and the model training set is input into the BiLSTM-CRF model for training to obtain a pre-trained model; Input the specified test set into the trained pre-trained model for testing, and obtain the correct probability that the predicted label of the specified test set is the correct label; Determine whether the correct probability is greater than a preset probability, and if so, combine the target public data and the specified domain named entity training set into a target training set; Perform replacement sampling on the target training set to obtain three sets of training data sets; The pre-training models are trained respectively based on the three groups of training data sets to obtain three training models.

6. A training device for a named entity recognition model, characterized in that: include: A request acquisition unit, configured to, upon receiving a request for training a named entity recognition model, acquire a designated domain in which a target text to be recognized by the named entity recognition model is located; and acquire a designated domain named entity training set according to the designated domain; The first training unit is used to train the BiLSTM-CRF model based on preset training samples to obtain three training models; wherein the preset training samples include two sets of labeled data sets, which are public data sets and the specified domain named entity training set; A first prediction unit is used to iteratively randomly select any two training models from the three training models, and select an unlabeled target data from the unlabeled data set in turn and input it into the two selected training models for prediction, so as to obtain the predicted labels predicted by the two training models; The second training unit is used to add the predicted label to the unlabeled target data and update it to the training samples of the unselected training model if the predicted labels predicted by the two training models are the same, so as to train the unselected training model; and put the unlabeled target data back into the unlabeled data set until the samples in the unlabeled data set are no longer updated, and then stop the iterative training, so that the three trained models are all used as the final named entity recognition models; wherein the named entity recognition model is used to perform named entity recognition on the target text in the specified field.

7. The training device for a named entity recognition model according to claim 6, characterized in that: The first training unit comprises: A first training subunit is used to perform training based on the public data set and the BiLSTM-CRF model to obtain a pre-trained model; The first sampling subunit is used to perform replacement sampling on the specified domain named entity training set to obtain three sets of training data sets; The second training subunit is used to train the pre-training models based on the three groups of training data sets respectively to obtain three training models.

8. The training device for a named entity recognition model according to claim 7, characterized in that: The second training subunit is specifically used for: The model parameters of the BiLSTM in the pre-trained model are retained, and the model parameters of the CRF layer in the pre-trained model are initialized to obtain a pre-processed training model; The preprocessing training model is trained respectively using the three sets of training data sets to obtain three training models.

9. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.