Method and apparatus for training named entity recognition model, electronic device and medium

Abstract

The application provides a training method and device of a named entity recognition model, an electronic device and a medium. The method comprises: obtaining a plurality of entity corpora in an irregular corpus, wherein the entity corpus is a corpus containing an entity; constructing a template library based on the plurality of entity corpora; repeatedly selecting a plurality of training data from the template library and an entity dictionary, wherein the training data is data containing an entity and label information of the entity; and training an initial network model using the plurality of training data to obtain a target named entity recognition model. The NER model trained by the existing method cannot overcome the problems of long-tail distribution and fast entity update iteration, thereby achieving the technical effects of improving the entity recognition effect of the NER model in a specific field and reducing the labeling difficulty and maintenance cost.

Description

Technical Field

[0001] This application relates to network model training technology, and more particularly to a training method, apparatus, electronic device and medium for a named entity recognition model. Background Technology

[0002] Named Entity Recognition (NER) is a crucial and fundamental problem in Natural Language Processing (NLP), and NER technology has developed rapidly in recent years. However, current NER models still struggle to be applied to certain domains, such as those with severe long-tail data distributions and rapidly evolving entity updates (e.g., after-sales service). This results in poor entity recognition performance, high annotation difficulty, and high maintenance costs for current NER models in these domains.

[0003] For example, the long-tail distribution problem makes it difficult and costly for learning-based named entity recognition models to obtain high-quality labeled data when extracting training data. Furthermore, it's challenging to obtain comprehensive training data, resulting in insufficient feature learning for less common products and lower recognition accuracy. Another example is the rapid update and iteration of entities. Rule-based named entity recognition models not only require significant resources to maintain an entity dictionary, but the time required for each match also increases linearly with the size of the dictionary. Moreover, they have weak generalization ability, performing poorly on entities not in the dictionary or those using slang or aliases.

[0004] Therefore, there is a need to provide a training method for NER models that can overcome the problems of long-tail distribution and fast entity update iteration, so as to improve the entity recognition performance of NER models in specific domains and reduce the difficulty of annotation and maintenance costs. Summary of the Invention

[0005] This application provides a training method, apparatus, electronic device, and storage medium for a named entity recognition model, which solves the problems of long-tail distribution and fast entity update iteration in existing NER models, thereby improving the entity recognition performance of NER models in specific domains and reducing annotation difficulty and maintenance costs.

[0006] On the one hand, this application provides a training method for a named entity recognition model, the method comprising:

[0007] Obtain multiple entity corpora from an unstructured corpus, wherein the aforementioned entity corpora are corpora containing entities;

[0008] A template library was constructed based on multiple entity corpora mentioned above;

[0009] Multiple training data were obtained by repeatedly selecting from the above template library and entity dictionary, wherein the above training data is data containing the above entities and the label information of the above entities;

[0010] The initial network model was trained using multiple sets of the aforementioned training data to obtain the target named entity recognition model.

[0011] Furthermore, the above-mentioned acquisition of multiple entity corpora from the ruleless corpus includes:

[0012] Using an exact matching algorithm, the aforementioned entity corpus and other corpus in the above-mentioned irregular corpus are filtered out. Among them, the aforementioned other corpus is the corpus in which it is not determined whether it contains entities.

[0013] A fuzzy matching algorithm was used to filter out entity corpora and non-entity corpora from the other corpora mentioned above. The non-entity corpora are those that do not contain entities.

[0014] Furthermore, after obtaining multiple entity corpora from the irregular corpus, the above method also includes:

[0015] The positions of the aforementioned entities in the aforementioned entity corpus are marked to obtain the aforementioned marking information.

[0016] Furthermore, the template library constructed based on the aforementioned entity corpora includes:

[0017] Multiple pre-specified replacement entities are used to replace entities at predetermined positions in multiple entity corpora to obtain multiple replaced entity corpora. The replacement process is used to replace non-entity characters in each entity corpus with predetermined symbols.

[0018] Multiple seed templates were determined based on multiple entity corpora with the above replacements;

[0019] Each validated seed template is processed to build a template library based on the processed seed templates.

[0020] Furthermore, based on the aforementioned replaced entity corpora, multiple seed templates are determined, including:

[0021] Based on the semantics of the replaced entity corpus, multiple replaced entity corpora are clustered to obtain multiple cluster centers;

[0022] Extract N sentences from each of the above cluster centers;

[0023] Each of the above N sentences for each of the above cluster centers is used as a seed template for the above cluster centers.

[0024] Furthermore, the above-mentioned training data were repeatedly selected from the template library and entity dictionary, including:

[0025] Each time, a training template is randomly selected from the template library mentioned above, and each time a training entity is randomly selected from the entity dictionary mentioned above;

[0026] The above training template and the above training entity are combined to obtain the above training data;

[0027] Repeat the above random sampling, random selection, and combination operations multiple times to obtain multiple sets of the above training data.

[0028] On the other hand, this application provides a recognition method based on a named entity recognition model, the method comprising:

[0029] Obtain the data to be identified;

[0030] A target named entity recognition model is used to process the above-mentioned data to be recognized and obtain the recognition result. The target named entity recognition model is obtained by training an initial network model with multiple training data. The multiple training data are repeatedly selected from a template library and an entity dictionary. The template library is constructed based on multiple entity corpora. The multiple entity corpora are obtained from unstructured corpora. The entity corpora are corpora containing entities. The training data are data containing the above-mentioned entities and their tag information.

[0031] On the other hand, this application provides a training apparatus for a named entity recognition model, the apparatus comprising:

[0032] The acquisition module is used to acquire multiple entity corpora from the irregular corpus, wherein the aforementioned entity corpora are corpora containing entities;

[0033] The building module is used to construct a template library based on multiple of the above entity corpora;

[0034] The selection module is used to repeatedly select multiple training data from the template library and entity dictionary, wherein the training data is data containing the entities and their tag information.

[0035] The training module is used to train an initial network model using multiple sets of the aforementioned training data to obtain a target named entity recognition model.

[0036] Furthermore, the aforementioned acquisition module includes:

[0037] The first matching submodule is used to use an exact matching algorithm to filter out the entity corpus and other corpus in the above-mentioned irregular corpus, wherein the above-mentioned other corpus is corpus in which it is not determined whether it contains entities;

[0038] The second matching submodule is used to use a fuzzy matching algorithm to filter out entity corpora and non-entity corpora from the other corpora mentioned above, wherein the non-entity corpora are corpora that do not contain entities.

[0039] Furthermore, the aforementioned device also includes:

[0040] The tagging module is used to tag the position of the above entities in the above entity corpus and obtain the above tagging information.

[0041] Furthermore, the aforementioned building modules include:

[0042] The replacement submodule is used to replace entities at predetermined positions in multiple entity corpora by using multiple pre-specified replacement entities to obtain multiple replaced entity corpora. The replacement process is used to replace non-entity characters in each entity corpus with predetermined symbols.

[0043] The first determination submodule is used to determine multiple seed templates based on multiple replaced entity corpora as described above.

[0044] The construction submodule is used to process the data of each verified seed template, and to build a template library based on the multiple processed seed templates.

[0045] Furthermore, the aforementioned determining module includes:

[0046] The clustering submodule is used to cluster multiple replaced entity corpora based on the semantics of the replaced entity corpora to obtain multiple cluster centers.

[0047] The extraction submodule is used to extract N sentences from each of the above cluster centers;

[0048] The second determining submodule is used to take the above N sentences of each of the above cluster centers as seed templates for the above cluster centers.

[0049] Furthermore, the above selection module includes:

[0050] The selection submodule is used to randomly select a training template from the template library each time, and to randomly select a training entity from the entity dictionary each time.

[0051] The combination submodule is used to combine the above training template and the above training entity to obtain the above training data;

[0052] The repeat operation submodule is used to repeat the above random sampling, random selection and combination operations multiple times to obtain multiple training data.

[0053] On the other hand, this application provides a recognition device based on a named entity recognition model, the device comprising:

[0054] The acquisition unit is used to acquire the data to be identified.

[0055] The recognition unit is used to perform recognition processing on the data to be recognized using a target named entity recognition model to obtain a recognition result. The target named entity recognition model is obtained by training an initial network model using multiple training data. The multiple training data are repeatedly selected from a template library and an entity dictionary. The template library is constructed based on multiple entity corpora. The multiple entity corpora are obtained from unstructured corpora. The entity corpora are corpora containing entities. The training data are data containing the entities and their tag information.

[0056] On the other hand, this application provides an electronic device, including: a processor and a memory connected to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement any of the methods described above.

[0057] On the other hand, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement any of the methods described above.

[0058] On the other hand, this application provides a computer program product, including a computer program that, when executed by a processor, implements any of the methods described above.

[0059] The naming entity recognition (NER) model training method, apparatus, electronic device, and storage medium provided in this application acquire multiple entity corpora from irregular corpora, wherein the entity corpora are corpora containing entities; construct a template library based on the multiple entity corpora; repeatedly select multiple training data from the template library and entity dictionary, wherein the training data are data containing the entities and their tag information; and train an initial network model using the multiple training data to obtain a target named entity recognition model. This addresses the problems of long-tail distribution and rapid entity update iteration in existing NER models, achieving the technical effect of improving the entity recognition performance of NER models in specific domains while reducing annotation difficulty and maintenance costs. Attached Figure Description

[0060] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0061] Figure 1 This is a flowchart illustrating a training method for a named entity recognition model provided in an embodiment of the present invention.

[0062] Figure 2 This is a flowchart illustrating an optional named entity recognition model training method provided in an embodiment of the present invention.

[0063] Figure 3 This is a flowchart illustrating an optional named entity recognition model training method provided in an embodiment of the present invention.

[0064] Figure 4 This is a flowchart illustrating a recognition method based on a named entity recognition model provided in an embodiment of the present invention.

[0065] Figure 5 A structural block diagram of a training device for a named entity recognition model provided in an embodiment of this application;

[0066] Figure 6 A structural block diagram of a recognition device based on a named entity recognition model provided in this application embodiment;

[0067] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0068] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0069] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0070] First, let me explain the terms used in this application:

[0071] Named Entity Recognition (NER) is the process of identifying predefined entity types (names of people, organizations, places, etc.) in a text; it is a very important and fundamental problem in Natural Language Processing (NLP).

[0072] Natural Language Processing (NLP) is a discipline that uses computer technology to analyze, understand, and process natural language. It uses computers as a powerful tool for language research, quantitatively studies language information with the support of computers, and provides language descriptions that can be used by both humans and computers.

[0073] The development of NER technology can be mainly divided into two stages: rule-based methods and learning-based methods. Early NER models were primarily based on manually defined rules for recognition. These methods are characterized by high accuracy and fast recognition speed, and have achieved impressive results in many fields. Common methods include entity dictionary-based matching methods and lexical-based matching methods; for example, edit distance is a typical dictionary-based method.

[0074] In some specialized fields, such as medicine, there are also field-specific rules. Nevertheless, rule-based methods often have poor generalization ability, poor accuracy in recognizing unfamiliar entities, and difficulty in recognizing slang and aliases. Furthermore, they require significant effort to maintain an entity dictionary or rule base. Therefore, learning-based methods have been proposed to build upon these approaches.

[0075] The biggest advantage of learning-based methods is their strong generalization ability and their capacity to identify previously unseen entities based on semantic information, thus mitigating the problem of poor recognition of slang and aliases to some extent. Currently, most mainstream methods are learning-based. These methods often require manually labeled training datasets of a sufficiently large size, but obtaining high-quality training data is often costly. To address this, many solutions have been proposed, such as unsupervised learning, few-shot learning, and remote supervision. Remote supervision, which combines the advantages of supervised and unsupervised learning, is a promising technique. Its principle can be summarized as automatically labeling training data using an entity dictionary and other algorithms. However, the data labeled using remote supervision methods often contains a significant amount of noise.

[0076] Traditional NER models are mostly rule-based, typically using entity dictionaries or word combination combinations for identification. These methods are widely used in the medical field. The drawbacks of these existing technologies are: First, they require a large entity dictionary, which is time-consuming and labor-intensive to maintain. Furthermore, the time required for each match increases linearly with the size of the entity dictionary. Entities that will no longer appear also need to be removed, further increasing the workload. Second, they have weak generalization ability. They perform poorly in recognizing entities not in the entity dictionary or those using slang or aliases, requiring the manual development of new rules, which is very costly. Moreover, methods suitable for one domain cannot be applied to other domains.

[0077] Learning-based methods generally require a high-quality, labeled training dataset to train on, thereby acquiring the corresponding entity recognition capabilities. The drawbacks of the aforementioned existing technologies are: these methods often require high-quality labeled data, which is difficult and costly to obtain. Furthermore, in industrial applications, due to the severe long-tail distribution of data, increasing the amount of data by 100 times may not even improve entity coverage by 1%. Therefore, it is difficult to obtain comprehensive training data, resulting in insufficient feature learning for less common products and lower recognition accuracy.

[0078] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0079] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0080] According to one or more embodiments of this application, the training method for the named entity recognition model provided in this application aims to solve the above-mentioned technical problems of the prior art. Figure 1 This is a flowchart illustrating a training method for a named entity recognition model provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes:

[0081] S101, obtain multiple entity corpora from the irregular corpus.

[0082] S102, a template library is constructed based on multiple of the above-mentioned entity corpora.

[0083] S103, repeatedly select multiple training data from the above template library and entity dictionary.

[0084] S104. The initial network model is trained using multiple training data sets mentioned above to obtain the target named entity recognition model.

[0085] Optionally, the above entity corpus is a corpus containing entities.

[0086] Optionally, the training data mentioned above is data containing the aforementioned entities and their labeling information.

[0087] The training method for a named entity recognition model provided in this embodiment of the invention can be used, but is not limited to, in business areas where data often has a severe long-tail distribution and rapid entity updates, such as the after-sales service field, specifically the after-sales service field in the industrial or commercial sectors.

[0088] Understandably, information extraction from unstructured text is a crucial task in Natural Language Processing (NLP), and Named Entity Recognition (NER) models form the foundation of this process. A well-trained NER can be defined as extracting predefined entities from unstructured corpus data and classifying them into corresponding categories (such as people, organizations, and locations).

[0089] Optionally, the irregular corpus in the embodiments of this application refers to irregular text or unstructured text, such as dialogues or documents.

[0090] Optionally, the entity corpus in this application embodiment can be text containing predefined entities. For example, entity corpus in irregular text can be obtained by using exact matching algorithm and fuzzy matching algorithm.

[0091] In one optional embodiment, after obtaining multiple entity corpora in the irregular corpus, the method further includes: marking the position of the entity in the entity corpus to obtain the marking information.

[0092] Optionally, the template library is constructed based on multiple entity corpora and contains multiple seed templates. When training the initial network model, multiple training templates, i.e. training code templates, are randomly selected from the template library multiple times. Then, by repeatedly selecting a training entity from the entity dictionary and combining the training template and the training entity, training data can be obtained.

[0093] Furthermore, using the multiple training data obtained in the embodiments of this application, an initial network model is trained to obtain a target named entity recognition model.

[0094] Optionally, in this embodiment, the initial network model can be a BERT (Bidirectional Encoder Representations from Transformer) + CRF (Conditional Random Field) model. The aforementioned target named entity recognition model is the NER model.

[0095] To address the issue of severe long-tail distribution of data in some domains, this application's embodiments extract seed templates from sentences containing entities to construct a template library. Then, combined with an entity dictionary, a large amount of labeled training data containing entities is automatically generated to train the initial network model, resulting in a target named entity recognition model. This allows for the easy deployment of named entity recognition models across various domains using a training method that requires minimal human intervention.

[0096] In this embodiment of the application, to obtain high-quality labeled data at minimal cost, an improvement is made to the concept of remote supervision. While remote supervision is used for locating entities, it is not used for generating training data, thus avoiding the drawback of high noise levels associated with remote supervision. Instead, training data is generated and labeled using a template library and entity dictionary produced by the aforementioned method. This avoids the noise problem introduced by remote supervision and reduces the accuracy requirements for remote supervision recognition, thereby alleviating to some extent the difficulties in acquiring industrial data and the severe uneven distribution of data.

[0097] In one example, the above method of obtaining multiple entity corpora from an unstructured corpus includes:

[0098] S201, using an exact matching algorithm, filter out the aforementioned entity corpus and other corpus from the aforementioned irregular corpus, wherein the aforementioned other corpus is corpus in which it is uncertain whether it contains entities.

[0099] S202, using a fuzzy matching algorithm, the entity corpus and non-entity corpus of the other corpus are selected. The non-entity corpus is the corpus that does not contain entities.

[0100] Optionally, the exact matching algorithm described above can be a one-to-one dictionary matching method, for example, directly searching for the entity in the string. The fuzzy matching algorithm uses edit distance, which is generally used to calculate the similarity between two strings. It is defined as the minimum number of operations required to transform one string into another, and optional operations include insertion, deletion, and replacement.

[0101] A fuzzy matching algorithm is used to filter other corpora, resulting in entity corpora and non-entity corpora. The aforementioned other corpora are those where it is uncertain whether they contain entities. For example, a threshold related to entity length can be predefined; if the edit distance is less than this threshold, a match is considered successful; otherwise, the entity is considered not to be contained.

[0102] As an optional example, such as Figure 2 As shown, entity-containing parts can be extracted from a large amount of irregular text using a combination of exact matching and fuzzy matching algorithms. For all the corpus data, exact matching is used first; if an entity is found, its location is marked. If no entity is found, fuzzy matching is performed. Fuzzy matching uses edit distance for matching; if an entity is matched, its location is marked. Finally, the results of the two methods are fused to obtain the final entity annotation result.

[0103] It should be noted that, in this embodiment of the application, the above matching and filtering steps do not require the extraction of all corpora containing entities, nor do they require the extracted entity positions to be completely accurate and noise-free.

[0104] In this embodiment of the application, the requirements for recognition accuracy are reduced when performing precise matching and fuzzy matching. By improving the remote supervision method, it has the advantages of remote supervision in automatically labeling training data, while eliminating the noise problem caused by remote supervision.

[0105] Furthermore, such as Figure 3 As shown, the template library constructed based on the aforementioned entity corpora includes:

[0106] S301, using multiple pre-specified replacement entities, the entities at predetermined positions in the multiple entity corpora mentioned above are replaced respectively, resulting in multiple replaced entity corpora.

[0107] S302, determine multiple seed templates based on multiple entity corpora after the above replacements.

[0108] S303, perform data processing on each verified seed template to build a template library based on the processed seed templates.

[0109] Optionally, the replacement process is used to replace the non-entity characters in each entity corpus with predetermined symbols, and subsequent data processing is performed on each verified seed template, which refers to data augmentation processing.

[0110] Understandably, data augmentation can generate more data from limited data, increasing the number and diversity of training samples (noisy data), thus improving model robustness. It is generally used on the training set. Neural networks require a large number of parameters; many neural networks have millions of parameters. For these parameters to function correctly, a large amount of data is needed for training. However, in many real-world projects, it is difficult to find sufficient data to complete the task. Randomly changing the training samples can reduce the model's dependence on certain attributes, thereby improving the model's generalization ability.

[0111] Optionally, for the extracted entity corpus containing entities, such as Figure 2 As shown, entity replacement is performed first. For example, 10 entities {e1, e2, ..., e...} can be specified. 10 For the nth entity appearing in an entity corpus, e can be used. i The replacement is performed. Then, the sentences are clustered semantically, and three sentences from each cluster center are extracted as seed templates for that cluster. These seed templates are then validated, for example, manually or by machine. Next, the seed templates are enhanced; for example, some non-entity characters are replaced with the [MASK] symbol. BERT is then asked to provide the five most likely characters for these [MASK] positions, and one of these characters is randomly selected to replace the original [MASK], resulting in an enhanced seed template. A corresponding template library can be generated based on the seed templates obtained in this way.

[0112] In this embodiment, data with different characteristics is categorized by replacing entities and then clustering. Multiple data points from the cluster centers are then manually verified to obtain seed templates. Finally, data augmentation is performed on these seed templates to obtain a template library. This method can obtain a high-quality template library with minimal manual effort, laying the foundation for subsequent steps.

[0113] In another example, multiple seed templates are determined based on multiple replaced entity corpora, including:

[0114] S501, based on the semantics of the above-mentioned replaced entity corpus, cluster the multiple replaced entity corpora to obtain multiple cluster centers.

[0115] S402, extract N sentences from each of the above cluster centers.

[0116] S403, take the above N sentences of each of the above cluster centers as the seed template of the above cluster centers.

[0117] In this embodiment, the part containing the entity and its position are extracted by precise matching and fuzzy matching, and then the entity is replaced. Then, according to the semantics of the replaced entity corpus, multiple replaced entity corpora are clustered. For example, three corpus data from each cluster center are extracted as seed templates. Then, 20% of the characters in the seed template that do not contain the entity part can be randomly replaced and BERT is used to predict. From the prediction results, any one of the five most likely results is randomly selected to fill the original position, thereby obtaining the enhanced template. The template library is obtained based on the seed template through this method.

[0118] To improve named entity recognition in industrial data, this application proposes a method based on template libraries and data augmentation. The basic idea is to first extract entities using edit distance, and then replace all entities with the same entity to avoid semantic differences caused by different entities.

[0119] Next, sentences containing entities are clustered semantically. Sentences with cluster centers are extracted from each cluster to generate templates, which are then manually validated. These templates are then augmented to create a template library. This library, along with an existing entity dictionary, is then used to automatically generate a large amount of labeled training data containing entities. This alleviates the problem of high noise in remote supervised annotation data and also addresses, to some extent, the severe long-tail distribution problem of industrial data.

[0120] In one example, multiple training data are repeatedly selected from the template library and entity dictionary, including:

[0121] S501, each time a training template is randomly selected from the above template library, and each time a training entity is randomly selected from the above entity dictionary.

[0122] S502, combine the above training template and the above training entity to obtain the above training data.

[0123] S503, repeat the above random sampling, random selection and combination operations multiple times to obtain multiple training data sets.

[0124] In the embodiments of this application, it is still as follows Figure 2As shown, training data can be automatically generated by combining a template library and an entity dictionary. For example, a training template can be randomly selected from the template library, and a training entity can be randomly selected from the entity dictionary to form a complete training dataset. This process is repeated continuously to obtain a large number of uniformly distributed training datasets that contain little or no noise. This data is then used to train the BERT+CRF model to obtain the final named entity recognition model.

[0125] Still Figure 2 As shown, data augmentation is performed using seed templates to generate a template library. Then, a template and an entity are randomly sampled from the template library and an entity dictionary, respectively, to form new training data. This process is repeated to augment the data, resulting in a large amount of high-quality labeled data. This invention can generate a large amount of evenly distributed, high-quality labeled training data, thus solving the problems of difficulty in obtaining original data and uneven distribution.

[0126] To obtain high-quality labeled data at minimal cost, this application improves upon the concept of remote supervision. While utilizing remote supervision for entity localization, it avoids the drawback of high noise levels associated with it by not using it for training data generation. Instead, it generates and labels training data using a template library and entity dictionary generated by the aforementioned method. Furthermore, it categorizes entity-containing corpora semantically through clustering, obtaining typical templates with different features for manual verification. This approach minimizes manual labor costs and reduces noise in the training data.

[0127] In one example, as can be seen from the above description, the training method of the named entity recognition model described above can be further divided into: a corpus data extraction method containing entities, a template library construction method based on clustering, and a training data generation method based on the template library and entity dictionary, for example:

[0128] First, entity-containing corpus data extraction methods address the issue of severe long-tail distribution in data, such as industrial data. In after-sales service scenarios, most of the data may not contain entities.

[0129] Therefore, it is necessary to extract the portion of the corpus containing entities for further processing. Here, a method based on edit-5 distance and dictionary matching is used to extract sentences containing entities from a large amount of irregular text.

[0130] Secondly, the clustering-based template library construction method refers to the need for clustering because many entities in the corpus contain the same features (semantic similarity). First, all entities are replaced with the same entity, then clustering is performed, and the sentences at each cluster center are extracted and manually processed.

[0131] The initial templates are verified. Then, data augmentation is performed on these templates to obtain the final template library. Furthermore, the training data generation method based on the template library and entity dictionary refers to randomly selecting data from the template library...

[0132] The system selects a template and then randomly selects an entity from the entity dictionary to form new labeled training data. Because of the random sampling method, this data can alleviate the severe long-tail distribution problem of the original data to some extent.

[0133] According to one or more embodiments of this application, this application provides a recognition method based on Named Entity Recognition Model 5. Figure 4 This is a flowchart illustrating a name entity recognition method based on a named entity recognition model provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the method includes:

[0134] S601, acquire the data to be identified.

[0135] S602, using a target named entity recognition model, the above-mentioned data to be recognized is processed to obtain the recognition result.

[0136] 0. Among them, the above-mentioned target named entity recognition model uses multiple training data to train the initial network.

[0137] The model is obtained by repeatedly selecting multiple training data from the template library and entity dictionary. The template library is constructed based on multiple entity corpora. The multiple entity corpora are obtained from irregular corpora. The entity corpora are corpora containing entities. The training data are data containing the entities and their tag information.

[0138] 5. Optionally, the above entity corpus is a corpus containing entities.

[0139] Optionally, the training data mentioned above is data containing the aforementioned entities and their labeling information.

[0140] The recognition method based on a named entity recognition model provided in this embodiment of the invention can, but

[0141] It is not limited to business areas where data often suffers from severe long-tail distribution and rapid entity updates.

[0142] Optionally, the above-mentioned recognition method based on the named entity recognition model can be used in the after-sales service field, specifically in the after-sales service field of the industrial or commercial sector; optionally, the data to be recognized is after-sales service data in the industrial or commercial sector.

[0143] For example, the aforementioned recognition method based on the named entity recognition model can be used to recognize after-sales service data, resulting in high-precision recognition results with good recognition performance and low annotation difficulty. It can solve the problems of long-tail distribution and rapid entity update iteration that existing named entity recognition models cannot overcome, thereby improving the entity recognition performance of the named entity recognition model in the after-sales service field and reducing annotation difficulty and maintenance costs.

[0144] Understandably, information extraction from unstructured text is a crucial task in Natural Language Processing (NLP), and Named Entity Recognition (NER) models form the foundation of this process. A well-trained NER can be defined as extracting predefined entities from unstructured corpus data and classifying them into corresponding categories (such as people, organizations, and locations).

[0145] Optionally, the irregular corpus in the embodiments of this application refers to irregular text or unstructured text, such as dialogues or documents.

[0146] Optionally, the entity corpus in this application embodiment can be text containing predefined entities. For example, entity corpus in irregular text can be obtained by using exact matching algorithm and fuzzy matching algorithm.

[0147] In one optional embodiment, after obtaining multiple entity corpora in the irregular corpus, the method further includes: marking the position of the entity in the entity corpus to obtain the marking information.

[0148] Optionally, the template library is constructed based on multiple entity corpora and contains multiple seed templates. When training the initial network model, multiple training templates, i.e. training code templates, are randomly selected from the template library multiple times. Then, by repeatedly selecting a training entity from the entity dictionary and combining the training template and the training entity, training data can be obtained.

[0149] Furthermore, using the multiple training data obtained in the embodiments of this application, an initial network model is trained to obtain a target named entity recognition model.

[0150] Optionally, in this embodiment, the initial network model can be a BERT (Bidirectional Encoder Representations from Transformer) + CRF (Conditional Random Field) model. The aforementioned target named entity recognition model is the NER model.

[0151] To address the issue of severe long-tail distribution of data in some domains, this application's embodiments extract seed templates from sentences containing entities to construct a template library. Then, combined with an entity dictionary, a large amount of labeled training data containing entities is automatically generated to train the initial network model, resulting in a target named entity recognition model. This allows for the easy deployment of named entity recognition models across various domains using a training method that requires minimal human intervention.

[0152] In this embodiment of the application, to obtain high-quality labeled data at minimal cost, an improvement is made to the concept of remote supervision. While remote supervision is used for locating entities, it is not used for generating training data, thus avoiding the drawback of high noise levels associated with remote supervision. Instead, training data is generated and labeled using a template library and entity dictionary produced by the aforementioned method. This avoids the noise problem introduced by remote supervision and reduces the accuracy requirements for remote supervision recognition, thereby alleviating to some extent the difficulties in acquiring industrial data and the severe uneven distribution of data.

[0153] According to one or more embodiments of this application, a training apparatus for a named entity recognition model is provided. Figure 5 A structural block diagram of a training device for a named entity recognition model provided in an embodiment of this application is shown below. Figure 5 As shown, the training device 500 for the aforementioned named entity recognition model includes:

[0154] The acquisition module 501 is used to acquire multiple entity corpora in the irregular corpus, wherein the aforementioned entity corpora are corpora containing entities;

[0155] Module 502 is used to construct a template library based on multiple entity corpora mentioned above;

[0156] The selection module 503 is used to repeatedly select multiple training data from the template library and entity dictionary, wherein the training data is data containing the entities and their tag information.

[0157] Training module 504 is used to train an initial network model using multiple training data sets to obtain a target named entity recognition model.

[0158] According to one or more embodiments of this application, the above-mentioned acquisition module includes:

[0159] The first matching submodule is used to use an exact matching algorithm to filter out the entity corpus and other corpus in the above-mentioned irregular corpus, wherein the above-mentioned other corpus is corpus in which it is not determined whether it contains entities;

[0160] The second matching submodule is used to use a fuzzy matching algorithm to filter out entity corpora and non-entity corpora from the other corpora mentioned above, wherein the non-entity corpora are corpora that do not contain entities.

[0161] According to one or more embodiments of this application, the above-described apparatus further includes:

[0162] The tagging module is used to tag the position of the above entities in the above entity corpus and obtain the above tagging information.

[0163] According to one or more embodiments of this application, the above-described building module includes:

[0164] The replacement submodule is used to replace entities at predetermined positions in multiple entity corpora by using multiple pre-specified replacement entities to obtain multiple replaced entity corpora. The replacement process is used to replace non-entity characters in each entity corpus with predetermined symbols.

[0165] The first determination submodule is used to determine multiple seed templates based on multiple replaced entity corpora as described above.

[0166] The construction submodule is used to process the data of each verified seed template, and to build a template library based on the multiple processed seed templates.

[0167] According to one or more embodiments of this application, the above-described determining module includes:

[0168] The clustering submodule is used to cluster multiple replaced entity corpora based on the semantics of the replaced entity corpora to obtain multiple cluster centers.

[0169] The extraction submodule is used to extract N sentences from each of the above cluster centers;

[0170] The second determining submodule is used to take the above N sentences of each of the above cluster centers as seed templates for the above cluster centers.

[0171] According to one or more embodiments of this application, the above-mentioned selected module includes:

[0172] The selection submodule is used to randomly select a training template from the template library each time, and to randomly select a training entity from the entity dictionary each time.

[0173] The combination submodule is used to combine the above training template and the above training entity to obtain the above training data;

[0174] The repeat operation submodule is used to repeat the above random sampling, random selection and combination operations multiple times to obtain multiple training data.

[0175] According to one or more embodiments of this application, a recognition device based on a named entity recognition model is provided. Figure 6 A structural block diagram of a recognition device based on a named entity recognition model provided in this application embodiment is shown below. Figure 6 As shown, the aforementioned device 600 based on the named entity recognition model includes:

[0176] Acquisition unit 601 is used to acquire data to be identified;

[0177] The recognition unit 602 is used to perform recognition processing on the data to be recognized using a target named entity recognition model to obtain a recognition result. The target named entity recognition model is obtained by training an initial network model using multiple training data. The multiple training data are repeatedly selected from a template library and an entity dictionary. The template library is constructed based on multiple entity corpora. The multiple entity corpora are obtained from unstructured corpora. The entity corpora are corpora containing entities. The training data are data containing the entities and their tag information.

[0178] In an exemplary embodiment, this application also provides an electronic device, including: a processor, and a memory connected to the processor;

[0179] The aforementioned memory stores instructions executed by the computer;

[0180] The processor executes computer execution instructions stored in the memory to implement any of the methods described above.

[0181] In an exemplary embodiment, this application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods described above.

[0182] In an exemplary embodiment, this application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the methods described above.

[0183] To implement the above embodiments, this application also provides an electronic device.

[0184] refer to Figure 7The diagram illustrates a structural schematic of an electronic device 700 suitable for implementing embodiments of this application. The electronic device 700 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 7 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0185] like Figure 7 As shown, the electronic device 700 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0186] Typically, the following devices can be connected to the I / O interface 705: input devices 707 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 707. The communication device 707 allows the electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although... Figure 7 An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0187] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 707, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined in the methods of embodiments of this application.

[0188] It should be noted that the computer-readable medium described above in this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor device or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution device or apparatus. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution device or apparatus. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0189] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0190] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0191] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0192] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0193] The units described in the embodiments of this application can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0194] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0195] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution apparatus or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor device or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

Claims

1. A training method for a named entity recognition model, characterized in that, The method includes: Obtain multiple entity corpora from an irregular corpus, wherein the entity corpus is a corpus containing entities; Multiple pre-specified replacement entities are used to replace entities at predetermined positions in multiple entity corpora to obtain multiple replaced entity corpora. The replacement process is used to replace non-entity characters in each entity corpus with predetermined symbols. Based on the semantics of the replaced entity corpus, multiple replaced entity corpora are clustered to obtain multiple cluster centers; Extract N sentences from each of the multiple cluster centers; Each of the N sentences for a given cluster center is used as a seed template for that cluster center. Each verified seed template is processed to build a template library based on the processed seed templates. Each time, a training template is randomly selected from the template library, and each time a training entity is randomly selected from the entity dictionary; The training template and the training entity are combined to obtain training data containing the entity and its labeled information; Repeat the above random sampling, random selection and combination operations multiple times to obtain multiple training data, wherein the training data is data containing the entity and the entity's label information; An initial network model is trained using multiple sets of training data to obtain a target named entity recognition model.

2. The method according to claim 1, characterized in that, The acquisition of multiple entity corpora from the irregular corpus includes: An exact matching algorithm is used to filter out the entity corpus and other corpus from the irregular corpus, wherein the other corpus is the corpus whose existence is not determined to contain entities; A fuzzy matching algorithm is used to filter out entity corpora and non-entity corpora from the other corpora, wherein the non-entity corpora are corpora that do not contain entities.

3. The method according to claim 1 or 2, characterized in that, After obtaining multiple entity corpora from the irregular corpus, the method further includes: The position of the entity in the entity corpus is marked to obtain the marking information.

4. A recognition method based on a named entity recognition model, characterized in that, The method includes: Obtain the data to be identified; A target named entity recognition model is used to process the data to be recognized, and a recognition result is obtained. The target named entity recognition model is obtained by training an initial network model using multiple training data. Multiple pre-specified replacement entities are used to replace entities at predetermined positions in multiple entity corpora, resulting in multiple replaced entity corpora. The replacement process involves replacing non-entity characters in each entity corpus with predetermined symbols. Based on the semantics of the replaced entity corpora, the multiple replaced entity corpora are clustered to obtain multiple cluster centers. N sentences are extracted from each of the multiple cluster centers, and the N sentences from each of the multiple cluster centers are... Sentences serve as seed templates for multiple cluster centers. Data processing is performed on each verified seed template to construct a template library. Each time, a training template is randomly selected from the template library, and each time a training entity is randomly selected from the entity dictionary. The training template and the training entity are combined to obtain training data containing the entity and its labeling information. This random sampling, selection, and combination operation is repeated multiple times to obtain multiple sets of training data. Multiple entity corpora are obtained from irregular corpora. The entity corpora are corpora containing entities, and the training data are data containing the entities and their labeling information.

5. A training device for a named entity recognition model, characterized in that, The device includes: The acquisition module is used to acquire multiple entity corpora from the irregular corpus, wherein the entity corpus is a corpus containing entities; Build modules, including: The replacement submodule is used to replace entities at predetermined positions in multiple entity corpora using multiple pre-specified replacement entities to obtain multiple replaced entity corpora. The replacement process is used to replace non-entity characters in each entity corpus with predetermined symbols. The clustering submodule is used to cluster multiple replaced entity corpora according to the semantics of the replaced entity corpora to obtain multiple cluster centers; The extraction submodule is used to extract N sentences from each of the multiple cluster centers; The second determining submodule is used to use the N sentences of each of the multiple cluster centers as seed templates for the multiple cluster centers; The construction submodule is used to process the data of each verified seed template, and to build a template library based on the multiple processed seed templates; Select modules, including: The selection submodule is used to randomly select a training template from the template library each time, and to randomly select a training entity from the entity dictionary each time; A combination submodule is used to combine the training template and the training entity to obtain training data containing the entity and its label information, wherein the training data is data containing the entity and its label information; The repeat operation submodule is used to repeat the above-mentioned random sampling, random selection and combination operations multiple times to obtain multiple training data; The training module is used to train an initial network model using multiple training data sets to obtain a target named entity recognition model.

6. A recognition device based on a named entity recognition model, characterized in that, The device includes: The acquisition unit is used to acquire the data to be identified. The recognition unit is used to process the data to be recognized using a target named entity recognition model to obtain a recognition result. The target named entity recognition model is obtained by training an initial network model using multiple training data. Multiple pre-specified replacement entities are used to replace entities at predetermined positions in multiple entity corpora, resulting in multiple replaced entity corpora. The replacement process involves replacing non-entity characters in each entity corpus with predetermined symbols. Based on the semantics of the replaced entity corpora, the multiple replaced entity corpora are clustered to obtain multiple cluster centers. N sentences from each cluster center are extracted, and the sentences from each cluster center are... N sentences are used as seed templates for multiple cluster centers. Data processing is performed on each verified seed template to build a template library. Each time, a training template is randomly selected from the template library, and a training entity is randomly selected from the entity dictionary. The training template and the training entity are combined to obtain training data containing the entity and its label information. The above random sampling, random selection, and combination operations are repeated multiple times to obtain multiple training data. The multiple entity corpora are obtained from irregular corpora. The entity corpora are corpora containing entities, and the training data are data containing the entities and their label information.

7. An electronic device, characterized in that, include: A processor, and a memory connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 4.

9. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 4.

Images