Named entity recognition method, device, electronic device and readable storage medium

Abstract

The present application provides a method, apparatus, electronic device and readable storage medium for named entity recognition, wherein candidate named entities are obtained by analyzing a text to be recognized based on grammatical rules, the candidate named entities in the text to be recognized are replaced with target words, and a named entity recognition model performs named entity recognition on the replaced text to be recognized to obtain an initial recognition result. Since the replaced target word is associated with the candidate named entity, the initial recognition result corresponds to the candidate named entity in the text to be recognized. Since the named entity recognition model can recognize the target word, by recognizing the text to be recognized containing the target word, it can be determined whether the candidate named entity in the text to be recognized is a named entity. When the named entity recognition model encounters a character / word that has not appeared in training, it can be realized by replacing the target word to identify whether the candidate named entity belongs to a named entity, thereby improving the recognition accuracy.

Description

Technical Field

[0001] The present application relates to the field of information technology, and more specifically, to a method, device, electronic device and readable storage medium for named entity recognition. Background Art

[0002] Named Entity Recognition (NER) refers to identifying entities with specific meanings in text, mainly including names of people, places, institutions, proper nouns, etc. Named entity recognition is the basis and hot issue of natural language understanding.

[0003] Currently, the text to be recognized is input into the NER model for recognition. During the recognition process, if a character / word that did not appear during training is encountered, the model cannot accurately give a statistical estimate of the current word, resulting in a low recognition accuracy of the model. Summary of the invention

[0004] In view of this, the present application provides a method for named entity recognition as follows:

[0005] A named entity recognition method, comprising:

[0006] Analyze the text to be recognized at least based on grammatical rules to obtain candidate named entities in the text to be recognized;

[0007] Replacing the candidate named entity in the text to be recognized with the target word to obtain the replaced text to be recognized, wherein the target word and the candidate named entity meet an association condition so that the named entity recognition model performs named entity recognition on the replaced text to be recognized to obtain an initial recognition result;

[0008] Based on the initial recognition result, it is determined whether at least one candidate named entity in the text to be recognized is a named entity, and a named entity recognition result of the text to be recognized is obtained.

[0009] Optionally, the above method, at least based on grammatical rules, analyzes the text to be recognized to obtain candidate named entities in the text to be recognized, including:

[0010] The text to be recognized is recognized based on prior knowledge and grammatical rules of natural language to obtain candidate named entities in the text to be recognized, wherein the prior knowledge at least includes entities whose occurrence times in a named entity recognition model are less than a threshold.

[0011] Optionally, the above method, at least based on grammatical rules, analyzes the text to be recognized to obtain candidate named entities in the text to be recognized, including:

[0012] Recognize the text to be recognized based on prior knowledge and grammatical rules of natural language to obtain a first named entity set consisting of at least two entities in the text to be recognized, wherein the prior knowledge at least includes entities whose number of appearances in a named entity recognition model is less than a threshold;

[0013] Acquire a named entity recognition model to perform named entity recognition on the to-be-recognized text to obtain a second named entity set, where the second named entity set includes at least one named entity;

[0014] The first named entity set and the second named entity set are compared to obtain candidate named entities, where the candidate named entities belong to the first named entity set and do not belong to the second named entity set.

[0015] Optionally, in the above method, replacing the candidate named entity in the text to be recognized with the target word to obtain the replaced text to be recognized includes at least one of the following:

[0016] Determine the entity type of the candidate named entity, replace the candidate named entity in the text to be recognized with a target word that satisfies the same entity type condition as the candidate named entity, and obtain the replaced text to be recognized;

[0017] The candidate named entities in the to-be-recognized text are replaced with target words that meet a semantic association condition with the candidate named entities to obtain the replaced text to be recognized.

[0018] Optionally, in the above method, determining the entity type of the candidate named entity includes:

[0019] Fill the candidate named entity into the named entity position of at least two entity type templates;

[0020] Analyzing the probability that the candidate named entity appears in the at least two entity type templates in accordance with natural language rules;

[0021] A first entity type template is selected in which the probability satisfies an agreed condition, and the entity type of the candidate named entity is determined to be a first entity type.

[0022] Optionally, the above method, replacing the candidate named entity in the to-be-recognized text with a target word that satisfies the same type condition as the candidate named entity, includes:

[0023] Selecting a first named entity belonging to the first entity type from a named entity set, wherein the named entity set includes named entities of at least two entity types;

[0024] The candidate named entity in the text to be recognized is replaced with the first named entity to obtain the replaced text to be recognized.

[0025] Optionally, in the above method, replacing the candidate named entity in the to-be-recognized text with a target word that satisfies a semantic association condition with the candidate named entity includes:

[0026] Removing candidate named entities from the text to be recognized to obtain the text to be recognized containing blanks;

[0027] Predicting a target word that may appear in the gap in the text to be recognized containing the gap based on the context of the gap in the text to be recognized containing the gap;

[0028] The target word is added into the blank to obtain the replaced text to be recognized.

[0029] A named entity recognition device, comprising:

[0030] An analysis module, configured to analyze the text to be recognized based at least on grammatical rules to obtain candidate named entities in the text to be recognized;

[0031] A processing module, used for replacing the candidate named entities in the text to be recognized with the target word to obtain the replaced text to be recognized, wherein the target word and the candidate named entity meet the association condition, so that the named entity recognition model performs named entity recognition on the replaced text to be recognized to obtain an initial recognition result;

[0032] A judgment module is used to determine whether at least one candidate named entity in the text to be recognized is a named entity based on the initial recognition result, and obtain a named entity recognition result of the text to be recognized.

[0033] An electronic device, comprising: a memory and a processor;

[0034] Wherein, the memory stores a processing program;

[0035] The processor is used to load and execute the processing program stored in the memory to implement each step of the named entity recognition method as described in any one of the above items.

[0036] A readable storage medium stores a computer program, wherein the computer program is called and executed by a processor to implement each step of the named entity recognition method as described in any one of the above items.

[0037] It can be seen from the above technical solution that the present application provides a method for named entity recognition, in which the text to be recognized is analyzed at least based on grammatical rules to obtain candidate named entities therein, and the candidate named entities in the text to be recognized are replaced with target words. Since the replaced target word is associated with the candidate named entity, the target word is used to replace the candidate named entity in the text to be recognized, so that the named entity recognition model performs named entity recognition on the replaced text to be recognized to obtain an initial recognition result, and the initial recognition result is related to the candidate named entities in the text to be recognized. If the target word in the initial recognition result is a named entity, then the corresponding candidate named entity in the text to be recognized is determined to be a named entity. Otherwise, the corresponding candidate named entity in the text to be recognized is not a named entity. Since the named entity recognition model can recognize the target word, by recognizing the text to be recognized containing the target word, it can be determined whether the candidate named entity in the text to be recognized is a named entity. When the named entity recognition model encounters a character / word that has not appeared in training, it can be realized by replacing the target word to identify whether the candidate named entity belongs to a named entity, thereby improving the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work.

[0039] Figure 1 This is a flowchart of a method for named entity recognition embodiment 1 provided by the present application;

[0040] Figure 2 This is a flowchart of a method for named entity recognition embodiment 2 provided by the present application;

[0041] Figure 3 is a flowchart of Embodiment 3 of a named entity recognition method provided by the present application;

[0042] Figure 4 is a flowchart of Embodiment 4 of a named entity recognition method provided by the present application;

[0043] Figure 5 is a flowchart of Embodiment 5 of a named entity recognition method provided by the present application;

[0044] Figure 6 is a flowchart of Embodiment 6 of a named entity recognition method provided by the present application;

[0045] Figure 7It is a structural diagram of an embodiment of a named entity recognition device provided by the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] like Figure 1 The figure is a flowchart of a method for named entity recognition embodiment 1 provided by the present application. The method is applied to an electronic device and includes the following steps:

[0048] Step S101: Analyze the text to be recognized at least based on grammatical rules to obtain candidate named entities in the text to be recognized;

[0049] The text to be recognized may be a sentence.

[0050] The electronic device is preset with grammar rules and other contents for analyzing the text to be recognized to obtain candidate named entities therein.

[0051] The text to be recognized may include one or more named entities.

[0052] Since the existing named entity recognition (NER) model may not be able to recognize one or several named entities in the text to be recognized, in the present application, the text to be recognized is analyzed in advance based on grammatical rules to obtain candidate named entities that may be named entities. In subsequent steps, the named entity recognition model is combined to further determine whether the candidate named entity is a true named entity.

[0053] Step S102: replacing the candidate named entities in the text to be recognized with the target words to obtain the replaced text to be recognized, so that the named entity recognition model performs named entity recognition on the replaced text to be recognized to obtain an initial recognition result;

[0054] Wherein, the target word and the candidate named entity meet an association condition.

[0055] Wherein, a target word associated with the candidate named entity is determined, and the target word is a word that can be recognized by a named entity recognition model.

[0056] Specifically, the association conditions between the target word and the candidate named entity include: semantic association and / or belonging to the same entity type.

[0057] After replacing the candidate named entity in the text to be recognized with the target word, a replaced text to be recognized is obtained, and the text to be recognized contains the target word but does not contain the candidate named entity.

[0058] The named entity recognition model performs named entity recognition on the replaced text to be recognized to obtain the named entities therein.

[0059] The initial recognition result is a result of recognizing the named entities in the replaced text to be recognized, and the initial recognition result includes the named entities contained in the replaced text to be recognized.

[0060] Among them, the named entity recognition model involved in the present application is a trained named entity recognition model, which is trained based on set training samples. The training samples mark the named entities and the entity types of the named entities, etc. The preset named entities, templates, etc. involved in subsequent embodiments are all obtained based on the training samples.

[0061] Step S103: determining whether at least one candidate named entity in the text to be recognized is a named entity based on the initial recognition result, and obtaining a named entity recognition result of the text to be recognized.

[0062] The initial recognition result includes the replaced named entities in the text to be recognized.

[0063] If the named entity in the text to be recognized contains the target word, it indicates that the target word is a named entity; if the named entity in the text to be recognized does not contain the target word, it indicates that the target word is not a named entity.

[0064] Here, since the target word is a candidate named entity to be replaced in the text to be recognized, the target word and the candidate named entity have the same properties in the text to be recognized.

[0065] Specifically, if the target word in the initial recognition result is a named entity, the candidate named entity that the target word replaces in the text to be recognized is also a named entity; otherwise, the target word is not a named entity, and its corresponding candidate named entity is not a named entity either.

[0066] For example, the text to be recognized is "I heard that Lord Linghu is in very good health". The analysis shows that the candidate named entity is "Lord Linghu". "Lord Linghu" is replaced with "Lord Wang", and the text to be recognized after the replacement is "I heard that Lord Wang is in very good health". The named entity recognition model performs named entity recognition on "I heard that Lord Wang is in very good health" and finds that the named entity is "Lord Wang". Accordingly, "Lord Linghu" in the text to be recognized is a named entity.

[0067] In summary, in a named entity recognition method provided by the present embodiment, the text to be recognized is analyzed at least based on grammatical rules to obtain candidate named entities therein, and the candidate named entities in the text to be recognized are replaced with target words. Since the replaced target word is associated with the candidate named entity, the target word is used to replace the candidate named entity in the text to be recognized, so that the named entity recognition model performs named entity recognition on the replaced text to be recognized to obtain an initial recognition result, and the initial recognition result is related to the candidate named entities in the text to be recognized. For example, in the initial recognition result, the target word is a named entity, and accordingly, the corresponding candidate named entity in the text to be recognized is determined to be a named entity. Otherwise, the corresponding candidate named entity in the text to be recognized is not a named entity. Since the named entity recognition model can recognize the target word, by recognizing the text to be recognized containing the target word, it can be determined whether the candidate named entity in the text to be recognized is a named entity. When the named entity recognition model encounters a character / word that has not appeared in training, it can be realized by replacing the target word to identify whether the candidate named entity belongs to a named entity, thereby improving the recognition accuracy.

[0068] like Figure 2 The figure is a flowchart of Embodiment 2 of a method for named entity recognition provided by the present application, and the method comprises the following steps:

[0069] Step S201: Recognize the text to be recognized based on prior knowledge and grammatical rules of natural language to obtain candidate named entities in the text to be recognized;

[0070] The prior knowledge at least includes entities whose occurrence times in the named entity recognition model are less than a threshold.

[0071] The prior knowledge includes a dictionary, which may include a Chinese surname dictionary, etc. The prior knowledge includes entities that cannot be accurately recognized by an existing named entity recognition model.

[0072] For example, entities not involved in training the named entity recognition model, or low-frequency long-tail entities involved in training the named entity recognition model.

[0073] Among them, since named entities are entities with specific meanings, such as names of people, places, organizations, proper nouns, etc., and these entities are generally nouns, we can perform sentence component analysis on the text to be recognized based on the grammatical rules of natural language to obtain the nouns therein, and the name can be used as a candidate named entity.

[0074] The grammatical rules also include specific composition rules corresponding to named entities, such as the composition rule of "surname + title". The grammatical rules of natural language are used to identify the content in the text to be identified that conforms to the composition rule.

[0075] Among them, the use of the prior knowledge and the grammatical rules of natural language can identify candidate named entities that may be named entities in the text to be recognized. In the subsequent steps, the candidate named entities are further determined. Compared with directly using the named entity recognition model to perform named entity recognition on the text to be recognized, words / characters that did not appear in the training process of the named entity recognition model will not be missed, thereby improving the recognition accuracy.

[0076] Step S202: replacing the candidate named entities in the text to be recognized with the target words to obtain the replaced text to be recognized, so that the named entity recognition model performs named entity recognition on the replaced text to be recognized to obtain an initial recognition result;

[0077] Step S203: determining whether at least one candidate named entity in the text to be recognized is a named entity based on the initial recognition result, and obtaining a named entity recognition result of the text to be recognized.

[0078] Among them, steps S202-203 are consistent with steps S102-103 in Example 1 and are not described in detail in this embodiment.

[0079] In summary, in a method for named entity recognition provided by the present embodiment, the text to be recognized is analyzed at least based on grammatical rules to obtain candidate named entities in the text to be recognized, including: the text to be recognized is recognized based on prior knowledge and grammatical rules of natural language to obtain candidate named entities in the text to be recognized, and the prior knowledge at least includes entities whose number of appearances in the named entity recognition model is less than a threshold. In this scheme, the use of prior knowledge and grammatical rules of natural language can identify candidate named entities in the text to be recognized that may be named entities, and the candidate named entities are further determined in subsequent steps. Compared with directly using a named entity recognition model to perform named entity recognition on the text to be recognized, words / characters that did not appear in the training process of the named entity recognition model will not be omitted, thereby improving the recognition accuracy.

[0080] like Figure 3 The figure is a flowchart of Embodiment 3 of a method for named entity recognition provided by the present application, and the method comprises the following steps:

[0081] Step S301: Recognize the text to be recognized based on prior knowledge and grammatical rules of natural language to obtain a first named entity set consisting of at least two entities in the text to be recognized;

[0082] Among them, the prior knowledge at least includes entities that appear less than a threshold number of times in the named entity recognition model.

[0083] Among them, this step S301 is the same as step S201 in Embodiment 2, and for a detailed description, refer to the explanation of step S201.

[0084] Among them, based on the prior knowledge and the grammar rules of natural language, the text to be recognized is recognized to obtain a first named entity set, and this set contains two or more entities obtained based on the prior knowledge and the grammar rules of natural language.

[0085] Step S302: Obtain a second named entity set obtained by the named entity recognition model for the text to be recognized;

[0086] Among them, the second named entity set includes at least one named entity.

[0087] Use the named entity recognition model to perform named entity recognition on the text to be recognized to obtain the named entities that the named entity recognition model can recognize.

[0088] Among them, the named entity recognition model may be able to recognize all the named entities in the text to be recognized, or may only be able to recognize some of the named entities.

[0089] Step S303: Compare the first named entity set and the second named entity set to obtain candidate named entities, and the candidate named entities belong to the first named entity set and do not belong to the second named entity set;

[0090] Among them, compare the first named entity set with the second named entity set to determine the candidate named entities that belong to the first named entity set but do not belong to the second named entity set.

[0091] Among them, the candidate named entities cannot be recognized by the named entity recognition model, and it is necessary to further determine whether the candidate named entities are real named entities.

[0092] For example, if the first named entity set includes "Lord He" and "the capital city", and the second named entity set includes "the capital city", then the candidate named entity is "Lord He", and in subsequent steps, it is further confirmed whether the candidate named entity is a real named entity.

[0093] Among them, using the prior knowledge and the grammar rules of natural language can recognize the entities in the text to be recognized that may be named entities, and eliminate the named entities that the named entity recognition model can first recognize in the text to be recognized. The remaining entities are those that need to be further determined whether they are named entities.

[0094] In this solution, there is no need to further confirm the named entities in the text to be recognized that can be directly recognized by the named entity recognition model, thereby reducing the amount of data processing for executing this method.

[0095] Step S304: replacing the candidate named entities in the text to be recognized with the target words to obtain the replaced text to be recognized, so that the named entity recognition model performs named entity recognition on the replaced text to be recognized to obtain an initial recognition result;

[0096] Step S305: determining whether at least one candidate named entity in the text to be recognized is a named entity based on the initial recognition result, and obtaining a named entity recognition result of the text to be recognized.

[0097] Among them, steps S304-305 are consistent with steps S102-103 in Example 1 and are not described in detail in this embodiment.

[0098] In summary, in a method for named entity recognition provided by this embodiment, the text to be recognized is analyzed at least based on grammatical rules to obtain candidate named entities in the text to be recognized, including: recognizing the text to be recognized based on prior knowledge and grammatical rules of natural language to obtain a first named entity set consisting of at least two named entities in the text to be recognized, wherein the prior knowledge at least includes entities whose number of appearances in the named entity recognition model is less than a threshold; obtaining a named entity recognition model to perform named entity recognition on the text to be recognized to obtain a second named entity set, wherein the second named entity set includes at least one named entity; comparing the first named entity set and the second named entity set to obtain candidate named entities, wherein the candidate named entities belong to the first named entity set and do not belong to the second named entity set. In this scheme, the prior knowledge and grammatical rules of natural language can be used to identify entities in the text to be recognized that may be named entities, and the named entities that can be recognized by the named entity recognition model in the text to be recognized are eliminated, and the remaining entities need to be further determined whether they are named entities, and there is no need to further confirm the named entities that can be directly recognized by the named entity recognition model in the text to be recognized, thereby reducing the amount of data processing for executing this method.

[0099] like Figure 4 The figure is a flowchart of Embodiment 4 of a method for named entity recognition provided by the present application, and the method comprises the following steps:

[0100] Step S401: Analyze the text to be recognized based on at least grammatical rules to obtain candidate named entities in the text to be recognized;

[0101] Among them, step S401 is consistent with step S101 in embodiment 1, and is not described in detail in this embodiment.

[0102] Step S402: Determine the entity type of the candidate named entity;

[0103] Among them, named entities have various types, such as personal names, place names, organization names, and proper nouns.

[0104] Specifically, the entity type of the candidate named entity is determined based on the named entity template.

[0105] Specifically, step S402 includes:

[0106] Step S4021: Fill the candidate named entity into the named entity position of at least two entity type templates;

[0107] Among them, corresponding entity type templates are set for various entity types, such as person name template, place name template, organizational structure template, etc.

[0108] The template is composed of one sentence or a set of multiple sentences, and the position of the named entity in the template is blank.

[0109] Specifically, the candidate named entity is added to each entity type template to determine the entity type to which the candidate named entity belongs.

[0110] Step S4022: analyzing the probability of the candidate named entity appearing in the at least two entity type templates in accordance with natural language rules;

[0111] The probability of the candidate named entity appearing in the entity type template is analyzed based on natural language rules.

[0112] Specifically, the probability that the sentence into which the candidate named entity is added complies with the rules of natural language is analyzed by combining the context of the blank position in the entity type template with the filled candidate named entity.

[0113] The greater the probability, the higher the degree of matching between the entity type template added to the candidate named entity and the candidate named entity, and the greater the possibility that the candidate named entity belongs to the entity type.

[0114] The entity type template may be a training sample for training a named entity recognition model, and the training sample may include a named entity and an entity type.

[0115] Step S4023: Selecting a first entity type template whose probability meets an agreed condition, and determining that the entity type of the candidate named entity is a first entity type.

[0116] Among them, the entity type template with a greater probability is selected as the template of the entity type corresponding to the candidate named entity.

[0117] For example, the probability that the candidate named entity has the entity type template of a name is 90%, the probability that the candidate named entity has the entity type template of a place name is 50%, and the probability that the candidate named entity has the entity type template of an organization is 12%, and it is determined that the candidate named entity is a name.

[0118] Step S403: replacing the candidate named entity in the text to be recognized with a target word that satisfies the same entity type condition as the candidate named entity, to obtain a replaced text to be recognized;

[0119] After the entity type of the candidate named entity is determined, a target word is selected from the vocabulary corresponding to the entity type, and the target word replaces the candidate named entity in the text to be recognized to obtain the replaced text to be recognized.

[0120] The step S403 specifically includes:

[0121] Step S4031: selecting a first named entity belonging to the first entity type from a named entity set, wherein the named entity set includes named entities of at least two entity types;

[0122] A named entity set is set for each entity type, and the named entities in the named entity set can be directly recognized by the named entity recognition type in the corresponding entity type template.

[0123] After determining that the candidate named entity is of the first entity type, one is selected from the named entities corresponding to the first entity type.

[0124] For example, it is determined that the entity type of "Mr. Linghu" is a name, and for the entity type of name, there are named entities such as "Mr. Wang" and "Wang Dawei", and one is selected from them, such as "Mr. Wang".

[0125] Step S4032: Replace the candidate named entity in the text to be recognized with the first named entity to obtain the replaced text to be recognized.

[0126] Here, the candidate named entity in the text to be recognized is replaced with the selected first named entity to obtain the replaced text to be recognized.

[0127] For example, the text to be recognized is "I heard that Lord Linghu is in very good health", in which the candidate named entity is "Lord Linghu", and it is determined that the first named entity corresponding to "Lord Linghu" is "Lord Wang", and the text to be recognized after replacement is "I heard that Lord Wang is in very good health".

[0128] Among them, the target word is obtained based on the context analysis of the candidate named entities in the text to be recognized and the entity type template. The replaced target word belongs to the same entity type as the word in the text to be recognized. Then the named entity recognition model recognizes that the target word in the replaced text to be recognized is a named entity, which means that in the current context, the possibility that the position is a named entity is high. Therefore, based on the recognition result of the replaced text to be recognized, it can be inferred whether the candidate named entity in the text to be recognized is a true named entity.

[0129] Then, the named entity recognition model performs named entity recognition on the replaced text to be recognized to obtain an initial recognition result.

[0130] Step S404: determining whether at least one candidate named entity in the text to be recognized is a named entity based on the initial recognition result, and obtaining a named entity recognition result of the text to be recognized.

[0131] Among them, step S404 is consistent with step S103 in embodiment 1, and is not described in detail in this embodiment.

[0132] In summary, in a method for named entity recognition provided by the present embodiment, the candidate named entity in the text to be recognized is replaced with a target word to obtain the replaced text to be recognized, including: determining the entity type of the candidate named entity, replacing the candidate named entity in the text to be recognized with a target word that satisfies the same entity type condition as the candidate named entity, and obtaining the replaced text to be recognized. In this scheme, based on the candidate named entity in the text to be recognized, the entity type to which it belongs is determined in combination with the context, and the candidate named entity is replaced with a target word of the same entity type. If the replaced text to be recognized recognizes that the target word is a named entity, it means that in the current context environment, the possibility that the position is a named entity is high. Therefore, based on the recognition result of the replaced text to be recognized, it can be inferred whether the candidate named entity in the text to be recognized is a true named entity.

[0133] like Figure 5 The figure is a flowchart of Embodiment 5 of a method for named entity recognition provided by the present application, and the method comprises the following steps:

[0134] Step S501: Analyze the text to be recognized at least based on grammatical rules to obtain candidate named entities in the text to be recognized;

[0135] Among them, step S501 is consistent with step S101 in embodiment 1, and is not described in detail in this embodiment.

[0136] Step S502: replacing the candidate named entities in the text to be recognized with target words that meet the semantic association condition with the candidate named entities, to obtain the replaced text to be recognized;

[0137] The candidate named entity is replaced with a semantically related target word to obtain a replaced text to be recognized.

[0138] In this embodiment, target words are selected for candidate named entities from another perspective.

[0139] Wherein, the step S502 includes:

[0140] Step S5021: removing candidate named entities from the text to be recognized, and obtaining the text to be recognized containing spaces;

[0141] The candidate named entities in the text to be recognized are removed, specifically by masking, to obtain the text to be recognized containing blanks.

[0142] For example, the text to be recognized is "Children like to read with adults", and the candidate named entity obtained by analysis is "and adults", then the text to be recognized containing the space is "Children like to read with____".

[0143] For example, the text to be recognized is "I heard that Lord Linghu is in very good health", and the candidate named entity obtained by analysis is "Lord Linghu", then the text to be recognized containing a blank is "I heard that ____ is in very good health".

[0144] Step S5022: predicting a target word that may appear in the gap in the text to be recognized containing the gap based on the context of the gap in the text to be recognized containing the gap;

[0145] Specifically, a cloze test algorithm is used to predict the target word that may appear in the gap based on the context of the gap in the text to be recognized that contains the gap.

[0146] Specifically, the cloze algorithm adopts a bidirectional prediction model, which uses a backward prediction method before the vacancy and a forward prediction method after the vacancy, and intersects the results of the two predictions to select the word with the highest probability of occurrence as the target word.

[0147] The predicted target word is predicted based on the context of the vacancy, and the target word is also associated with subsequent named entities, such as semantic association.

[0148] Step S5023: Add the target word into the blank to obtain the replaced text to be recognized.

[0149] Among them, the target word predicted in the foregoing step is added to the blank space to obtain the text to be recognized after replacement.

[0150] For example, for the blank space in the text to be recognized "Children like to ____ read together" containing a blank space, the predicted target word is "with parents", then the text to be recognized after replacement is "Children like to read together with parents".

[0151] In subsequent steps, the named entity recognition model performs named entity recognition on the text to be recognized with the candidate named entity replaced by the target word.

[0152] Among them, the target word is obtained through context analysis of the candidate named entity in the text to be recognized. The replaced target word is related to the candidate named entity. If the named entity recognition model recognizes that the target word in the text to be recognized after replacement is a named entity, it can be stated that in the current context environment, the possibility of this position being a named entity is relatively high. Therefore, based on the recognition result of the text to be recognized after replacement, it can be inferred whether the candidate named entity in the text to be recognized is a true named entity.

[0153] For example, for the named entity recognition in the text to be recognized after replacement, it is recognized that "with parents" among them does not belong to a named entity. Correspondingly, the candidate named entity "and adults" is not a named entity either.

[0154] Then, the named entity recognition model performs named entity recognition on the text to be recognized after replacement to obtain an initial recognition result.

[0155] Step S503: Based on the initial recognition result, determine whether at least one candidate named entity in the text to be recognized is a named entity, and obtain the named entity recognition result of the text to be recognized.

[0156] Among them, step S503 is the same as step S103 in Embodiment 1, and will not be elaborated in this embodiment.

[0157] In summary, a named entity recognition method provided in this embodiment replaces the candidate named entity in the text to be recognized with a target word to obtain the text to be recognized after replacement, including: replacing the candidate named entity in the text to be recognized with a target word that satisfies the semantic association condition with the candidate named entity to obtain the text to be recognized after replacement. In this solution, based on the candidate named entity in the text to be recognized, the target word that can replace the candidate named entity is determined in combination with the context. If the target word recognized in the text to be recognized after replacement is a named entity, it indicates that in the current context environment, the possibility of this position being a named entity is relatively high. Therefore, based on the recognition result of the text to be recognized after replacement, it can be inferred whether the candidate named entity in the text to be recognized is a true named entity.

[0158] like Figure 6 The figure is a flowchart of Embodiment 6 of a method for named entity recognition provided by the present application, and the method comprises the following steps:

[0159] Step S601: Analyze the text to be recognized at least based on grammatical rules to obtain candidate named entities in the text to be recognized;

[0160] Among them, step S601 is consistent with step S101 in embodiment 1, and is not described in detail in this embodiment.

[0161] Step S602: determining the entity type of the candidate named entity, replacing the candidate named entity in the text to be recognized with a target word that satisfies the same entity type condition as the candidate named entity, and obtaining the replaced text to be recognized;

[0162] Among them, step S602 is consistent with step S402 in Example 4. The specific process refers to the explanation of step S402 in Example 4, and will not be repeated in this embodiment.

[0163] Step S603: replacing the candidate named entities in the text to be recognized with target words that meet the semantic association condition with the candidate named entities, to obtain the replaced text to be recognized;

[0164] Among them, step S603 is consistent with step S502 in Example 5. The specific process refers to the explanation of step S502 in Example 5 and is not repeated in this embodiment.

[0165] Then, the named entity recognition model performs named entity recognition on the replaced text to be recognized, and obtains an initial recognition result;

[0166] The named entity recognition model performs named entity recognition on the replaced text to be recognized obtained in step S602 and step S603 respectively, and obtains two initial recognition results.

[0167] Step S604: determining whether at least one candidate named entity in the text to be recognized is a named entity based on the initial recognition result, and obtaining a named entity recognition result of the text to be recognized.

[0168] Among them, in this scheme, two methods are used to replace the candidate named entities in the text to be recognized respectively, and two replaced texts to be recognized are obtained. The named entity recognition model performs named entity recognition on the two replaced texts to be recognized respectively, and obtains two initial recognition results.

[0169] The two initial recognition results are judged. When both initial recognition results are true, that is, the target words in the two initial recognition results are named entities, it is determined that the candidate named entity is a named entity. Otherwise, the candidate named entity is not a named entity.

[0170] In summary, in a named entity recognition method provided in this embodiment, the candidate named entities in the text to be recognized are replaced with target words to obtain the replaced text to be recognized, including: determining the entity type of the candidate named entity, replacing the candidate named entity in the text to be recognized with a target word that satisfies the same entity type condition as the candidate named entity, and obtaining the replaced text to be recognized; replacing the candidate named entity in the text to be recognized with a target word that satisfies the semantic association condition with the candidate named entity, and obtaining the replaced text to be recognized. In this scheme, the candidate named entities in the text to be recognized are replaced from two dimensions, and are recognized based on the named entity recognition model respectively, and the two initial recognition results are combined to determine whether the candidate named entity corresponding to the target word is a named entity, and the recognition accuracy is high.

[0171] Corresponding to the above-mentioned embodiment of a named entity recognition method provided by the present application, the present application also provides an embodiment of a device applying the named entity recognition method.

[0172] like Figure 7 The structure diagram of a named entity recognition device embodiment provided by the present application is shown, and the device includes the following structures: an analysis module 701, a processing module 702 and a judgment module 703;

[0173] The analysis module 701 is used to analyze the text to be recognized based on at least grammatical rules to obtain candidate named entities in the text to be recognized;

[0174] The processing module 702 is used to replace the candidate named entity in the text to be recognized with the target word to obtain the replaced text to be recognized, and the target word and the candidate named entity meet the association condition, so that the named entity recognition model performs named entity recognition on the replaced text to be recognized to obtain an initial recognition result;

[0175] The judgment module 703 is used to determine whether at least one candidate named entity in the text to be recognized is a named entity based on the initial recognition result, and obtain a named entity recognition result of the text to be recognized.

[0176] Optional analysis modules include:

[0177] The first recognition unit is used to recognize the text to be recognized based on prior knowledge and grammatical rules of natural language to obtain candidate named entities in the text to be recognized, and the prior knowledge at least includes entities whose number of appearances in the named entity recognition model is less than a threshold.

[0178] Optional, analysis modules, including:

[0179] A second recognition unit is used to recognize the text to be recognized based on prior knowledge and grammatical rules of natural language to obtain a first named entity set consisting of at least two entities in the text to be recognized, wherein the prior knowledge at least includes entities whose number of appearances in the named entity recognition model is less than a threshold;

[0180] An acquisition unit, configured to acquire a named entity recognition model to perform named entity recognition on the to-be-recognized text to obtain a second named entity set, wherein the second named entity set includes at least one named entity;

[0181] The comparison unit is used to compare the first named entity set and the second named entity set to obtain a candidate named entity, where the candidate named entity belongs to the first named entity set and does not belong to the second named entity set.

[0182] Optionally, the processing module includes at least one of the following:

[0183] A first replacement unit is used to determine the entity type of the candidate named entity, replace the candidate named entity in the to-be-recognized text with a target word that satisfies the same entity type condition as the candidate named entity, and obtain the replaced text to be recognized;

[0184] The second replacement unit is used to replace the candidate named entities in the text to be recognized with target words that meet the semantic association condition with the candidate named entities to obtain the replaced text to be recognized.

[0185] Optionally, the processing module further includes:

[0186] An analysis unit is used to fill the candidate named entity into the position of the named entity of at least two entity type templates; analyze the probability that the candidate named entity appears in the at least two entity type templates in accordance with the rules of natural language; select the first entity type template whose probability meets the agreed conditions, and determine that the entity type of the candidate named entity is the first entity type.

[0187] Optionally, the first replacement unit is specifically used for:

[0188] Selecting a first named entity belonging to the first entity type from a named entity set, wherein the named entity set includes named entities of at least two entity types;

[0189] The candidate named entity in the text to be recognized is replaced with the first named entity to obtain the replaced text to be recognized.

[0190] Optionally, the second replacement unit is specifically used for:

[0191] Removing candidate named entities from the text to be recognized to obtain the text to be recognized containing blanks;

[0192] Predicting a target word that may appear in the gap in the text to be recognized containing the gap based on the context of the gap in the text to be recognized containing the gap;

[0193] The target word is added into the blank to obtain the replaced text to be recognized.

[0194] Among them, the specific explanation of the functions of each part in the named entity recognition device can be referred to the explanation in the method embodiment, which will not be repeated in this embodiment.

[0195] In summary, the present embodiment provides a named entity recognition device, which analyzes the text to be recognized based on at least grammatical rules to obtain candidate named entities therein, and replaces the candidate named entities in the text to be recognized with the target word. Since the replaced target word is associated with the candidate named entity, the target word is used to replace the candidate named entity in the text to be recognized, so that the named entity recognition model performs named entity recognition on the replaced text to be recognized to obtain an initial recognition result, and the initial recognition result is related to the candidate named entities in the text to be recognized. For example, in the initial recognition result, the target word is a named entity, and accordingly, the corresponding candidate named entity in the text to be recognized is determined to be a named entity. Otherwise, the corresponding candidate named entity in the text to be recognized is not a named entity. Since the named entity recognition model can recognize the target word, by recognizing the text to be recognized containing the target word, it can be determined whether the candidate named entity in the text to be recognized is a named entity. When the named entity recognition model encounters a character / word that has not appeared in training, it can be realized by replacing the target word to identify whether the candidate named entity belongs to a named entity, thereby improving the recognition accuracy.

[0196] Corresponding to the above-mentioned embodiment of a named entity recognition method provided by the present application, the present application also provides an electronic device and a readable storage medium corresponding to the named entity recognition method.

[0197] The electronic device includes: a memory and a processor;

[0198] Wherein, the memory stores a processing program;

[0199] The processor is used to load and execute the processing program stored in the memory to implement each step of the named entity recognition method as described in any one of the above items.

[0200] For details on how to implement the named entity recognition method in the electronic device, please refer to the aforementioned named entity recognition method embodiment.

[0201] The readable storage medium stores a computer program thereon, and the computer program is called and executed by a processor to implement each step of the method for named entity recognition as described in any one of the above claims.

[0202] Specifically, the computer program stored in the readable storage medium executes the method for implementing the named entity recognition method, and reference may be made to the aforementioned embodiment of the method for named entity recognition.

[0203] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device provided in the embodiment, since it corresponds to the method provided in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0204] The above description of the embodiments provided enables professionals and technicians in the field to implement or use the present application. Various modifications to these embodiments will be apparent to professionals and technicians in the field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features provided herein.

Claims

1. A method for named entity recognition, comprising: Analyze the text to be recognized based on prior knowledge and grammatical rules of natural language to obtain candidate named entities in the text to be recognized, wherein the prior knowledge at least includes entities whose occurrence times in the named entity recognition model are less than a threshold; Replacing the candidate named entity in the text to be recognized with the target word to obtain the replaced text to be recognized, wherein the target word and the candidate named entity satisfy an association condition, so that the named entity recognition model performs named entity recognition on the replaced text to be recognized to obtain an initial recognition result; Based on the initial recognition result, it is determined whether at least one candidate named entity in the text to be recognized is a named entity, and a named entity recognition result of the text to be recognized is obtained.

2. The method according to claim 1, analyzing the text to be recognized based on prior knowledge and grammatical rules of natural language to obtain candidate named entities in the text to be recognized, comprising: Recognize the text to be recognized based on prior knowledge and grammatical rules of natural language to obtain a first named entity set consisting of at least two entities in the text to be recognized; Acquire a named entity recognition model to perform named entity recognition on the to-be-recognized text to obtain a second named entity set, where the second named entity set includes at least one named entity; The first named entity set and the second named entity set are compared to obtain candidate named entities, where the candidate named entities belong to the first named entity set and do not belong to the second named entity set.

3. According to the method of claim 1, the step of replacing the candidate named entities in the text to be recognized with the target words to obtain the replaced text to be recognized comprises at least one of the following: Determine the entity type of the candidate named entity, replace the candidate named entity in the text to be recognized with a target word that satisfies the same entity type condition as the candidate named entity, and obtain the replaced text to be recognized; The candidate named entities in the to-be-recognized text are replaced with target words that meet a semantic association condition with the candidate named entities to obtain the replaced text to be recognized.

4. The method according to claim 3, wherein determining the entity type of the candidate named entity comprises: Fill the candidate named entity into the named entity position of at least two entity type templates; Analyzing the probability that the candidate named entity appears in the at least two entity type templates in accordance with natural language rules; A first entity type template is selected in which the probability satisfies an agreed condition, and the entity type of the candidate named entity is determined to be a first entity type.

5. The method according to claim 4, wherein the candidate named entity in the to-be-recognized text is replaced with a target word that satisfies the same type condition as the candidate named entity, comprising: Selecting a first named entity belonging to the first entity type from a named entity set, wherein the named entity set includes named entities of at least two entity types; The candidate named entity in the text to be recognized is replaced with the first named entity to obtain the replaced text to be recognized.

6. According to the method of claim 3, the step of replacing the candidate named entity in the to-be-recognized text with a target word that satisfies a semantic association condition with the candidate named entity comprises: Removing candidate named entities from the text to be recognized to obtain the text to be recognized containing blanks; Predicting a target word that may appear in the gap in the text to be recognized containing the gap based on the context of the gap in the text to be recognized containing the gap; The target word is added into the blank to obtain the replaced text to be recognized.

7. A named entity recognition device, comprising: An analysis module, configured to analyze the text to be recognized based on prior knowledge and grammatical rules of natural language to obtain candidate named entities in the text to be recognized, wherein the prior knowledge at least includes entities whose occurrence times in the named entity recognition model are less than a threshold; A processing module, used for replacing the candidate named entities in the text to be recognized with the target word to obtain the replaced text to be recognized, wherein the target word and the candidate named entity meet the association condition, so that the named entity recognition model performs named entity recognition on the replaced text to be recognized to obtain an initial recognition result; A judgment module is used to determine whether at least one candidate named entity in the text to be recognized is a named entity based on the initial recognition result, and obtain a named entity recognition result of the text to be recognized.

8. An electronic device, comprising: Memory, processor; Wherein, the memory stores a processing program; The processor is used to load and execute the processing program stored in the memory to implement each step of the named entity recognition method as described in any one of claims 1-6.

9. A readable storage medium having a computer program stored thereon, wherein the computer program is called and executed by a processor to implement the steps of the named entity recognition method according to any one of claims 1 to 6.

Images