Named entity recognition method and device
By performing name matching and correlation feature analysis of the identified named entities and empirical named entities in the entity knowledge base, the problem of low classification accuracy of named entities is solved, and high accuracy recognition of named entities is achieved.
Patent Information
- Application Number
- CN202010184831.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-03-17
- Publication Date
- 2025-01-07
- Estimated Expiration
- 2040-03-17
AI Technical Summary
In the prior art, the accuracy of named entities classification is low, especially when the same named entity appears in different expressions, which leads to difficulty in identifying.
By matching the named entity to be identified with the empirical named entity in the entity knowledge base, a candidate empirical named entity is obtained, and similarity is recalculated based on similarity and preset correlation characteristics to classify named entities.
Improve the accuracy of named entities classification, and enhance the recognition ability of named entities through comprehensive recognition from the dimensions of name itself and non-name itself.
Smart Images

Figure CN113408283B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method and device for identifying a named entity. Background Art
[0002] In real life, different expressions may be used for the same named entity. For example, for an eye drop that relieves eye fatigue, some doctors will write sodium hyaluronate eye drops in the medical book, and some doctors may directly use the product brand to imply that it is a certain type of sodium hyaluronate eye drops. Since sodium hyaluronate eye drops are a type of artificial tears, some doctors may write artificial tears. This brings difficulties to the recognition of named entities, and normalization processing has become the key to solving this problem. At present, the method for normalizing named entities is mainly: matching the entity names in the text to be identified with the entity names in the entity knowledge base one by one, calculating the similarity of the entity names of the two, and classifying the entity names in the text to be identified into the entity names in the entity knowledge base with the greatest similarity. However, sometimes although the similarity of two entity names is large, only one word is different, but they may belong to two different entities, for example, penicillin and erythromycin are two different drugs. Therefore, the prior art method of only matching the words in the entity name itself to realize the normalization of named entities has a low accuracy rate. Summary of the invention
[0003] In view of this, the method and device for identifying named entities provided by the present invention aim to solve the problem of low accuracy in classifying named entities in the prior art.
[0004] In a first aspect, the present invention provides a method for identifying a named entity, the method comprising:
[0005] By matching the names of the named entities to be identified with the empirical named entities in the entity knowledge base, the similarity between the named entities to be identified and the empirical named entities is obtained, and N candidate empirical named entities are obtained based on the similarity; the empirical named entities include named entities obtained from historical texts and classified as standard named entities; N is a positive integer;
[0006] Acquire specific features of the candidate experience named entity; the specific features include the similarity between the candidate experience named entity and the named entity to be identified and preset correlation features between the candidate experience named entity and other named entities;
[0007] Based on the specific features, the similarity between the candidate empirical named entity and the named entity to be identified is recalculated to obtain a candidate empirical named entity whose similarity with the named entity to be identified meets preset conditions, and the named entity to be identified is classified into the standard named entity corresponding to the candidate empirical named entity that meets the preset conditions.
[0008] Optionally, the preset correlation feature includes at least one or a combination of the following:
[0009] Whether the name of the standard named entity corresponding to the candidate empirical named entity is exactly the same as the name of the named entity to be identified;
[0010] The ratio of the number of times the name of the standard named entity corresponding to the candidate empirical named entity appears in the corresponding empirical named entity;
[0011] The similarity between the semantic type of the candidate empirical named entity and the semantic type of the named entity to be identified.
[0012] Optionally, if the preset relevance feature includes the name of the standard named entity corresponding to the candidate empirical named entity and the proportion of the number of times the name of the standard named entity appears in the corresponding empirical named entity, obtaining the specific feature of the candidate empirical named entity includes:
[0013] Searching the entity knowledge base for the name of the standard named entity corresponding to the candidate empirical named entity, and the proportion of the number of times the name of the standard named entity appears in the corresponding empirical named entity;
[0014] Alternatively, the name of the standard named entity corresponding to the candidate empirical named entity is obtained, and the number of empirical named entities corresponding to the obtained standard named entity and the number of empirical named entities with the same name as the obtained standard named entity are counted, and based on the counted number, the proportion of the number of times the name of the obtained standard named entity appears in the corresponding empirical named entity is calculated.
[0015] Optionally, if the preset relevance feature includes the similarity between the semantic type of the candidate experience named entity and the semantic type of the named entity to be identified, obtaining the specific feature of the candidate experience named entity includes:
[0016] Obtaining the text to be identified that records the named entity to be identified;
[0017] Determine the semantic type of the named entity to be identified in combination with the text to be identified;
[0018] Acquire the semantic type of the candidate experience named entity from the entity knowledge base;
[0019] The similarity between the semantic type of the candidate empirical named entity and the semantic type of the named entity to be identified is calculated.
[0020] Optionally, recalculating the similarity between the candidate empirical named entity and the named entity to be identified includes:
[0021] assigning a preset weight to the specific feature;
[0022] A weighted calculation is performed based on the assigned weights to obtain a recalculated similarity between the candidate empirical named entity and the named entity to be identified.
[0023] Optionally, recalculating the similarity between the candidate empirical named entity and the named entity to be identified includes:
[0024] The specific feature is input into a preset recognition model to recalculate the similarity between the candidate empirical named entity and the named entity to be recognized.
[0025] Optionally, the method further includes:
[0026] Obtaining N candidate empirical named entities determined based on similarities between named entities in historical texts and empirical named entities in the entity knowledge base and specific features of the candidate empirical named entities;
[0027] Model training is performed on specific features of the candidate empirical named entities after adding labels to obtain the preset recognition model; the labels are used to indicate the standard named entities to which the named entities in the historical text belong.
[0028] Optionally, the method further includes:
[0029] The preset recognition model is updated according to the named entity to be recognized and the corresponding standard named entity.
[0030] Optionally, the preset recognition model includes a linear regression model;
[0031] The optimization goal of the linear regression model is to minimize ‖Xw-Y‖;
[0032] Among them, X is a specific feature, Y is 0 or 1, and w is the weight of the specific feature.
[0033] In a second aspect, the present invention provides a method for identifying a named entity, the method comprising:
[0034] Obtaining the medical named entity to be identified in the medical text to be identified;
[0035] By matching the names of the medical named entity to be identified with the empirical medical named entity in the medical entity knowledge base, the similarity between the medical named entity to be identified and the empirical medical named entity is obtained, and N candidate empirical medical named entities are obtained based on the similarity; the empirical medical named entity includes a named entity obtained from historical medical texts and classified into a standard medical named entity; N is a positive integer;
[0036] Acquire specific features of the candidate empirical medical named entity; the specific features include the similarity between the candidate empirical medical named entity and the medical named entity to be identified and preset correlation features between the candidate empirical medical named entity and other named entities;
[0037] Based on the specific features, the similarity between the candidate empirical medical named entity and the medical named entity to be identified is recalculated to obtain a candidate empirical medical named entity whose similarity with the medical named entity to be identified meets the preset conditions, and the medical named entity to be identified is classified into the standard medical named entity corresponding to the candidate empirical medical named entity that meets the preset conditions.
[0038] In a third aspect, the present invention provides a method for identifying a named entity, the method comprising:
[0039] Collecting text to be recognized, and obtaining named entities to be recognized from the text to be recognized;
[0040] By matching the names of the named entities to be identified with the empirical named entities in the entity knowledge base, the similarity between the named entities to be identified and the empirical named entities is obtained, and N candidate empirical named entities are obtained based on the similarity; the empirical named entities include named entities obtained from historical texts and classified as standard named entities; N is a positive integer;
[0041] Acquire specific features of the candidate experience named entity; the specific features include the similarity between the candidate experience named entity and the named entity to be identified and preset correlation features between the candidate experience named entity and other named entities;
[0042] Based on the specific features, recalculate the similarity between the candidate empirical named entity and the named entity to be identified, obtain a candidate empirical named entity whose similarity with the named entity to be identified meets a preset condition, and classify the named entity to be identified into a standard named entity corresponding to the candidate empirical named entity that meets the preset condition;
[0043] According to the classification result, the corresponding relationship between the named entity to be identified and the standard named entity is output.
[0044] In a fourth aspect, the present invention provides a device for identifying a named entity, the device comprising:
[0045] A name matching unit, used to obtain the similarity between the named entity to be identified and the empirical named entity by matching the named entity to be identified with the empirical named entity in the entity knowledge base;
[0046] A first acquisition unit is used to acquire N candidate empirical named entities based on the similarity; the empirical named entities include named entities obtained from historical texts and classified as standard named entities; N is a positive integer;
[0047] A second acquisition unit is used to acquire specific features of the candidate experience named entity; the specific features include the similarity between the candidate experience named entity and the named entity to be identified and the preset correlation features between the candidate experience named entity and other named entities;
[0048] A calculation unit, configured to recalculate the similarity between the candidate empirical named entity and the named entity to be identified based on the specific feature, and obtain a candidate empirical named entity whose similarity with the named entity to be identified meets a preset condition;
[0049] The classification unit is used to classify the to-be-recognized named entity into a standard named entity corresponding to the candidate empirical named entity that meets a preset condition.
[0050] Optionally, the preset correlation feature acquired by the second acquisition unit includes at least one of the following or a combination of several of the following:
[0051] Whether the name of the standard named entity corresponding to the candidate empirical named entity is exactly the same as the name of the named entity to be identified;
[0052] The ratio of the number of times the name of the standard named entity corresponding to the candidate empirical named entity appears in the corresponding empirical named entity;
[0053] The similarity between the semantic type of the candidate empirical named entity and the semantic type of the named entity to be identified.
[0054] Optionally, the second acquisition unit is used to search the entity knowledge base for the name of the standard named entity corresponding to the candidate empirical named entity and the ratio of the number of times the name of the standard named entity corresponding to the candidate empirical named entity appears in the corresponding empirical named entity if the preset correlation feature includes the ratio of the number of times the name of the standard named entity corresponding to the candidate empirical named entity appears in the corresponding empirical named entity; or, to obtain the name of the standard named entity corresponding to the candidate empirical named entity, and count the number of empirical named entities corresponding to the obtained standard named entity, and the number of empirical named entities that are the same as the name of the obtained standard named entity, and calculate the ratio of the number of times the name of the obtained standard named entity appears in the corresponding empirical named entity based on the counted number.
[0055] Optionally, the second acquisition unit is used to obtain the text to be identified that records the named entity to be identified if the preset correlation feature includes the similarity between the semantic type of the candidate empirical named entity and the semantic type of the named entity to be identified; determine the semantic type of the named entity to be identified in combination with the text to be identified; obtain the semantic type of the candidate empirical named entity from the entity knowledge base; and calculate the similarity between the semantic type of the candidate empirical named entity and the semantic type of the named entity to be identified.
[0056] Optionally, the computing unit includes:
[0057] An allocating module, used for allocating a preset weight to the specific feature;
[0058] The first calculation module is used to perform weighted calculation based on the assigned weights to obtain the recalculated similarity between the candidate empirical named entity and the named entity to be identified.
[0059] Optionally, the computing unit includes:
[0060] The second calculation module is used to input the specific feature into a preset recognition model to recalculate the similarity between the candidate empirical named entity and the named entity to be recognized.
[0061] Optionally, the device further comprises:
[0062] A third acquisition unit is used to acquire N candidate empirical named entities determined based on similarities between the named entities in the historical text and the empirical named entities in the entity knowledge base, and specific features of the candidate empirical named entities;
[0063] The training unit is used to perform model training on the specific features of the candidate empirical named entities after adding labels to obtain the preset recognition model; the labels are used to indicate the standard named entities to which the named entities in the historical text belong.
[0064] Optionally, the device further comprises:
[0065] An updating unit is used to update the preset recognition model according to the named entity to be recognized and the corresponding standard named entity.
[0066] Optionally, the preset recognition model used by the second calculation module includes a linear regression model;
[0067] The optimization goal of the linear regression model is to minimize ‖Xw-Y‖;
[0068] Among them, X is a specific feature, Y is 0 or 1, and w is the weight of the specific feature.
[0069] In a fifth aspect, the present invention provides a device for identifying a named entity, the device comprising:
[0070] A first acquisition unit is used to acquire a medical named entity to be identified in the medical text to be identified;
[0071] A name matching unit, used for obtaining the similarity between the to-be-identified medical named entity and the empirical medical named entity by matching the name of the to-be-identified medical named entity with the empirical medical named entity in the medical entity knowledge base;
[0072] A second acquisition unit is used to acquire N candidate empirical medical named entities based on the similarity; the empirical medical named entities include named entities obtained from historical medical texts and classified into standard medical named entities; N is a positive integer;
[0073] A third acquisition unit is used to acquire specific features of the candidate empirical medical named entity; the specific features include the similarity between the candidate empirical medical named entity and the medical named entity to be identified and the preset correlation features between the candidate empirical medical named entity and other named entities;
[0074] A calculation unit, used for recalculating the similarity between the candidate empirical medical named entity and the medical named entity to be identified based on the specific feature, and obtaining a candidate empirical medical named entity whose similarity with the medical named entity to be identified meets a preset condition;
[0075] The classification unit is used to classify the medical named entity to be identified into the standard medical named entity corresponding to the candidate empirical medical named entity that meets the preset conditions.
[0076] In a sixth aspect, the present invention provides a device for identifying a named entity, the device comprising:
[0077] A collection unit, used for collecting text to be recognized;
[0078] A first acquisition unit, used for acquiring a named entity to be recognized from the text to be recognized;
[0079] A name matching unit, used for obtaining the similarity between the named entity to be identified and the empirical named entity by matching the named entity to be identified with the empirical named entity in the entity knowledge base;
[0080] A second acquisition unit is used to acquire N candidate empirical named entities based on the similarity; the empirical named entities include named entities obtained from historical texts and classified as standard named entities; N is a positive integer;
[0081] A third acquisition unit is used to acquire specific features of the candidate experience named entity; the specific features include the similarity between the candidate experience named entity and the named entity to be identified and the preset correlation features between the candidate experience named entity and other named entities;
[0082] A calculation unit, configured to recalculate the similarity between the candidate empirical named entity and the named entity to be identified based on the specific feature, and obtain a candidate empirical named entity whose similarity with the named entity to be identified meets a preset condition;
[0083] A classification unit, used to classify the to-be-recognized named entity into a standard named entity corresponding to the candidate empirical named entity that meets a preset condition;
[0084] The output unit is used to output the corresponding relationship between the named entity to be identified and the standard named entity according to the classification result.
[0085] In a seventh aspect, the present invention provides a storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the method for recognizing named entities as described in any of the first to third aspects.
[0086] In an eighth aspect, the present invention provides an electronic device, the electronic device comprising a storage medium and a processor;
[0087] The processor is adapted to implement each instruction;
[0088] The storage medium is suitable for storing a plurality of instructions;
[0089] The instructions are suitable for being loaded by the processor and executing the method for recognizing named entities as described in any of the first to third aspects.
[0090] By means of the above technical scheme, the method and device for identifying named entities provided by the present invention can first match the names of the named entities to be identified with the empirical named entities in the entity knowledge base, and screen out N candidate empirical named entities from all the empirical named entities, thereby achieving a rough identification of the named entities from the dimension of the name itself, and then combining other features of the N candidate empirical named entities, determine a candidate empirical named entity that best matches the named entity to be identified from the N candidate empirical named entities, and classify the named entity to be identified into the standard named entity corresponding to the last selected candidate empirical named entity, thereby achieving detailed identification of the named entity from the dimension other than the name itself. In other words, compared with the prior art that identifies named entities only from the dimension of the name itself, the present invention can comprehensively identify named entities from the dimension of the name itself and the dimension other than the name itself, thereby improving the accuracy of the classification of named entities.
[0091] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0092] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0093] Figure 1 A flowchart of a method for identifying a named entity provided by an embodiment of the present invention is shown;
[0094] Figure 2 A flowchart of another method for identifying named entities provided by an embodiment of the present invention is shown;
[0095] Figure 3 A flowchart of another method for identifying named entities provided by an embodiment of the present invention is shown;
[0096] Figure 4 An example diagram of annotating a named entity provided by an embodiment of the present invention is shown;
[0097] Figure 5 Another example diagram of annotating named entities provided by an embodiment of the present invention is shown;
[0098] Figure 6 A block diagram showing a composition of a named entity recognition device provided by an embodiment of the present invention;
[0099] Figure 7 A block diagram showing another apparatus for identifying named entities provided by an embodiment of the present invention is shown;
[0100] Figure 8 A block diagram showing a composition of another named entity recognition device provided by an embodiment of the present invention;
[0101] Fig. 9 A block diagram showing the composition of another named entity recognition device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0102] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0103] The embodiment of the present invention provides a method for identifying a named entity. Figure 1 As shown, the method includes:
[0104] 101. By matching the names of the named entities to be identified with the empirical named entities in the entity knowledge base, the similarity between the named entities to be identified and the empirical named entities is obtained, and N candidate empirical named entities are obtained based on the similarity.
[0105] Among them, the so-called named entities are names of people, institutions, places, products, diseases and all other entities identified by names. More extensive entities also include numbers, dates, currencies, addresses, etc. The entity knowledge base includes standard named entities and empirical named entities classified under the standard named entities. The empirical named entities include named entities obtained from historical texts and classified under the standard named entities, that is, some named entities accumulated from experience; standard named entities are formal named entities officially announced in the field to which they belong.
[0106] For example, the standard named entity is "Peking University Law School", an empirical named entity corresponding to the standard named entity is "Peking University Court", and the named entity to be identified in the text to be identified is "Peking University Court".
[0107] When identifying the standard named entity to which the named entity to be identified belongs in the text to be identified, the named entity to be identified can be first matched with the empirical named entity in the entity knowledge base, and the similarity of the two entity names can be calculated. According to the similarity, N empirical named entities are determined from a large number of empirical named entities, and then the N empirical named entities are re-sorted using the rearrangement algorithm in the following steps, and the empirical named entities that meet the preset conditions are determined to be the empirical named entities that are closest to the named entity to be identified. In general, the higher the name similarity, the closer the two named entities are, so when determining N empirical named entities from a large number of empirical named entities according to the similarity, the empirical named entities whose similarity ranks in the top N can be determined from the large number of empirical named entities. Among them, N is a positive integer, and a reasonable value can be determined based on experience. The so-called reasonable value is to try to select a smaller N while ensuring the quality of the secondary sorting, for example, it can be 100.
[0108] In addition to the names of named entities, the entity knowledge base also includes other features of named entities, such as the proportion of the number of times the name of the standard named entity appears in the corresponding empirical named entity and the semantic type of the named entity. Semantic types include manufacturer names, brand names, product specifications, etc., which are used to assist in describing the characteristics of named entities. Before performing name matching, a query index for the entity knowledge base can be established first, so that the query index can be used to quickly find the name of the named entity from the entity knowledge base that stores a large amount of other feature information.
[0109] When calculating the similarity of two entity names, the two nouns can be matched by unigram matching, bigram matching, trigram matching, and longest common string matching, and the following features can be used to calculate the similarity during matching: N-Gram matching, edit distance on characters. In addition, the similarity can be a calculated similarity value or a score determined according to a preset scoring rule and a similarity value.
[0110] 102. Obtain specific features of candidate empirical named entities.
[0111] The specific features include the similarity between the candidate experience named entity and the named entity to be identified and the preset correlation features between the candidate experience named entity and other named entities.
[0112] The preset relevance feature of the named entity includes at least one or a combination of the following:
[0113] (1) Whether the name of the standard named entity corresponding to the candidate empirical named entity is exactly the same as the name of the named entity to be identified.
[0114] Among them, since the ultimate goal of identifying named entities is to classify the named entities to be identified into standard named entities, when the named entity to be identified is exactly the same as the standard named entity in name, it is more likely to be classified into the standard named entity, so this feature can be added as one of the judgment bases.
[0115] (2) The ratio of the number of times the name of the standard named entity corresponding to the candidate empirical named entity appears in the corresponding empirical named entity.
[0116] Since the entity knowledge base contains not only the name of the named entity but also other features besides the name to describe the uniqueness of the named entity, when establishing the entity knowledge base, different empirical named entities may have the same name but different other features, resulting in different corresponding standard named entities. In this case, among the candidate empirical named entities obtained by matching based solely on the name, there may be candidate empirical named entities with the same name but different other features. In order to further determine which candidate empirical named entity the named entity to be identified is closer to, the ratio of the number of times the name of the standard named entity corresponding to the candidate empirical named entity appears in the corresponding empirical named entity can be used as a basis for judgment.
[0117] If the preset association feature includes the name of the standard named entity corresponding to the candidate empirical named entity, and the ratio of the number of times it appears in the corresponding empirical named entity, then obtaining the specific features of the candidate empirical named entity includes: searching the entity knowledge base for the name of the standard named entity corresponding to the candidate empirical named entity, and the ratio of the number of times it appears in the corresponding empirical named entity; or, obtaining the name of the standard named entity corresponding to the candidate empirical named entity, and counting the number of empirical named entities corresponding to the obtained standard named entity, and the number of empirical named entities that are the same as the name of the obtained standard named entity, and calculating the ratio of the number of times the name of the obtained standard named entity appears in the corresponding empirical named entity based on the counted number.
[0118] For example, a standard named entity is named A, which corresponds to 10 empirical named entities, of which 8 empirical named entities are named A and the other two are named B. Then the name of the standard named entity appears in the empirical named entity at a ratio of 80%, while the other name B appears at a ratio of 20%. Another example is that another standard named entity is named B, which corresponds to 10 empirical named entities, of which 7 empirical named entities are named B and the other 3 are named A. Then the name of the standard named entity appears in the empirical named entity at a ratio of 70%, while the other name A appears at a ratio of 30%. Therefore, when the name of the candidate empirical named entity is A, it is more likely that the named entity to be identified belongs to the first type of standard named entity.
[0119] (3) The similarity between the semantic type of the candidate empirical named entity and the semantic type of the named entity to be identified.
[0120] Since the attributes of a named entity such as manufacturer, specification, and brand may be different, when the name of the named entity to be identified is very similar to the name of the empirical named entity, they may represent two different named entities. In order to make the identification more accurate, the semantic type can be added as a judgment basis.
[0121] If the preset association feature includes the similarity between the semantic type of the candidate empirical named entity and the semantic type of the named entity to be identified, obtaining the specific features of the candidate empirical named entity includes: obtaining the text to be identified that records the named entity to be identified; determining the semantic type of the named entity to be identified in combination with the text to be identified; obtaining the semantic type of the candidate empirical named entity from the entity knowledge base; and calculating the similarity between the semantic type of the candidate empirical named entity and the semantic type of the named entity to be identified.
[0122] When determining the semantic type of the named entity to be identified in combination with the text to be identified, the context in the text to be identified can be used for determination. For example, for a drug, a hospital pharmacy may have multiple brands. When a doctor prescribes a drug, in addition to entering the name of the drug itself, he or she will also write the brand to which it belongs. Therefore, the semantic type can be inferred from the context on the prescription that it is a certain brand.
[0123] 103. Based on the specific features, recalculate the similarity between the candidate empirical named entity and the named entity to be identified, obtain a candidate empirical named entity whose similarity with the named entity to be identified meets preset conditions, and classify the named entity to be identified into the standard named entity corresponding to the candidate empirical named entity that meets the preset conditions.
[0124] When recalculating the similarity between the candidate empirical named entity and the named entity to be identified based on the specific features, there are multiple algorithms used. The following two algorithms are used as examples for explanation:
[0125] The first method is to assign a preset weight to the specific feature; perform weighted calculation based on the assigned weight to obtain a recalculated similarity between the candidate empirical named entity and the named entity to be identified.
[0126] Specifically, based on historical experience, a relatively reasonable weight can be set for each specific feature. For example, based on historical experience, it can be determined that the feature "whether the name of the standard named entity corresponding to the candidate empirical named entity is exactly the same as the name of the named entity to be identified" is more important than other features. Assigning a higher weight to this feature will make the final result more accurate. The reasonable weight corresponding to this feature can be determined based on a large number of experiments.
[0127] Wherein, when the result of "whether the name of the standard named entity corresponding to the candidate empirical named entity is exactly the same as the name of the named entity to be identified" is "yes", it can be set to 1, and when the result is "no", it can be set to 0. In this case, the weighting formula can be final similarity = preliminary calculated similarity * weight 1 + (1 or 0) * weight 2 + number of times * weight 3 + semantic type similarity * weight 3.
[0128] The second method is to input the specific feature into a preset recognition model and recalculate the similarity between the candidate empirical named entity and the named entity to be recognized.
[0129] The preset recognition model is a machine self-learning model trained based on historical texts, including but not limited to linear regression models and support vector machine models. After obtaining the preset recognition model, the specific features of the named entity to be recognized can be directly input into the preset recognition model, and the final similarity result can be directly and automatically output.
[0130] After recalculating the similarity between the N candidate experience named entities and the named entity to be identified, the N candidate experience named entities can be reordered to obtain candidate experience named entities that meet the preset conditions, and the standard named entities corresponding to the candidate experience named entities that meet the preset conditions are used as the standard named entities mapped to the named entity to be identified. The preset condition can be the one with the highest similarity, or it can be one of the candidate experience named entities with a similarity greater than a preset threshold. That is to say, after reordering the N candidate experience named entities, the candidate experience named entity ranked first in similarity can be used as the final candidate experience named entity required, or one of the candidate experience named entities with a similarity greater than the preset threshold can be used as the final candidate experience named entity required.
[0131] The method for identifying named entities provided in the embodiment of the present invention can first match the names of the named entities to be identified with the empirical named entities in the entity knowledge base, and screen out N candidate empirical named entities from all the empirical named entities, thereby achieving rough identification of the named entities from the dimension of the name itself, and then combining other features of the N candidate empirical named entities, determine a candidate empirical named entity that best matches the named entity to be identified from the N candidate empirical named entities, and classify the named entity to be identified into the standard named entity corresponding to the last selected candidate empirical named entity, thereby achieving detailed identification of the named entity from the dimension other than the name itself. In other words, compared with the prior art that identifies named entities only from the dimension of the name itself, the present invention can comprehensively identify named entities from the dimension of the name itself and the dimension other than the name itself, thereby improving the accuracy of named entity classification.
[0132] Optionally, when the preset recognition model is used to recalculate the similarity and reorder the N candidate empirical named entities, the preset recognition model needs to be trained first. The training method of the preset recognition model is described below:
[0133] Obtain N candidate empirical named entities determined based on the similarity between the named entities in the historical text and the empirical named entities in the entity knowledge base, as well as specific features of the candidate empirical named entities; perform model training on the specific features of the candidate empirical named entities after adding labels to obtain the preset recognition model; the labels are used to indicate the standard named entities to which the named entities in the historical text belong.
[0134] The historical naming text is a large amount of text. For each named entity of each historical naming text, N candidate empirical named entities (preferably the top N) are preliminarily determined, and then M (a positive integer) specific features corresponding to each candidate empirical named entity are obtained. Labels for indicating the standard named entities to which the named entities in the historical text belong are manually added to the M specific features as a whole. A machine self-learning algorithm is used to allow the computer to continuously learn and adjust the weight corresponding to each specific feature. Finally, when the recognition accuracy reaches a preset threshold, the final required preset recognition model is obtained.
[0135] It is pointed out in the above embodiments that the preset recognition model may be a linear regression model, a support vector machine model, or other machine self-learning models.
[0136] The following takes the linear regression model as an example to explain the specific parameters:
[0137] The optimization goal of the linear regression model is to minimize ‖Xw-Y‖;
[0138] Among them, X is a specific feature, Y is 0 or 1, and w is the weight of the specific feature. By continuously adjusting the value of w, ‖Xw-Y‖ is calculated. When ‖Xw-Y‖ is the smallest, the recognition accuracy reaches the highest, thus obtaining the preset recognition model.
[0139] In addition, in order to further improve the accuracy of the preset recognition model in recognizing named entities, after determining the standard named entity corresponding to the named entity to be recognized, the preset recognition model can be updated according to the named entity to be recognized and the corresponding standard named entity.
[0140] In a specific embodiment, after a certain number of results are identified using the preset recognition model, the preset recognition model may be updated based on these results.
[0141] The embodiments of the present invention can be applied to named entities in various fields. As long as there are named entity names with diverse names, the embodiments of the present invention can achieve accurate classification of named entities. Figure 2 As shown, the implementation process of the present invention is described below by taking the medical field with diverse named entity names as an example:
[0142] 201. Obtain a medical named entity to be identified in the medical text to be identified.
[0143] Methods for obtaining the medical text to be recognized include but are not limited to: receiving the medical text to be recognized input by a user; or, receiving the medical text to be recognized sent by other devices; or, obtaining the medical text to be recognized from a database.
[0144] The method of obtaining the medical named entity to be identified includes but is not limited to: receiving the medical named entity to be identified input by the user (or sent by other devices); or extracting the medical named entity to be identified from the medical text to be identified by semantic analysis of the medical text to be identified. For example, matching the text of the medical text to be identified with a preset medical vocabulary, if the match is successful, the successfully matched word is used as the medical named entity to be identified.
[0145] 202. Obtain the similarity between the medical named entity to be identified and the empirical medical named entity by name matching the medical named entity to be identified with the empirical medical named entity in the medical entity knowledge base, and obtain N candidate empirical medical named entities based on the similarity.
[0146] The empirical medical named entities include named entities obtained from historical medical texts and classified into the standard medical named entities; N is a positive integer. The medical named entities include drug names, disease names, symptom names, treatment names, etc. The medical entity knowledge base can be a Unified Medical Language System (UMLS). Medical texts include prescriptions, test sheets, examination sheets, medical papers, medical journals, etc.
[0147] 203. Obtain specific features of candidate empirical medical named entities.
[0148] The specific features include the similarity between the candidate empirical medical named entity and the medical named entity to be identified and preset correlation features between the candidate empirical medical named entity and other named entities.
[0149] 204. Based on the specific features, recalculate the similarity between the candidate empirical medical named entity and the medical named entity to be identified, obtain a candidate empirical medical named entity whose similarity with the medical named entity to be identified meets a preset condition, and classify the medical named entity to be identified into the standard medical named entity corresponding to the candidate empirical medical named entity that meets the preset condition.
[0150] The method for identifying named entities provided in the embodiment of the present invention can first match the names of the medical named entities to be identified with the empirical medical named entities in the medical entity knowledge base, and screen out N candidate medical named entities from all the empirical medical named entities, thereby achieving a rough identification of the medical named entities from the dimension of the name itself, and then combining other features of the N candidate empirical medical named entities, determine a candidate empirical medical named entity that best matches the medical named entity to be identified from the N candidate empirical medical named entities, and classify the medical named entity to be identified into the standard medical named entity corresponding to the candidate empirical medical named entity finally selected, thereby achieving detailed identification of the medical named entity from a dimension other than the name itself. In other words, compared with the prior art that identifies medical named entities only from the dimension of the name itself, the present invention can comprehensively identify medical named entities from the dimension of the name itself and the dimension other than the name itself, thereby improving the accuracy of medical named entity classification.
[0151] Further, according to the above method embodiment, another embodiment of the present invention also provides a method for identifying a named entity, such as Figure 3 As shown, the method includes:
[0152] 301. Collect text to be recognized, and obtain named entities to be recognized from the text to be recognized.
[0153] Methods of collecting text to be recognized include, but are not limited to: receiving text to be recognized input by a user; or, receiving text to be recognized sent by other devices; or, obtaining text to be recognized from a database.
[0154] The method of obtaining the named entity to be recognized from the text to be recognized includes but is not limited to: receiving the text to be recognized input by the user (or sent by other devices); or extracting the named entity to be recognized from the text to be recognized by performing semantic analysis on the text to be recognized. For example, matching the text to be recognized with a preset word library, if the match is successful, the successfully matched word is used as the named entity to be recognized.
[0155] 302. Obtain similarity between the named entity to be identified and the empirical named entity by matching the names of the named entity to be identified with the empirical named entity in the entity knowledge base, and obtain N candidate empirical named entities based on the similarity.
[0156] The empirical named entities include named entities obtained from historical texts and classified into standard named entities; N is a positive integer.
[0157] 303. Obtain specific features of the candidate empirical named entity.
[0158] The specific features include the similarity between the candidate experience named entity and the named entity to be identified and preset correlation features between the candidate experience named entity and other named entities.
[0159] 304. Based on the specific features, recalculate the similarity between the candidate empirical named entity and the named entity to be identified, obtain a candidate empirical named entity whose similarity with the named entity to be identified meets a preset condition, and classify the named entity to be identified into a standard named entity corresponding to the candidate empirical named entity that meets the preset condition.
[0160] 305. Output the corresponding relationship between the named entity to be identified and the standard named entity according to the classification result.
[0161] When outputting the correspondence between the named entity to be identified and the standard named entity, only the correspondence between the named entity to be identified and the standard named entity may be output, or the correspondence between the named entity to be identified and the standard named entity may be marked in the text to be identified. There are many specific implementation methods for marking the correspondence between the named entity to be identified and the standard named entity in the text to be identified. For example, Figure 4 As shown in , after the named entities to be recognized are classified, when the text to be recognized is output, corresponding annotations can be added next to the named entities to indicate their corresponding standard named entities. Figure 5As shown, annotations with correspondence between the named entities to be identified and the standard named entities can also be added around the text content to be identified (as shown below).
[0162] The named entity recognition method provided by the embodiment of the present invention can not only improve the accuracy of named entity classification, but also enable the user to quickly learn the standard meaning of the named entity to be recognized by outputting the correspondence between the named entity to be recognized and the standard named entity to the user.
[0163] Further, according to the above method embodiment, another embodiment of the present invention also provides a device for identifying a named entity, such as Figure 6 As shown, the device comprises:
[0164] A name matching unit 41 is used to obtain the similarity between the named entity to be identified and the empirical named entity by matching the named entity to be identified with the empirical named entity in the entity knowledge base;
[0165] A first acquisition unit 42 is used to acquire N candidate empirical named entities based on the similarity; the empirical named entities include named entities obtained from historical texts and classified as standard named entities; N is a positive integer;
[0166] A second acquisition unit 43 is used to acquire specific features of the candidate experience named entity; the specific features include the similarity between the candidate experience named entity and the named entity to be identified and the preset correlation features between the candidate experience named entity and other named entities;
[0167] A calculation unit 44 is used to recalculate the similarity between the candidate experience named entity and the named entity to be identified based on the specific feature, and obtain a candidate experience named entity whose similarity with the named entity to be identified meets a preset condition;
[0168] The classification unit 45 is used to classify the to-be-recognized named entity into a standard named entity corresponding to the candidate empirical named entity that meets a preset condition.
[0169] Optionally, the preset correlation feature acquired by the second acquisition unit 43 includes at least one of the following or a combination of several of the following:
[0170] Whether the name of the standard named entity corresponding to the candidate empirical named entity is exactly the same as the name of the named entity to be identified;
[0171] The ratio of the number of times the name of the standard named entity corresponding to the candidate empirical named entity appears in the corresponding empirical named entity;
[0172] The similarity between the semantic type of the candidate empirical named entity and the semantic type of the named entity to be identified.
[0173] Optionally, the second acquisition unit 43 is used to search the entity knowledge base for the name of the standard named entity corresponding to the candidate empirical named entity and the ratio of the number of times it appears in the corresponding empirical named entity if the preset correlation feature includes the ratio of the number of times the name of the standard named entity corresponding to the candidate empirical named entity appears in the corresponding empirical named entity; or, obtain the name of the standard named entity corresponding to the candidate empirical named entity, and count the number of empirical named entities corresponding to the obtained standard named entity, and the number of empirical named entities that are the same as the name of the obtained standard named entity, and calculate the ratio of the number of times the name of the obtained standard named entity appears in the corresponding empirical named entity based on the counted number.
[0174] Optionally, the second acquisition unit 43 is used to obtain the text to be identified that records the named entity to be identified if the preset correlation feature includes the similarity between the semantic type of the candidate empirical named entity and the semantic type of the named entity to be identified; determine the semantic type of the named entity to be identified in combination with the text to be identified; obtain the semantic type of the candidate empirical named entity from the entity knowledge base; and calculate the similarity between the semantic type of the candidate empirical named entity and the semantic type of the named entity to be identified.
[0175] Optional, such as Figure 7 As shown, the calculation unit 44 includes:
[0176] An allocating module 441, configured to allocate a preset weight to the specific feature;
[0177] The first calculation module 442 is used to perform weighted calculation based on the assigned weights to obtain the recalculated similarity between the candidate empirical named entity and the named entity to be identified.
[0178] Optional, such as Figure 7 As shown, the calculation unit 44 includes:
[0179] The second calculation module 443 is used to input the specific feature into a preset recognition model to recalculate the similarity between the candidate empirical named entity and the named entity to be recognized.
[0180] Optional, such as Figure 7 As shown, the device also includes:
[0181] A third acquisition unit 46 is used to acquire N candidate empirical named entities determined based on similarities between the named entities in the historical text and the empirical named entities in the entity knowledge base and specific features of the candidate empirical named entities;
[0182] The training unit 47 is used to perform model training on the specific features of the candidate empirical named entities after adding labels to obtain the preset recognition model; the labels are used to indicate the standard named entities to which the named entities in the historical text belong.
[0183] Optional, such as Figure 7 As shown, the device also includes:
[0184] The updating unit 48 is used to update the preset recognition model according to the named entity to be recognized and the corresponding standard named entity.
[0185] Optionally, the preset recognition model used by the second calculation module 443 includes a linear regression model;
[0186] The optimization goal of the linear regression model is to minimize ‖Xw-Y‖;
[0187] Among them, X is a specific feature, Y is 0 or 1, and w is the weight of the specific feature.
[0188] The named entity recognition device provided by the embodiment of the present invention can first match the name of the named entity to be recognized with the empirical named entity in the entity knowledge base, and screen out N candidate empirical named entities from all the empirical named entities, thereby achieving rough recognition of the named entity from the dimension of the name itself, and then combining other features of the N candidate empirical named entities, determine a candidate empirical named entity that best matches the named entity to be recognized from the N candidate empirical named entities, and classify the named entity to be recognized into the standard named entity corresponding to the last selected candidate empirical named entity, thereby achieving detailed recognition of the named entity from a dimension other than the name itself. In other words, compared with the prior art that recognizes named entities only from the dimension of the name itself, the present invention can comprehensively recognize named entities from the dimension of the name itself and the dimension other than the name itself, thereby improving the accuracy of named entity classification.
[0189] Further, according to the above method embodiment, another embodiment of the present invention also provides a device for identifying a named entity, such as Figure 8 As shown, the device comprises:
[0190] A first acquisition unit 51 is used to acquire a medical named entity to be identified in the medical text to be identified;
[0191] A name matching unit 52, configured to obtain the similarity between the to-be-identified medical named entity and the empirical medical named entity by matching the to-be-identified medical named entity with the empirical medical named entity in the medical entity knowledge base;
[0192] The second acquisition unit 53 is used to acquire N candidate empirical medical named entities based on the similarity; the empirical medical named entities include named entities obtained from historical medical texts and classified into standard medical named entities; N is a positive integer;
[0193] The third acquisition unit 54 is used to acquire specific features of the candidate empirical medical named entity; the specific features include the similarity between the candidate empirical medical named entity and the medical named entity to be identified and the preset correlation features between the candidate empirical medical named entity and other named entities;
[0194] A calculation unit 55, configured to recalculate the similarity between the candidate empirical medical named entity and the medical named entity to be identified based on the specific feature, and obtain a candidate empirical medical named entity whose similarity with the medical named entity to be identified meets a preset condition;
[0195] The classification unit 56 is used to classify the medical named entity to be identified into the standard medical named entity corresponding to the candidate empirical medical named entity that meets the preset condition.
[0196] Further, according to the above method embodiment, another embodiment of the present invention also provides a device for identifying a named entity, such as Fig. 9 As shown, the device comprises:
[0197] A collection unit 61, used for collecting text to be recognized;
[0198] A first acquisition unit 62, configured to acquire a named entity to be recognized from the text to be recognized;
[0199] A name matching unit 63, configured to obtain a similarity between the named entity to be identified and the empirical named entity by performing name matching between the named entity to be identified and the empirical named entity in the entity knowledge base;
[0200] The second acquisition unit 64 is used to acquire N candidate empirical named entities based on the similarity; the empirical named entities include named entities obtained from historical texts and classified as standard named entities; N is a positive integer;
[0201] The third acquisition unit 65 is used to acquire specific features of the candidate experience named entity; the specific features include the similarity between the candidate experience named entity and the named entity to be identified and the preset correlation features between the candidate experience named entity and other named entities;
[0202] A calculation unit 66, configured to recalculate the similarity between the candidate empirical named entity and the named entity to be identified based on the specific feature, and obtain a candidate empirical named entity whose similarity with the named entity to be identified meets a preset condition;
[0203] A classification unit 67, configured to classify the to-be-recognized named entity into a standard named entity corresponding to the candidate empirical named entity that meets a preset condition;
[0204] The output unit 68 is used to output the corresponding relationship between the named entity to be identified and the standard named entity according to the classification result.
[0205] Furthermore, another embodiment of the present invention provides a storage medium, wherein the storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the method for identifying named entities as described above.
[0206] The instructions stored in the storage medium provided by the embodiment of the present invention can first match the names of the named entities to be identified with the empirical named entities in the entity knowledge base, and screen out N candidate empirical named entities from all the empirical named entities, thereby achieving rough identification of the identified named entities from the dimension of the name itself, and then combining other features of the N candidate empirical named entities, determine a candidate empirical named entity that best matches the named entity to be identified from the N candidate empirical named entities, and classify the named entity to be identified into the standard named entity corresponding to the last selected candidate empirical named entity, thereby achieving detailed identification of the named entity from a dimension other than the name itself. In other words, compared with the prior art that identifies named entities only from the dimension of the name itself, the present invention can comprehensively identify named entities from both the dimension of the name itself and the dimension other than the name itself, thereby improving the accuracy of named entity classification.
[0207] Furthermore, another embodiment of the present invention provides an electronic device, the electronic device comprising a storage medium and a processor;
[0208] The processor is adapted to implement each instruction;
[0209] The storage medium is suitable for storing a plurality of instructions;
[0210] The instructions are suitable for being loaded by the processor and executing the method for identifying named entities as described above.
[0211] The electronic device provided by the embodiment of the present invention can first match the names of the named entities to be identified with the empirical named entities in the entity knowledge base, and select N candidate empirical named entities from all the empirical named entities, thereby achieving rough recognition of the identified named entities from the dimension of the name itself, and then combining other features of the N candidate empirical named entities, determine a candidate empirical named entity that best matches the named entity to be identified from the N candidate empirical named entities, and classify the named entity to be identified into the standard named entity corresponding to the last selected candidate empirical named entity, thereby achieving detailed recognition of the named entity from the dimension other than the name itself. In other words, compared with the prior art that identifies named entities only from the dimension of the name itself, the present invention can comprehensively identify named entities from the dimension of the name itself and the dimension other than the name itself, thereby improving the accuracy of named entity classification.
[0212] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0213] It is understandable that the related features in the above methods and devices can be referenced to each other. In addition, the "first", "second" and the like in the above embodiments are used to distinguish the embodiments, but do not represent the advantages and disadvantages of the embodiments.
[0214] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0215] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages can be utilized to realize the content of the present invention described herein, and the description of the above specific language is for disclosing an embodiment of the present invention.
[0216] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.
[0217] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.
[0218] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.
[0219] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.
[0220] The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the method and device for identifying named entities according to an embodiment of the present invention. The present invention may also be implemented as a device or apparatus program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.
[0221] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.
Claims
1. A method for identifying named entities, characterized in that: The method comprises: By matching the names of the named entities to be identified with the empirical named entities in the entity knowledge base, the similarity between the named entities to be identified and the empirical named entities is obtained, and N candidate empirical named entities are obtained based on the similarity; the empirical named entities include named entities obtained from historical texts and classified as standard named entities; N is a positive integer; Acquire specific features of the candidate experience named entity; the specific features include the similarity between the candidate experience named entity and the named entity to be identified and preset correlation features between the candidate experience named entity and other named entities; Based on the specific features, the similarity between the candidate empirical named entity and the named entity to be identified is recalculated to obtain a candidate empirical named entity whose similarity with the named entity to be identified meets preset conditions, and the named entity to be identified is classified into the standard named entity corresponding to the candidate empirical named entity that meets the preset conditions.
2. The method according to claim 1, characterized in that The preset correlation feature includes at least one or a combination of the following: Whether the name of the standard named entity corresponding to the candidate empirical named entity is exactly the same as the name of the named entity to be identified; The ratio of the number of times the name of the standard named entity corresponding to the candidate empirical named entity appears in the corresponding empirical named entity; The similarity between the semantic type of the candidate empirical named entity and the semantic type of the named entity to be identified.
3. The method according to claim 2, characterized in that If the preset relevance feature includes the name of the standard named entity corresponding to the candidate empirical named entity and the proportion of the number of times the name appears in the corresponding empirical named entity, then obtaining the specific features of the candidate empirical named entity includes: Searching the entity knowledge base for the name of the standard named entity corresponding to the candidate empirical named entity, and the proportion of the number of times the name of the standard named entity appears in the corresponding empirical named entity; Alternatively, the name of the standard named entity corresponding to the candidate empirical named entity is obtained, and the number of empirical named entities corresponding to the obtained standard named entity and the number of empirical named entities with the same name as the obtained standard named entity are counted, and based on the counted number, the proportion of the number of times the name of the obtained standard named entity appears in the corresponding empirical named entity is calculated.
4. The method according to claim 2, characterized in that: If the preset relevance feature includes the similarity between the semantic type of the candidate experience named entity and the semantic type of the named entity to be identified, obtaining the specific feature of the candidate experience named entity includes: Obtaining the text to be identified that records the named entity to be identified; Determine the semantic type of the named entity to be identified in combination with the text to be identified; Acquire the semantic type of the candidate experience named entity from the entity knowledge base; The similarity between the semantic type of the candidate empirical named entity and the semantic type of the named entity to be identified is calculated.
5. The method according to any one of claims 1 to 4, characterized in that: Recalculating the similarity between the candidate empirical named entity and the named entity to be identified includes: assigning a preset weight to the specific feature; A weighted calculation is performed based on the assigned weights to obtain a recalculated similarity between the candidate empirical named entity and the named entity to be identified.
6. The method according to any one of claims 1 to 4, characterized in that: Recalculating the similarity between the candidate empirical named entity and the named entity to be identified includes: The specific feature is input into a preset recognition model to recalculate the similarity between the candidate empirical named entity and the named entity to be recognized.
7. The method according to claim 6, characterized in that The method further comprises: Obtaining N candidate empirical named entities determined based on similarities between named entities in historical texts and empirical named entities in the entity knowledge base and specific features of the candidate empirical named entities; Model training is performed on specific features of the candidate empirical named entities after adding labels to obtain the preset recognition model; the labels are used to indicate the standard named entities to which the named entities in the historical text belong.
8. The method according to claim 6, characterized in that The method further comprises: The preset recognition model is updated according to the named entity to be recognized and the corresponding standard named entity.
9. The method according to claim 6, characterized in that The preset recognition model includes a linear regression model; The optimization goal of the linear regression model is to minimize ‖Xw-Y‖; Among them, X is a specific feature, Y is 0 or 1, and w is the weight of the specific feature.
10. A method for identifying a named entity, characterized in that: The method comprises: Obtaining the medical named entity to be identified in the medical text to be identified; By matching the names of the medical named entity to be identified with the empirical medical named entity in the medical entity knowledge base, the similarity between the medical named entity to be identified and the empirical medical named entity is obtained, and N candidate empirical medical named entities are obtained based on the similarity; the empirical medical named entity includes a named entity obtained from historical medical texts and classified into a standard medical named entity; N is a positive integer; Acquire specific features of the candidate empirical medical named entity; the specific features include the similarity between the candidate empirical medical named entity and the medical named entity to be identified and preset correlation features between the candidate empirical medical named entity and other named entities; Based on the specific features, the similarity between the candidate empirical medical named entity and the medical named entity to be identified is recalculated to obtain a candidate empirical medical named entity whose similarity with the medical named entity to be identified meets the preset conditions, and the medical named entity to be identified is classified into the standard medical named entity corresponding to the candidate empirical medical named entity that meets the preset conditions.
11. A method for identifying a named entity, characterized in that: The method comprises: Collecting text to be recognized, and obtaining named entities to be recognized from the text to be recognized; By matching the names of the named entities to be identified with the empirical named entities in the entity knowledge base, the similarity between the named entities to be identified and the empirical named entities is obtained, and N candidate empirical named entities are obtained based on the similarity; the empirical named entities include named entities obtained from historical texts and classified as standard named entities; N is a positive integer; Acquire specific features of the candidate experience named entity; the specific features include the similarity between the candidate experience named entity and the named entity to be identified and preset correlation features between the candidate experience named entity and other named entities; Based on the specific features, recalculate the similarity between the candidate empirical named entity and the named entity to be identified, obtain a candidate empirical named entity whose similarity with the named entity to be identified meets a preset condition, and classify the named entity to be identified into a standard named entity corresponding to the candidate empirical named entity that meets the preset condition; According to the classification result, the corresponding relationship between the named entity to be identified and the standard named entity is output.
12. A named entity recognition device, characterized in that: The device comprises: A name matching unit, used to obtain the similarity between the named entity to be identified and the empirical named entity by matching the named entity to be identified with the empirical named entity in the entity knowledge base; A first acquisition unit is used to acquire N candidate empirical named entities based on the similarity; the empirical named entities include named entities obtained from historical texts and classified as standard named entities; N is a positive integer; A second acquisition unit is used to acquire specific features of the candidate experience named entity; the specific features include the similarity between the candidate experience named entity and the named entity to be identified and the preset correlation features between the candidate experience named entity and other named entities; A calculation unit, configured to recalculate the similarity between the candidate empirical named entity and the named entity to be identified based on the specific feature, and obtain a candidate empirical named entity whose similarity with the named entity to be identified meets a preset condition; The classification unit is used to classify the to-be-recognized named entity into a standard named entity corresponding to the candidate empirical named entity that meets a preset condition.
13. A named entity recognition device, characterized in that: The device comprises: A first acquisition unit is used to acquire a medical named entity to be identified in the medical text to be identified; A name matching unit, used for obtaining the similarity between the to-be-identified medical named entity and the empirical medical named entity by matching the name of the to-be-identified medical named entity with the empirical medical named entity in the medical entity knowledge base; A second acquisition unit is used to acquire N candidate empirical medical named entities based on the similarity; the empirical medical named entities include named entities obtained from historical medical texts and classified into standard medical named entities; N is a positive integer; A third acquisition unit is used to acquire specific features of the candidate empirical medical named entity; the specific features include the similarity between the candidate empirical medical named entity and the medical named entity to be identified and the preset correlation features between the candidate empirical medical named entity and other named entities; A calculation unit, used for recalculating the similarity between the candidate empirical medical named entity and the medical named entity to be identified based on the specific feature, and obtaining a candidate empirical medical named entity whose similarity with the medical named entity to be identified meets a preset condition; The classification unit is used to classify the medical named entity to be identified into the standard medical named entity corresponding to the candidate empirical medical named entity that meets the preset conditions.
14. A named entity recognition device, characterized in that: The device comprises: A collection unit, used for collecting text to be recognized; A first acquisition unit, used for acquiring a named entity to be recognized from the text to be recognized; A name matching unit, configured to obtain a similarity between the named entity to be identified and the empirical named entity by performing name matching between the named entity to be identified and the empirical named entity in the entity knowledge base; A second acquisition unit is used to acquire N candidate empirical named entities based on the similarity; the empirical named entities include named entities obtained from historical texts and classified as standard named entities; N is a positive integer; A third acquisition unit is used to acquire specific features of the candidate experience named entity; the specific features include the similarity between the candidate experience named entity and the named entity to be identified and the preset correlation features between the candidate experience named entity and other named entities; A calculation unit, configured to recalculate the similarity between the candidate empirical named entity and the named entity to be identified based on the specific feature, and obtain a candidate empirical named entity whose similarity with the named entity to be identified meets a preset condition; A classification unit, used to classify the to-be-recognized named entity into a standard named entity corresponding to the candidate empirical named entity that meets a preset condition; The output unit is used to output the corresponding relationship between the named entity to be identified and the standard named entity according to the classification result.
15. A storage medium, characterized in that: The storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method for recognizing a named entity according to any one of claims 1 to 11.
16. An electronic device, characterized in that: The electronic device includes a storage medium and a processor; The processor is adapted to implement each instruction; The storage medium is suitable for storing a plurality of instructions; The instructions are suitable for being loaded by the processor and executing the method for recognizing a named entity as claimed in any one of claims 1 to 11.
Citation Information
Patent Citations
Method and apparatus for speech recognition
CN107016994A
News screening method and device, computer device and storage medium
CN109446336A