A Chinese entity recognition method, device, equipment and medium

Abstract

The application relates to the technical field of text labeling, in particular to a Chinese entity recognition method and device, equipment and medium, which performs character labeling on to-be-recognized text, obtains a first recognition entity set, labels a text category label, obtains a first text category set, obtains a second text category set according to a preset corresponding relationship, constructs a question set, obtains a second recognition entity set and a third recognition entity set, and finally performs fusion and discrimination processing on the first recognition entity set, the second recognition entity set and the third recognition entity set to obtain an entity recognition result of the to-be-recognized text. In this way, when entity types in the to-be-recognized text are similar, all entity types corresponding to the to-be-recognized text can be recognized, the to-be-recognized text is not disturbed by similar entity types, the accuracy of the Chinese recognition result is improved, the problem of entity nesting of the same entity corresponding to multiple entity categories is solved, and the accuracy and comprehensiveness of the Chinese entity recognition result are improved.

Description

Technical Field

[0001] The present application relates to the field of text annotation technology, and in particular to a Chinese entity recognition method, device, equipment and medium. Background Art

[0002] Chinese named entity recognition, as a basic technology in the field of natural language processing, is widely used in all aspects of social production and life, and also provides technical support for the development of technologies such as knowledge graphs and event extraction. At present, Chinese named entity recognition mainly adopts an end-to-end sequence labeling method based on pre-training technology, in which the entity recognition task is converted into a labeling problem, and each character in the text is labeled to obtain the identified entity. With the development of machine reading comprehension technology, Chinese named entity recognition can also be converted into reading comprehension for processing. By constructing questions corresponding to each entity category, the identified entities of the corresponding category are obtained according to the questions.

[0003] The above method still has the following defects:

[0004] 1. The situation where entity categories are similar cannot be handled well.

[0005] Since a single sequence labeling model needs to identify all entities at the same time, different categories of entities interfere with each other during the recognition process, which affects the accuracy of the recognition results.

[0006] 2. Unable to solve the problem of identifying nested entities.

[0007] There are situations where multiple entity categories correspond to multiple sequence labeling models. The sequence labeling model of single entity recognition cannot solve the entity nesting problem where the same entity corresponds to multiple entity categories. Summary of the Invention

[0008] The embodiments of the present application provide a Chinese entity recognition method, apparatus, device and medium for improving the accuracy and comprehensiveness of Chinese entity recognition results.

[0009] In a first aspect, an embodiment of the present application provides a Chinese entity recognition method, comprising:

[0010] Performing character annotation on the text to be recognized to obtain a first recognition entity set and a first entity type corresponding to each first recognition entity, wherein each first recognition entity represents a character with entity attributes contained in the text to be recognized;

[0011] For the text to be recognized, annotate the text category label according to the preset mapping relationship to obtain a first text category set, wherein each first text category representation, the text category label includes the first text category and the corresponding second entity type;

[0012] According to a preset correspondence, each first entity type obtained is converted into a corresponding second text category, and a second text category set is obtained;

[0013] For the first text category set and the second text category set, a question set is constructed, information matching question and answer processing is performed on the question set, and a second recognized entity set and a third recognized entity set containing answer entities are obtained respectively;

[0014] The first recognized entity set, the second recognized entity set, and the third recognized entity set are subjected to fusion and discrimination processing, and an entity recognition result of the text to be recognized is obtained.

[0015] In a second aspect, an embodiment of the present application provides a Chinese entity recognition device, which comprises:

[0016] The first obtaining module is configured to perform character labeling on the text to be recognized, and obtain a first recognized entity set and a first entity type corresponding to each first recognized entity, wherein each first recognized entity represents a character with an entity attribute contained in the text to be recognized.

[0017] The second obtaining module is configured to label text category labels on the text to be recognized according to a preset mapping relationship, and obtain a first text category set, wherein each first text category represents a first text category and a corresponding second entity type contained in the text category labels.

[0018] The conversion module is configured to convert each first entity type obtained into a corresponding second text category according to a preset correspondence, and obtain a second text category set.

[0019] The question and answer module is configured to construct a question set for the first text category set and the second text category set, perform information matching question and answer processing on the question set, and obtain a second recognized entity set and a third recognized entity set containing answer entities respectively.

[0020] The fusion module is configured to perform fusion and discrimination processing on the first recognized entity set, the second recognized entity set, and the third recognized entity set, and obtain an entity recognition result of the text to be recognized.

[0021] In an optional embodiment, character labeling is performed on the text to be recognized to obtain a first recognized entity set and a first entity type corresponding to each first recognized entity, and the first obtaining module is specifically configured to:

[0022] Each candidate character in the text to be recognized is labeled to obtain a plurality of labeling labels of each candidate character respectively, and each labeling label is used to represent an entity attribute and an entity type of the corresponding candidate character.

[0023] Based on each annotation label, target characters with entity attributes are screened out from each candidate character to generate a first recognition entity set, wherein the entity attributes include: being located at any position of the entity;

[0024] The corresponding entity type represented by the annotation tag corresponding to each target character is used as the first entity type corresponding to each target character.

[0025] In an optional embodiment, for the text to be recognized, the text category labels are marked according to a preset mapping relationship to obtain a first text category set, and the second acquisition module is specifically configured to:

[0026] According to a preset mapping relationship, the entire text in the text to be recognized is annotated with a text category label to obtain at least one text category label corresponding to the entire text;

[0027] Based on the first text category corresponding to each of the at least one text category labels, a first text category set is generated, where each first text category is used to represent a second entity type corresponding to the entire text.

[0028] In an optional embodiment, to obtain a preset mapping relationship, the first obtaining module is further configured to:

[0029] Based on the historical text collection, obtain the historical text category label corresponding to each historical text;

[0030] Based on semantic similarity, each historical text is clustered to obtain multiple historical text sets, wherein the semantic similarity between each historical text belonging to the same historical text set reaches a set threshold;

[0031] For each historical text collection, do the following:

[0032] Based on the historical text category labels corresponding to the respective historical texts contained in a historical text set, a corresponding historical text category label set is generated;

[0033] A mapping relationship is established between each historical text contained in the obtained historical text set and the historical text category label set.

[0034] In an optional embodiment, a question set is constructed for the first text category set and the second text category set, and the question-answering module is specifically configured to:

[0035] Combining each first text category in the first text category set with the text to be recognized based on the sentence structure to obtain a corresponding first sub-question and generate a first sub-question set;

[0036] Combining each second text category in the second text category set with the text to be recognized based on the sentence structure to obtain a corresponding second sub-problem and generate a second sub-problem set;

[0037] The first sub-problem set and the second sub-problem set are combined to generate a problem set.

[0038] In an optional embodiment, a question-answering process of information matching is performed on the question set to obtain a second set of recognized entities and a third set of recognized entities containing answer entities, respectively. The question-answering module is further configured to:

[0039] According to the historical question and answer records, each text in the to-be-recognized text is respectively determined to have a first semantic matching degree with each first sub-question in the first sub-question set, and the first target text whose first semantic matching degree reaches the set question and answer threshold is selected from the to-be-recognized text as the first answer entity to obtain the second recognition entity set;

[0040] According to the historical question and answer records, the second semantic matching degree of each text in the text to be recognized and each second sub-question in the second sub-question set is determined respectively, and the second target text whose second semantic matching degree reaches the set question and answer threshold is screened out from the text to be recognized as the second answer entity to obtain the third recognition entity set.

[0041] In an optional embodiment, when performing fusion discrimination processing on the first recognition entity set, the second recognition entity set, and the third recognition entity set to obtain the entity recognition result of the text to be recognized, the fusion module is specifically used to:

[0042] Obtaining a set of candidate entity types based on entity types that appear in the first identified entity set, the second identified entity set, and the third identified entity set;

[0043] For each candidate entity type, do the following:

[0044] Selecting, from the first recognition entity set, the second recognition entity set, and the third recognition entity set, recognition entities associated with a candidate entity type;

[0045] If it is determined that the three selected recognition entities are the same recognition entity, then a candidate entity type is used as a target entity type, and the same recognition entity is used as the entity recognition result;

[0046] If it is determined that there is at least one different identified entity among the three selected identified entities, a candidate entity type is used as a non-target entity type, and the three identified entities are determined not to belong to the entity recognition result.

[0047] In an optional embodiment, after obtaining the result to be recognized, the fusion module is further configured to:

[0048] If the recognition result does not contain any recognized entity, the second recognized entity set is used as the entity recognition result.

[0049] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0050] a memory for storing program instructions;

[0051] The processor is used to call the program instructions stored in the memory and execute any one of the Chinese entity recognition methods in any of the above aspects according to the obtained program instructions.

[0052] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method described in the first aspect.

[0053] In a fifth aspect, an embodiment of the present application further provides a computer program product, which includes: a computer program code, which, when executed on a computer, enables the computer to execute the method described in the first aspect.

[0054] The embodiment of the present application provides a Chinese entity recognition method, apparatus, device and medium, which performs character annotation on the text to be recognized to obtain a first recognition entity set, annotates the text category label according to a preset mapping relationship to obtain a first text category set, obtains a second text category set according to a preset corresponding relationship, constructs a question set for the first text category set and the second text category set, obtains a second recognition entity set and a third recognition entity set containing answer entities respectively, and finally performs fusion discrimination processing on the first recognition entity set, the second recognition entity set, and the third recognition entity set to obtain the entity recognition result of the text to be recognized. In this way, when the entity types in the text to be recognized are similar, all entity types corresponding to the text to be recognized can be identified without being disturbed by similar entity types, thereby improving the accuracy of the Chinese recognition results, solving the entity nesting problem of the same entity corresponding to multiple entity categories, and improving the accuracy and comprehensiveness of the Chinese entity recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 An application scenario of the Chinese entity recognition method provided in the embodiment of the present application;

[0056] Figure 2 A schematic diagram of a process for the Chinese entity recognition method provided in the embodiment of the present application Figure One ;

[0057] Figure 3 A flowchart of a Chinese entity recognition method provided for an embodiment of the present application Figure Two

[0058] Figure 4 A flowchart of a Chinese entity recognition method provided for an embodiment of the present application Figure Three

[0059] Figure 5 A principle diagram of a Chinese entity recognition method provided for an embodiment of the present application Figure One

[0060] Figure 6 A principle diagram of a Chinese entity recognition method provided for an embodiment of the present application Figure Two

[0061] Figure 7 A flowchart of a Chinese entity recognition method provided for an embodiment of the present application Figure Four

[0062] Figure 8 A flowchart of a Chinese entity recognition method provided for an embodiment of the present application Figure Five

[0063] Figure 9 A flowchart of a Chinese entity recognition method provided for an embodiment of the present application Figure Six

[0064] Figure 10 A flowchart of a Chinese entity recognition method provided for an embodiment of the present application Figure Seven

[0065] Figure 11 A flowchart of a Chinese entity recognition method provided for an embodiment of the present application Figure Eight

[0066] Figure 12 A principle diagram of a Chinese entity recognition method provided for an embodiment of the present application Figure Three

[0067] Figure 13 A principle diagram of a Chinese entity recognition method provided for an embodiment of the present application Figure Four

[0068] Figure 14 A principle diagram of a Chinese entity recognition method provided for an embodiment of the present application Figure Five

[0069] Figure 15 A principle diagram of a Chinese entity recognition method provided for an embodiment of the present application​​​​​​​​​​​​Figure Six ;

[0070] Figure 16 A schematic diagram of the principle of the Chinese entity recognition method provided in the embodiment of the present application Figure Seven ;

[0071] Figure 17 A schematic diagram of the principle of the Chinese entity recognition method provided in the embodiment of the present application Figure Eight ;

[0072] Figure 18 A schematic diagram of the principle of the Chinese entity recognition method provided in the embodiment of the present application Figure Nine ;

[0073] Figure 19 A schematic diagram of the principle of the Chinese entity recognition method provided in the embodiment of the present application Figure Ten ;

[0074] Figure 20 Schematic diagram of the structure of a Chinese entity recognition device in an embodiment of the present application;

[0075] Figure 21 This is a schematic diagram of the structure of an electronic device for Chinese entity recognition in an embodiment of the present application. DETAILED DESCRIPTION

[0076] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0077] In order to improve the accuracy and comprehensiveness of Chinese entity recognition results, the embodiments of the present application provide a Chinese entity recognition method, apparatus, device and medium, which performs character annotation on the text to be recognized to obtain a first recognition entity set, annotates text category labels according to a preset mapping relationship to obtain a first text category set, obtains a second text category set according to a preset correspondence relationship, constructs a question set for the first text category set and the second text category set, obtains a second recognition entity set and a third recognition entity set containing answer entities, and finally performs fusion discrimination processing on the first recognition entity set, the second recognition entity set, and the third recognition entity set to obtain the entity recognition result of the text to be recognized. In this way, when the entity types in the text to be recognized are similar, all entity types corresponding to the text to be recognized can be identified without being disturbed by similar entity types, thereby improving the accuracy of the Chinese recognition results, solving the entity nesting problem of the same entity corresponding to multiple entity categories, and improving the accuracy and comprehensiveness of the Chinese entity recognition results.

[0078] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are first explained below.

[0079] Entity: refers to any object with a specific set of attributes, generally including three major categories: entity class, time class, and number class, and seven minor categories: person name, organization name, place name, time, date, currency and percentage.

[0080] It should be noted that the above-mentioned naming method of technical terms is only an example, and the embodiments of the present application do not limit the naming method of the above-mentioned technical terms.

[0081] In the description of this application, words such as "first" and "second" are used only for descriptive purposes and should not be understood as indicating or implying relative importance or order. "Multiple" in this application can mean at least two, for example, two, three, or more, and this embodiment of the application does not impose any limitation.

[0082] See Figure 1 FIG2 is a schematic diagram of a possible application scenario in an embodiment of the present application, which includes multiple terminal devices 110 (including terminal device 1101, terminal device 1102, ..., terminal device 110n), a server 120, and a processing device 130.

[0083] In the embodiment of the present application, the terminal device 110 includes but is not limited to electronic devices such as desktop computers, mobile phones, mobile computers, tablet computers, media players, smart wearable devices, smart TVs, vehicle-mounted devices, and personal digital assistants (PDAs).

[0084] Server 120 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0085] Processing device 130 is a device that implements Chinese entity recognition in the embodiments of this application. Processing device 130 can be a designated server, or a separate server independent of the server, or a separate electronic device independent of the server. Processing device 130 is capable of identifying entities and entity types in the text to be recognized. Processing device 130 can specifically be an electronic device such as a desktop computer, mobile phone, mobile computer, tablet computer, or a server device such as an independent physical server or cloud server.

[0086] In an embodiment of the present application, a wired or wireless connection is used between the terminal device 110 and the server 120 to establish a communication connection through a communication network, and a wired or wireless connection is used between the processing device 130 and the server 120 to establish a communication connection through a communication network.

[0087] The technical solution proposed in this application can be used in various application scenarios to obtain a first recognition entity set, a second recognition entity set, and a third recognition entity set based on the text to be recognized, perform fusion judgment, and realize Chinese entity recognition of the text to be recognized.

[0088] In the following description of this application, the Chinese entity recognition process in this application will be described by taking the processing device 130 performing Chinese entity recognition on the text to be recognized as an example.

[0089] The following first describes the technical solutions provided by the embodiments of the present application in conjunction with the accompanying drawings.

[0090] See Figure 2 As shown, in a possible embodiment, in the preprocessing stage, the processing device obtains a preset mapping relationship and specifically performs the following steps:

[0091] Step 21: The processing device obtains the historical text category label corresponding to each historical text based on the historical text set.

[0092] Specifically, the processing device will process all historical texts to generate a historical text collection, and obtain the historical text category label corresponding to each historical text based on the historical annotation of the historical text category label corresponding to the historical text collection, wherein each historical text category label pair contains the corresponding historical text category.

[0093] Step 22: The processing device clusters the historical texts based on semantic similarity to obtain multiple historical text sets, wherein the semantic similarity between the historical texts belonging to the same historical text set reaches a set threshold.

[0094] Specifically, the processing device obtains the semantic similarity between all historical texts, and when the semantic similarity between multiple historical texts reaches 80%, the multiple historical texts are divided into a historical text set.

[0095] For example, assuming that the number of all historical texts is 7, specifically historical text 1, historical text 2, historical text 3, historical text 4, historical text 5, historical text 6, and historical text 7, the processing device obtains the semantic similarity between each two historical texts, and obtains the semantic similarity between historical text 1 and historical text 2, the semantic similarity between historical text 1 and historical text 3, and the semantic similarity between historical text 2 and historical text 3, as shown in Table 1:

[0096] Table 1

[0097] Historical Text Number Semantic Similarity Historical Text 1 and Historical Text 2 92％ Historical Text 1 and Historical Text 3 85％ Historical Text 2 and Historical Text 3 96％

[0098] As can be seen from the above table, the semantic matching degree between each two historical texts of historical text 1, historical text 2 and historical text 3 is greater than 80%. Therefore, the processing device divides historical text 1, historical text 2 and historical text 3 into a historical text set.

[0099] Step 23: The processing device performs the following operations on each historical text set:

[0100] Based on the historical text category labels corresponding to each historical text contained in a historical text set, a corresponding historical text category label set is generated; and a mapping relationship is established between each historical text contained in a historical text set and the historical text category label set.

[0101] Specifically, the processing device can obtain the historical text category labels corresponding to all historical texts contained in a historical text set as a historical text category label set based on the historical annotations of the historical text category labels corresponding to each historical text, and each historical text contained in a historical text set corresponds to the obtained historical text category label set.

[0102] For example, assuming that the processing device targets a historical text set {historical text 1, historical text 2, historical text 3}, and obtains historical text category label 1 corresponding to historical text 1, historical text category label 2 corresponding to historical text 2, and historical text category label 3 corresponding to historical text 3 based on the historical annotation of the historical text category labels corresponding to these three historical texts, and obtains a historical text category label set of {historical text category label 1, historical text category label 2, historical text category label 3}. Then, the three historical texts contained in a historical text set all correspond to the obtained historical text category label set, as shown in Table 2:

[0103] Table 2

[0104]

[0105] See Figure 3 As shown, the embodiment of the present application provides a Chinese entity recognition method, comprising the following steps:

[0106] Step 31: perform character annotation on the text to be recognized, and obtain a first recognition entity set and a first entity type corresponding to each first recognition entity, wherein each first recognition entity represents a character with entity attributes contained in the text to be recognized.

[0107] Specifically, the processing device adopts a sequence labeling model and a Begin-Inside-Outside (BIO) labeling method to label each character in the text to be recognized, and uses all the labeled labels as a labeling sequence. According to the labeling sequence, all characters with entity attributes are screened out as the first recognition entity set. At the same time, according to the labeling sequence, the first entity type corresponding to all characters with entity attributes can be obtained. In the BIO labeling method, B represents the beginning part of the entity, I represents the other part of the entity, and O represents the part that does not belong to the entity.

[0108] For example, suppose n entities are of type

[0109] {ENT1,ENT2,ENT3,…,ENT i ,…,ENT n};

[0110] The text to be recognized is:

[0111] S={C1,C2,C3,C4,C5,…,C n};

[0112] Through the BIO annotation method, the annotation sequence obtained is:

[0113]

[0114] Each character in the text to be recognized corresponds to each annotation label of the obtained annotation sequence, as shown in Table 3:

[0115] Table 3

[0116]

[0117] Finally, according to the obtained annotation sequence, the first recognized entities belonging to the first entity type ENT2 are C2 and C3, and the first recognized entity set is {C2, C3}.

[0118] Among them, ENT i (i=1, 2, ..., n) represents entity type i, S represents the text to be recognized, C i (i=1, 2, ..., n) represents the i-th character in the text to be recognized S, O represents that the marked character is not part of the entity, Indicates that the marked character is the beginning of an entity, and the corresponding entity type is ENT i , Indicates that the marked characters are the rest of the entity, and the corresponding entity type is ENT i See Figure 4 As shown, when executing step 31, the following steps are specifically performed:

[0119] Step 311: The processing device labels each candidate character in the text to be recognized, and obtains each label for each candidate character. Each label is used to represent the entity attribute and entity type of the corresponding candidate character.

[0120] Specifically, the processing device uses the BIO annotation method to annotate all candidate characters in the text to be recognized, obtains the annotation label corresponding to each candidate character according to the annotation results, and uses all the obtained annotation labels as a annotation sequence, wherein the annotation labels are divided into three categories: the annotated characters are parts that do not belong to the entity, the annotated characters are the starting part of the entity, and the annotated characters are other parts of the entity.

[0121] For example, assuming that there are four candidate characters in the text to be recognized S1 = {C1, C2, C3, C4}, the processing device uses the BIO annotation method to annotate the four candidate characters to obtain four annotation labels. Each candidate character corresponds to the corresponding annotation label one by one, as shown in Table 4:

[0122] Table 4

[0123]

[0124] The obtained 4 annotation labels are used as a annotation sequence Indicates that the marked character C2 is the beginning of the entity, and the corresponding entity type is ENT2. Indicates that the marked character C3 is the other part of the entity, and the corresponding entity type is ENT2.

[0125] Step 312: The processing device selects target characters with entity attributes from candidate characters based on the respective annotation tags, and generates a first recognition entity set, wherein the entity attributes include: being located at any position of the entity.

[0126] For details, see Figure 5 As shown, the processing device removes the characters corresponding to "the marked characters are not part of the entity" in the annotation tags according to the various annotation tags in the obtained annotation sequence, and filters out characters with entity attributes, including the characters corresponding to "the marked characters are the starting part of the entity" and the characters corresponding to "the marked characters are the other parts of the entity", and uses all the filtered characters with entity attributes as target characters, and uses all the obtained target characters to form a first recognition entity set, wherein the entity attributes include being located at any position of the entity, and being located at any position of the entity includes the starting part of the entity and the other parts of the entity.

[0127] For example, suppose the processing device follows the label sequence Among the four annotation labels, remove the two characters C1 and C4 corresponding to the annotation label O and filter out The corresponding characters C2 and The corresponding character C3, and C2 and C3 are used as target characters, then {C2, C3} is used as the first recognition entity set.

[0128] Step 313: The processing device uses the corresponding entity type represented by the annotation tag corresponding to each target character as the first entity type corresponding to each target character.

[0129] Specifically, the processing device extracts the corresponding entity type represented by each annotation according to each annotation label corresponding to each target character, as the first entity type corresponding to each target character.

[0130] For example, suppose the processing device identifies the target characters C2 and C3 according to their corresponding annotation labels. and Extract The entity type represented is ENT2, The corresponding entity type is ENT2, then the first entity type corresponding to the target character C2 is ENT2, and the first entity type corresponding to the target character C3 is ENT2.

[0131] Step 32: For the text to be recognized, mark the text category label according to the preset mapping relationship to obtain a first text category set, wherein each first text category represents the first text category and the corresponding second entity type included in the text category label.

[0132] For details, see Figure 6 As shown, if it is determined that the semantic matching degree between the text to be recognized and a historical text in a historical text set reaches 100%, the processing device will classify the text to be recognized into the corresponding historical text set and obtain the corresponding historical text category label set as the text category label set corresponding to the text to be recognized.

[0133] See Figure 7 As shown, when executing step 32, the following steps are specifically performed:

[0134] Step 321: The processing device labels the entire text in the text to be recognized with a text category label to obtain at least one text category label corresponding to the entire text.

[0135] Specifically, the processing device adopts a multi-label text classification model for the text to be recognized, classifies the text to be recognized into a corresponding historical text set, obtains a corresponding historical text category label set, and thereby obtains a text category label set corresponding to the text to be recognized, wherein the text category label set contains at least one text category label, and therefore, the text to be recognized corresponds to at least one text category label.

[0136] For example, suppose a multi-label text classification model is used for the text to be identified S1, and a text category label set is obtained as {S1 is S1 is }, then the text category label 1 "S1 is " indicates that the text category corresponding to the text S1 to be recognized is Text category label 2 "S1 is " indicates that the text category corresponding to the text S1 to be recognized is The text category Represented as containing entity type ENT i (i=1, 2, ..., n) text.

[0137] Step 322: The processing device generates a first text category set based on the first text category corresponding to each of the at least one text category labels, where each first text category is used to represent a second entity type corresponding to the entire text.

[0138] Specifically, the processing device extracts the first text category corresponding to each text category label from the text category label set, takes the extracted first text categories as a first text category set, and obtains the second entity type corresponding to each first text category in the first text category set based on the entity type represented by the text category.

[0139] For example, suppose the processing device receives a text category label set {S1 is S1 is }, extract the text category label 1 "S1 is "The first text category corresponding to Text category label 2 "S1 is "The first text category corresponding to Get the first text category set Thus, the first text category is obtained The corresponding second entity type is ENT2, the first text category The corresponding second entity type is ENT3.

[0140] Step 33: According to the preset correspondence, each of the obtained first entity types is converted into a corresponding second text category to obtain a second text category set.

[0141] Specifically, entity types correspond to text categories one by one, such as entity type ENT i (i=1, 2, ..., n) is converted to text category The first entity type corresponding to each first recognized entity is converted into a corresponding second text category, and each converted second entity category is used as a second text category set.

[0142] For example, assuming that the first entity type corresponding to the first recognized entity C2 in the first recognized entity set {C2, C3} is ENT2, the corresponding second text category is The first entity type corresponding to the first recognized entity C3 is ENT2, which is converted into the corresponding second text category: The second text category set is obtained as ENT2.

[0143] Step 34: Construct a question set for the first text category set and the second text category set, perform question-answering processing for information matching on the question set, and obtain a second recognition entity set and a third recognition entity set containing answer entities, respectively.

[0144] Specifically, the processing device generates a first sub-question set and a second sub-question set according to the sentence structure for the first text category set and the second text category set, combines the first sub-question set and the second sub-question set to construct a question set, and then obtains answer entities for the first sub-question set and the second sub-question set in the question set based on historical question and answer records.

[0145] See Figure 8 As shown, when executing step 34, the device construction problem set is processed, and the following steps are specifically performed:

[0146] Step 81: The processing device combines each first text category in the first text category set with the text to be recognized based on the sentence structure, obtains a corresponding first sub-question, and generates a first sub-question set.

[0147] For example, assuming that the processing device respectively processes the first text category set The two first text categories in and According to the combination of sentence structure and text to be recognized, two first sub-questions are obtained, namely "Find text containing entity type ENT2 in the text to be recognized?", "Find text containing entity type ENT3 in the text to be recognized?", then the first sub-question set is {"Find text containing entity type ENT2 in the text to be recognized?", "Find text containing entity type ENT3 in the text to be recognized?"}.

[0148] Step 82: The processing device combines each second text category in the second text category set with the text to be recognized based on the sentence structure, obtains corresponding second sub-questions, and generates a second sub-question set.

[0149] For example, assuming that the processing device respectively processes the second text category set The second text category in According to the combination of sentence structure and text to be recognized, a second sub-problem is obtained, which is "Find text containing entity type ENT2 in the text to be recognized?", so the second sub-problem set is {"Find text containing entity type ENT2 in the text to be recognized?"}.

[0150] Step 83: The processing device combines the first sub-question set and the second sub-question set to generate a question set.

[0151] For example, assume that the processing device combines the first sub-problem set {"Find the text containing the entity type ENT2 in the text to be recognized?", "Find the text containing the entity type ENT3 in the text to be recognized?"} with the second sub-problem set {"Find the text containing the entity type ENT2 in the text to be recognized?"} to obtain the question set {{"Find the text containing the entity type ENT2 in the text to be recognized?", "Find the text containing the entity type ENT3 in the text to be recognized?"}, {"Find the text containing the entity type EN2 in the text to be recognized?"}}.

[0152] See Figure 9 As shown, when executing step 34, a second recognition entity set and a third recognition entity set containing answer entities are obtained respectively, and the following steps are specifically performed:

[0153] Step 91: The processing device determines the first semantic matching degree of each text in the text to be recognized with each first sub-question in the first sub-question set based on the historical question and answer records, and selects the first target text whose first semantic matching degree reaches the set question and answer threshold from the text to be recognized as the first answer entity to obtain the second recognition entity set.

[0154] Specifically, the processing device applies a reading comprehension model to determine the first semantic matching degree of each text in the text to be recognized and each first sub-question in the first sub-question set, and selects the first target text with a first semantic matching degree of 100% from the text to be recognized as the first answer entity.

[0155] For example, assuming that the processing device applies a reading comprehension model and, based on historical question and answer records, determines that the first semantic matching degree of each of the four texts in the text to be recognized S1 = {C1, C2, C3, C4} with the first sub-question "Find the text containing the entity type ENT2 in the text to be recognized?" is 0, 100%, 0, 0, then the first target text obtained is C2; and determines that the first semantic matching degree of each of the four texts in the text to be recognized S1 = {C1, C2, C3, C4} with the first sub-question "Find the text containing the entity type ENT3 in the text to be recognized?" is 0, 0, 100%, 0, then the first target text obtained is C3; the representation of the question and answer result is:

[0156] {Find the text containing entity type ENT2 in the text to be recognized? ,S1,C2}

[0157] {Find the text containing entity type ENT3 in the text to be recognized? ,S1,C3};

[0158] Among them, {Find the text containing entity type ENT2 in the text to be recognized? , S1, C2} means:

[0159] Find the text containing entity type ENT2 in the text to be recognized S1 as C2.

[0160] {Find the text containing entity type ENT3 in the text to be recognized? ,S1,C3} means:

[0161] In the text to be recognized S1, the text containing the entity type ENT3 is found as C3, and the second recognized entity set obtained is {C2, C3}.

[0162] Step 92: The processing device determines the second semantic matching degree of each text in the text to be recognized and each second sub-question in the second sub-question set based on the historical question and answer records, and selects the second target text whose second semantic matching degree reaches the set question and answer threshold from the text to be recognized as the second answer entity, thereby obtaining a third recognition entity set.

[0163] Specifically, the processing device applies a reading comprehension model to determine the second semantic matching degree of each text in the text to be recognized and each second sub-question in the second sub-question set, and selects the second target text with a second semantic matching degree of 100% from the text to be recognized as the second answer entity.

[0164] For example, assuming that the processing device applies a reading comprehension model and, based on historical question-and-answer records, determines that the four texts in the to-be-recognized text S1 = {C1, C2, C3, C4} have a second semantic match of 0, 100%, 0, 0 for each of the four texts in the second sub-question "Find the text containing entity type ENT2 in the to-be-recognized text?", then the second target text obtained is C2; the question-and-answer result is represented as:

[0165] {Find the text containing entity type ENT2 in the text to be recognized? ,S1,C2};

[0166] Among them, {Find the text containing entity type ENT2 in the text to be recognized? , S1, C2} means:

[0167] In the text to be recognized S1, the text containing the entity type ENT2 is found as C2, and the third recognized entity set obtained is {C2}.

[0168] Step 35: Perform fusion discrimination processing on the first recognition entity set, the second recognition entity set, and the third recognition entity set to obtain entity recognition results of the text to be recognized.

[0169] Specifically, the processing device obtains the entity types that appear and the recognized entities associated with the entity types that appear as entity recognition results of the text to be recognized for the entity types in the first recognized entity set, the second recognized entity set, and the third recognized entity set.

[0170] See Figure 10 As shown, when executing step 35, the following steps are specifically performed:

[0171] Step 351: The processing device obtains a candidate entity type set based on the entity types that appear in the first identified entity set, the second identified entity set, and the third identified entity set.

[0172] Specifically, the processing device extracts entity types that appear in the first identified entity set, the second identified entity set, and the third identified entity set as candidate entity types, and generates a candidate entity type set.

[0173] For example, suppose the set of recognized entities is:

[0174]

[0175] Where p∈[1,2,3] represents the first, second and third identified entity sets respectively, q∈[1,2,3,…,n] represents the entity category to be identified, and p_n means that the pth identified entity set identifies num entities.

[0176] For the categories ENT in the three entity sets i Calculate the score:

[0177]

[0178]

[0179] when When the value is greater than or equal to the preset threshold of 0.5, it indicates that the text to be recognized must contain ENT i Otherwise, it cannot be confirmed whether the entity type exists in the text to be recognized.

[0180] Assuming the processing device targets ENT2, obtain The value of is 1, which is greater than the preset threshold 0.5. The entity types appearing in the first identified entity set {C2, C3}, the second identified entity set {C2, C3} and the third identified entity set {C2} are ENT2 and ENT3, then the candidate entity types are ENT2 and ENT3, and the obtained candidate entity type set is {ENT2, ENT3}.

[0181] Step 352: The processing device performs discrimination processing on each candidate entity type.

[0182] Specifically, the processing device performs discrimination processing on the entity type and the identified entity for each candidate entity type to obtain an entity recognition result.

[0183] See Figure 11 As shown, when executing step 352, the following steps are specifically performed:

[0184] Step 352 - a : The processing device selects a recognition entity associated with a candidate entity type from the first recognition entity set, the second recognition entity set, and the third recognition entity set, respectively.

[0185] For example, assume that the processing device selects the recognized entities associated with a candidate entity type ENT2 as C2, C2, and C2 from the first recognized entity set {C2, C3}, the second recognized entity set {C2, C3}, and the third recognized entity set {C2}, respectively.

[0186] Step 352 - b : If the processing device determines that the three selected recognition entities are the same recognition entity, the one candidate entity type is used as a target entity type, and the same recognition entity is used as the entity recognition result.

[0187] For example, assuming that the processing device determines that the three selected recognition entities C2, C2, and C2 are the same recognition entity C2, then the candidate entity type ENT2 can be used as the target entity type, and C2 can be used as the result of entity recognition.

[0188] Step 352 - c : If the processing device determines that there is at least one different identified entity among the three selected identified entities, a candidate entity type is treated as a non-target entity type, and the three identified entities are determined not to belong to the entity recognition result.

[0189] For example, suppose the processing device obtains The value of is 0.2, which is less than the preset threshold 0.5. The candidate entity type ENT3 is regarded as a non-target entity type, and the entity C3 corresponding to the non-target entity type ENT3 is not used as the entity recognition result.

[0190] After the processing device obtains the result to be identified, the processing device further includes:

[0191] If the recognition result does not include any recognized entity, the processing device uses the second recognized entity set as the entity recognition result.

[0192] Assume, for example, that if the result to be recognized does not contain any recognized entity, the processing device uses the second recognized entity set {C2, C3} as the entity recognition result, then the entity recognition result of the text to be recognized is specifically that the entity with entity type ENT2 is C2, and the entity with entity type ENT3 is C3.

[0193] The following describes in detail the Chinese entity recognition method provided in the embodiment of the present application through a specific embodiment.

[0194] Assume that the text to be recognized is "The tree next to Xiaoer's house is a book". The processing device performs Chinese entity recognition on the text to be recognized "The tree next to Xiaoer's house is a book". The specific Chinese entity recognition process is as follows:

[0195] First, the processing device needs to label each character in the text to be recognized "The Tree Next to Xiaoer's House is a Book" to obtain a label.

[0196] For example, see Figure 12 As shown, the processing device obtains all the annotation labels of the text to be recognized as: the starting part of the entity and the entity type is a human name, the other part of the entity and the entity type is a human name, the part that does not belong to the entity, the part that does not belong to the entity, the part that does not belong to the entity, the part that does not belong to the entity, the part that does not belong to the entity, the part that does not belong to the entity, the part that does not belong to the entity, the part that does not belong to the entity, the part that does not belong to the entity, the part that does not belong to the entity, the part that does not belong to the entity, the part that does not belong to the entity, the part that does not belong to the entity, the part that does not belong to the entity}.

[0197] Furthermore, the processing device filters out all characters with entity attributes in the text to be recognized "The Tree Next to Xiaoer's House is a Book" according to the annotation sequence as a first recognition entity set.

[0198] For example, see Figure 13 As shown, the processing device obtains the text to be recognized "The Tree Next to Xiaoer's House is a Book" based on the annotation sequence, and the character "小" is the beginning part of the entity and the entity type is a person's name, the character "二" is the other part of the entity and the entity type is a person's name, and the remaining characters correspond to parts that do not belong to the entity. The characters "小" and "二" located at any position of the entity are filtered out, and the first recognized entity set generated is {小,二}, and the first entity type corresponding to the first recognized entity "小" is "person's name", and the first entity type corresponding to the first recognized entity "二" is "person's name".

[0199] Secondly, the processing device labels the text to be identified, "The Tree Next to Xiaoer's House is a book," with a text category label according to a preset mapping relationship to obtain a first text category set.

[0200] For example, see Figure 14As shown, the processing device determines that the text to be identified "The Tree Next to Xiaoer's House is a book" belongs to the historical text set {Xiaoer, Xiaoer's house, the tree next to Xiaoer's house, "The Tree Next to Xiaoer's House" is a book}, and based on the historical text set, the corresponding historical text label set can be obtained {The text category corresponding to "The Tree Next to Xiaoer's House" is a book is "text containing entity type person's name", and the text category corresponding to "The Tree Next to Xiaoer's House" is a book is "text containing entity type book title"}, and the text category label 1 "The text category corresponding to "The Tree Next to Xiaoer's House" is a book is "text containing entity type person's name"" and the corresponding first text category is "text containing entity type person's name", and the corresponding second entity type is "name", and the text category label 2 "The text category corresponding to "The Tree Next to Xiaoer's House" is a book is "text containing entity type book title"" and the corresponding first text category is "text containing entity type book title", and the corresponding second entity type is "book title", and the first text category set is {text containing entity type person's name, text containing entity type book title}.

[0201] Thirdly, the processing device converts each of the obtained first entity types "personal names" into a corresponding second text category according to a preset correspondence relationship, and obtains a second text category set.

[0202] For example, see Figure 15 As shown, the processing device converts the obtained first entity type "personal name" into the corresponding second text category "text containing entity type person's name", and obtains the second text category set of {text containing entity type person's name}.

[0203] Afterwards, the processing device combines the two first text categories in the first text category set {text containing entity type person name, text containing entity type book title} with the text to be recognized according to the sentence structure, and obtains two first sub-problems; the processing device combines the second text category in the second text category set {text containing entity type person name} with the text to be recognized according to the sentence structure, and obtains a second sub-problem.

[0204] For example, see Figure 16As shown, the processing device combines the two first text categories, namely, text containing the entity type person name and text containing the entity type book title, in the first text category set {text containing the entity type person name, text containing the entity type book title} with the text to be recognized based on the sentence structure, and obtains two first sub-problems, namely, "Find text containing the entity type person name in "The Tree Next to Xiaoer's House is a Book"?," and "Find text containing the entity type book title in "The Tree Next to Xiaoer's House is a Book"?", thereby obtaining the first sub-problem set {Find text containing the entity type person name in "The Tree Next to Xiaoer's House is a Book"?, and "Find text containing the entity type book title in "The Tree Next to Xiaoer's House is a Book"}.

[0205] See Figure 17 As shown, the processing device combines the text containing the entity type person name in the second text category in the second text category set {text containing the entity type person name} with the text to be recognized based on the sentence structure, and obtains a second sub-problem: "Find text containing the entity type person name in "The Tree Next to Xiaoer's House is a Book"? ", thus obtaining the second sub-problem set {Find text containing the entity type person name in "The Tree Next to Xiaoer's House is a Book"?}.

[0206] Furthermore, the processing device applies a reading comprehension model to determine, based on historical question and answer records, the first semantic matching degree of each text in the text to be recognized, "The Tree Next to the Waiter's House is a Book", with each text having a first semantic matching degree with the two first sub-questions in the first sub-question set, and selects from the text to be recognized a first target text with a first semantic matching degree of 100%, as the first answer entity, to obtain a second recognition entity set; the processing device applies a reading comprehension model to determine, based on historical question and answer records, the second semantic matching degree of each text in the text to be recognized, with each text having a second semantic matching degree with one second sub-question in the second sub-question set, and selects from the text to be recognized a second target text with a second semantic matching degree of 100%, as the second answer entity, to obtain a third recognition entity set.

[0207] For example, see Figure 18 As shown, the processing device applies a reading comprehension model to determine the first semantic matching degree of each text in the text to be recognized with the two sub-questions in the first sub-question set, and selects the text with a first semantic matching degree of 100% with the question "Is "The Tree Next to Xiaoer's House" a book that contains the entity type person's name?" as Xiaoer, and takes "Xiaoer" as the first answer entity. The text with a first semantic matching degree of 100% with the question "Is "The Tree Next to Xiaoer's House" a book that contains the entity type book title?" is selected from the text to be recognized as the tree next to Xiaoer's house, and takes "the tree next to Xiaoer's house" as the first answer entity. The second recognition entity set obtained is {Xiaoer, the tree next to Xiaoer's house}.

[0208] See Figure 19 As shown, the processing device applies a reading comprehension model to determine the second semantic matching degree of each text in the text to be recognized with the second sub-problem in the second sub-problem set, and filters out the text with a first semantic matching degree of 100% with the question "Is "The Tree Next to Xiaoer's House" a book that contains the entity type person's name?" from the text to be recognized, which is Xiaoer, and "Xiaoer" is taken as the second answer entity, and the third recognition entity set obtained is {Xiaoer}.

[0209] Finally, the processing device performs fusion and discrimination processing on the first recognition entity set {小, 儿}, the second recognition entity set {小等, the tree next to 小等's house}, and the third recognition entity set {小等} to obtain the entity recognition result of the text to be recognized.

[0210] For example, the processing device selects the entity type as human name for the first recognition entity set, the second recognition entity set, and the third recognition entity set, and takes the common entity "Xiaoer" corresponding to the entity type human name in the three sets as the entity recognition result.

[0211] The processing device selects the entity type as book title for the first identified entity set, the second identified entity set, and the third identified entity set. However, the entity "book next to Xiao Er's house" corresponding to the entity type "book title" only appears in the second identified entity set and is not in the first identified entity set or the third identified entity set. Therefore, the "tree next to Xiao Er's house" cannot be used as an entity recognition result.

[0212] In summary, the entity corresponding to the Chinese entity recognition result of the text to be recognized "The Tree Next to Xiaoer's House is a Book" is {Xiaoer}, and the entity type corresponding to "Xiaoer" is a person's name.

[0213] In another case, when the result to be identified is empty, the second identification entity set is used as the entity recognition result of the text to be identified. Then, the entity corresponding to the Chinese entity recognition result of the text to be identified "The Tree Next to Xiaoer's House is a Book" is {Xiaoer, The Tree Next to Xiaoer's House}, and the entity type corresponding to "Xiaoer" is a person's name, and the entity type corresponding to "The Tree Next to Xiaoer's House" is a book title.

[0214] See Figure 20 As shown, an embodiment of the present application provides a Chinese entity recognition device, which includes: a first acquisition module 2001, a second acquisition module 2002, a conversion module 2003, a question and answer module 2004, and a fusion module 2005.

[0215] The first acquisition module 2001 is used to perform character annotation on the text to be recognized, obtain a first recognition entity set and a first entity type corresponding to each first recognition entity, wherein each first recognition entity represents a character with entity attributes contained in the text to be recognized.

[0216] The second acquisition module 2002 is used to annotate the text to be recognized with a text category label according to a preset mapping relationship to obtain a first text category set, wherein each first text category representation, the text category label includes a first text category and a corresponding second entity type.

[0217] The conversion module 2003 is configured to convert each of the obtained first entity types into a corresponding second text category according to a preset correspondence relationship, thereby obtaining a second text category set.

[0218] The question-answering module 2004 is used to construct a question set for the first text category set and the second text category set, perform question-answering processing of information matching on the question set, and obtain a second recognition entity set and a third recognition entity set containing answer entities respectively.

[0219] The fusion module 2005 is used to perform fusion discrimination processing on the first recognition entity set, the second recognition entity set, and the third recognition entity set to obtain entity recognition results of the text to be recognized.

[0220] In an optional embodiment, character annotation is performed on the text to be recognized to obtain a first recognition entity set and a first entity type corresponding to each first recognition entity. The first acquisition module 2001 is specifically configured to:

[0221] Annotate each candidate character in the text to be recognized, and obtain each annotation label of each candidate character, each annotation label is used to represent the entity attribute and entity type of the corresponding candidate character;

[0222] Based on each annotation label, target characters with entity attributes are screened out from each candidate character to generate a first recognition entity set, wherein the entity attributes include: being located at any position of the entity;

[0223] The corresponding entity type represented by the annotation tag corresponding to each target character is used as the first entity type corresponding to each target character.

[0224] In an optional embodiment, for the text to be recognized, the text category labels are marked according to a preset mapping relationship to obtain a first text category set. The second acquisition module 2002 is specifically configured to:

[0225] According to a preset mapping relationship, the entire text in the text to be recognized is annotated with a text category label to obtain at least one text category label corresponding to the entire text;

[0226] Based on the first text category corresponding to each of the at least one text category labels, a first text category set is generated, where each first text category is used to represent a second entity type corresponding to the entire text.

[0227] In an optional embodiment, to obtain a preset mapping relationship, the first obtaining module 2001 is further configured to:

[0228] Based on the historical text collection, obtain the historical text category label corresponding to each historical text;

[0229] Based on semantic similarity, each historical text is clustered to obtain multiple historical text sets, wherein the semantic similarity between each historical text belonging to the same historical text set reaches a set threshold;

[0230] For each historical text collection, do the following:

[0231] Based on the historical text category labels corresponding to the respective historical texts contained in a historical text set, a corresponding historical text category label set is generated;

[0232] A mapping relationship is established between each historical text contained in the obtained historical text set and the historical text category label set.

[0233] In an optional embodiment, a question set is constructed for the first text category set and the second text category set, and the question-answering module 2004 is specifically configured to:

[0234] Combining each first text category in the first text category set with the text to be recognized based on the sentence structure to obtain a corresponding first sub-question and generate a first sub-question set;

[0235] Combining each second text category in the second text category set with the text to be recognized based on the sentence structure to obtain a corresponding second sub-problem and generate a second sub-problem set;

[0236] The first sub-problem set and the second sub-problem set are combined to generate a problem set.

[0237] In an optional embodiment, question-answering processing of information matching is performed on the question set to obtain a second identified entity set and a third identified entity set containing answer entities, respectively. The question-answering module 2004 is further configured to:

[0238] According to the historical question and answer records, each text in the to-be-recognized text is respectively determined to have a first semantic matching degree with each first sub-question in the first sub-question set, and the first target text whose first semantic matching degree reaches the set question and answer threshold is selected from the to-be-recognized text as the first answer entity to obtain the second recognition entity set;

[0239] According to the historical question and answer records, the second semantic matching degree of each text in the text to be recognized and each second sub-question in the second sub-question set is determined respectively, and the second target text whose second semantic matching degree reaches the set question and answer threshold is screened out from the text to be recognized as the second answer entity to obtain the third recognition entity set.

[0240] In an optional embodiment, when performing fusion discrimination processing on the first recognition entity set, the second recognition entity set, and the third recognition entity set to obtain the entity recognition result of the text to be recognized, the fusion module 2005 is specifically used to:

[0241] Obtaining a set of candidate entity types based on entity types that appear in the first identified entity set, the second identified entity set, and the third identified entity set;

[0242] For each candidate entity type, do the following:

[0243] Selecting, from the first recognition entity set, the second recognition entity set, and the third recognition entity set, recognition entities associated with a candidate entity type;

[0244] If it is determined that the three selected recognition entities are the same recognition entity, then a candidate entity type is used as a target entity type, and the same recognition entity is used as the entity recognition result;

[0245] If it is determined that there is at least one different identified entity among the three selected identified entities, a candidate entity type is used as a non-target entity type, and the three identified entities are determined not to belong to the entity recognition result.

[0246] In an optional embodiment, after obtaining the result to be recognized, the fusion module 2005 is further configured to:

[0247] If the recognition result does not contain any recognized entity, the second recognized entity set is used as the entity recognition result.

[0248] Based on the same inventive concept as the above-mentioned application embodiment, the present application embodiment also provides an electronic device that can be used for Chinese entity recognition. In one embodiment, the electronic device can be a server, or a terminal device or other electronic device. In this embodiment, the structure of the electronic device can be as follows: Figure 21 As shown, it includes a memory 2101 , a communication interface 2103 and one or more processors 2102 .

[0249] Memory 2101 is used to store computer programs executed by processor 2102. Memory 2101 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and programs required for running instant messaging functions, while the data storage area may store various instant messaging messages and operating instruction sets.

[0250] Memory 2101 may be a volatile memory, such as random-access memory (RAM); a non-volatile memory, such as read-only memory, flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 2101 may be a combination of the above memories.

[0251] The processor 2102 may include one or more central processing units (CPUs) or digital processing units, etc. The processor 2102 is configured to implement the above-mentioned Chinese entity recognition method when calling the computer program stored in the memory 2101 .

[0252] The communication interface 2103 is used to communicate with terminal devices and other servers.

[0253] The specific connection medium between the memory 2101, the communication interface 2103 and the processor 2102 is not limited in the embodiment of the present application. Figure 21 In the embodiment, the memory 2101 and the processor 2102 are connected via a bus 2104. Figure 21 The connections between the other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus 2104 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 21 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0254] Based on the same inventive concept, an embodiment of the present application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the computer device performs any Chinese entity recognition method in the above-mentioned embodiment. The program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0255] Based on the same inventive concept, an embodiment of the present application further provides a computer program product, which, when called by a computer, enables the computer to execute the method described in the first aspect.

[0256] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure One a process or multiple processes and / or boxes Figure One A device that provides the functions specified in a block or multiple blocks.

[0257] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure One a process or multiple processes and / or boxes Figure One The function specified in one or more boxes.

[0258] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of user-operated steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure One a process or multiple processes and / or boxes Figure One A step that specifies a function in one or more boxes.

[0259] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A Chinese entity recognition method, characterized in that: include: Performing character annotation on the text to be recognized to obtain a first recognition entity set and a first entity type corresponding to each first recognition entity, wherein each first recognition entity represents a character with entity attributes contained in the text to be recognized; A multi-label text classification model is adopted for the text to be recognized, and the text to be recognized is classified into a corresponding historical text set. According to the mapping relationship established between the historical text set and the historical text category label set, a historical text category label set corresponding to the historical text set is obtained, and the corresponding historical text category label set is used as the text category label set corresponding to the text to be recognized; from the text category label set, the first text category corresponding to each text category label is extracted, and each extracted first text category is used as a first text category set, and according to the entity type represented by the text category, the second entity type corresponding to each first text category in the first text category set is obtained; According to a preset correspondence relationship, each of the obtained first entity types is converted into a corresponding second text category to obtain a second text category set; wherein the entity type corresponds to the text category one by one; Combining each first text category in the first text category set with the text to be recognized based on the sentence structure to obtain a corresponding first sub-question and generate a first sub-question set; wherein the first sub-question is a question asking about the entity type corresponding to the first text category; Combining each second text category in the second text category set with the text to be recognized based on the sentence structure to obtain corresponding second sub-questions and generate a second sub-question set; wherein the second sub-questions are questions about the entity type corresponding to the second text category; Combine the first sub-problem set and the second sub-problem set to generate a problem set; Performing question-answer processing for information matching on the question set to obtain a second recognition entity set and a third recognition entity set containing answer entities; The first recognition entity set, the second recognition entity set, and the third recognition entity set are subjected to fusion discrimination processing to obtain entity recognition results of the text to be recognized.

2. The method according to claim 1, wherein The step of performing character annotation on the text to be recognized to obtain a first recognition entity set and a first entity type corresponding to each first recognition entity includes: Annotate each candidate character in the to-be-recognized text to obtain an annotation tag for each candidate character, each annotation tag being used to represent an entity attribute and an entity type of the corresponding candidate character; Based on the respective annotation tags, target characters having entity attributes are screened out from the respective candidate characters to generate the first recognition entity set, wherein the entity attributes include: being located at any position of the entity; The corresponding entity type represented by the annotation tag corresponding to each target character is used as the first entity type corresponding to each target character.

3. The method according to claim 1 or 2, wherein: Obtaining the preset mapping relationship includes: Based on the historical text collection, obtain the historical text category label corresponding to each historical text; Clustering the historical texts based on semantic similarity to obtain multiple historical text sets, wherein the semantic similarity between the historical texts belonging to the same historical text set reaches a set threshold; For each historical text collection, do the following: Based on the historical text category labels corresponding to the respective historical texts contained in a historical text set, a corresponding historical text category label set is generated; A mapping relationship is established between each historical text included in the obtained historical text set and the historical text category label set.

4. The method according to claim 1, wherein The question-answering process of information matching is performed on the question set to obtain a second recognition entity set and a third recognition entity set containing answer entities, respectively, including: Determining, based on historical question-and-answer records, a first semantic matching degree between each text in the to-be-recognized text and each first sub-question in the first sub-question set, and selecting, from the to-be-recognized text, a first target text whose first semantic matching degree reaches a set question-and-answer threshold as a first answer entity, thereby obtaining a second set of recognized entities; According to the historical question and answer records, the second semantic matching degree of each text in the text to be recognized and each second sub-question in the second sub-question set is determined respectively, and the second target text whose second semantic matching degree reaches the set question and answer threshold is screened out from the text to be recognized as the second answer entity to obtain a third recognition entity set.

5. The method according to claim 1, wherein When the first recognition entity set, the second recognition entity set, and the third recognition entity set are subjected to fusion discrimination processing to obtain the entity recognition result of the text to be recognized, the method includes: Obtaining a candidate entity type set based on entity types that appear in the first identified entity set, the second identified entity set, and the third identified entity set; For each candidate entity type, do the following: Selecting, from the first recognition entity set, the second recognition entity set, and the third recognition entity set, a recognition entity associated with a candidate entity type; If it is determined that the three selected recognition entities are the same recognition entity, then the candidate entity type is used as a target entity type, and the same recognition entity is used as the entity recognition result; If it is determined that there is at least one different recognized entity among the three selected recognized entities, the one candidate entity type is regarded as a non-target entity type, and it is determined that the three recognized entities do not belong to the entity recognition result.

6. The method according to claim 5, wherein After obtaining the entity recognition result of the text to be recognized, the method further includes: If the recognition result does not include any recognized entity, the second recognized entity set is used as the entity recognition result.

7. A Chinese entity recognition device, characterized in that: include: A first acquisition module is configured to perform character annotation on the text to be recognized, and obtain a first recognition entity set and a first entity type corresponding to each first recognition entity, wherein each first recognition entity represents a character with entity attributes contained in the text to be recognized; The second acquisition module is used to adopt a multi-label text classification model for the text to be recognized, classify the text to be recognized into a corresponding historical text set, obtain a historical text category label set corresponding to the historical text set based on a mapping relationship established between the historical text set and the historical text category label set, and use the corresponding historical text category label set as a text category label set corresponding to the text to be recognized; extract the first text category corresponding to each text category label from the text category label set, use the extracted first text categories as a first text category set, and obtain the second entity type corresponding to each first text category in the first text category set based on the entity type represented by the text category; a conversion module, configured to convert each of the obtained first entity types into a corresponding second text category according to a preset correspondence relationship, thereby obtaining a second text category set; wherein the entity types correspond to the text categories one by one; a question-answering module, configured to combine each first text category in the first text category set with the text to be recognized based on the sentence structure, obtain corresponding first sub-questions, and generate a first sub-question set; wherein the first sub-questions are questions about the entity type corresponding to the first text category; Combining each second text category in the second text category set with the text to be recognized based on the sentence structure to obtain corresponding second sub-questions and generate a second sub-question set; wherein the second sub-questions are questions about the entity type corresponding to the second text category; Combine the first sub-problem set and the second sub-problem set to generate a problem set; Performing question-answer processing for information matching on the question set to obtain a second recognition entity set and a third recognition entity set containing answer entities; The fusion module is used to perform fusion and discrimination processing on the first recognition entity set, the second recognition entity set, and the third recognition entity set to obtain an entity recognition result of the text to be recognized.

8. The device according to claim 7, wherein The step of performing character annotation on the text to be recognized to obtain a first recognition entity set and a first entity type corresponding to each first recognition entity includes: Annotate each candidate character in the to-be-recognized text to obtain an annotation tag for each candidate character, each annotation tag being used to represent an entity attribute and an entity type of the corresponding candidate character; Based on the respective annotation tags, target characters having entity attributes are screened out from the respective candidate characters to generate the first recognition entity set, wherein the entity attributes include: being located at any position of the entity; The corresponding entity type represented by the annotation tag corresponding to each target character is used as the first entity type corresponding to each target character.

9. The device according to claim 7 or 8, characterized in that Obtaining the preset mapping relationship includes: Based on the historical text collection, obtain the historical text category label corresponding to each historical text; Clustering the historical texts based on semantic similarity to obtain multiple historical text sets, wherein the semantic similarity between the historical texts belonging to the same historical text set reaches a set threshold; For each historical text collection, do the following: Based on the historical text category labels corresponding to the respective historical texts contained in a historical text set, a corresponding historical text category label set is generated; A mapping relationship is established between each historical text included in the obtained historical text set and the historical text category label set.

10. The device according to claim 7, wherein The question-answering process of information matching is performed on the question set to obtain a second recognition entity set and a third recognition entity set containing answer entities, respectively, including: Determining, based on historical question-and-answer records, a first semantic matching degree between each text in the to-be-recognized text and each first sub-question in the first sub-question set, and selecting, from the to-be-recognized text, a first target text whose first semantic matching degree reaches a set question-and-answer threshold as a first answer entity, thereby obtaining a second set of recognized entities; According to the historical question and answer records, the second semantic matching degree of each text in the text to be recognized and each second sub-question in the second sub-question set is determined respectively, and the second target text whose second semantic matching degree reaches the set question and answer threshold is screened out from the text to be recognized as the second answer entity to obtain a third recognition entity set.

11. The device according to claim 7, wherein When the first recognition entity set, the second recognition entity set, and the third recognition entity set are subjected to fusion discrimination processing to obtain the entity recognition result of the text to be recognized, the method includes: Obtaining a candidate entity type set based on entity types that appear in the first identified entity set, the second identified entity set, and the third identified entity set; For each candidate entity type, do the following: Selecting, from the first recognition entity set, the second recognition entity set, and the third recognition entity set, a recognition entity associated with a candidate entity type; If it is determined that the three selected recognition entities are the same recognition entity, then the candidate entity type is used as a target entity type, and the same recognition entity is used as the entity recognition result; If it is determined that there is at least one different recognized entity among the three selected recognized entities, the one candidate entity type is regarded as a non-target entity type, and it is determined that the three recognized entities do not belong to the entity recognition result.

12. The device according to claim 11, wherein After obtaining the entity recognition result of the text to be recognized, the method further includes: If the recognition result does not include any recognized entity, the second recognized entity set is used as the entity recognition result.

13. An electronic device, characterized in that: include: a memory for storing program instructions; A processor is configured to call the program instructions stored in the memory, and execute the steps included in the method according to any one of claims 1 to 6 according to the obtained program instructions.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a computer, the computer is caused to perform the method according to any one of claims 1 to 6.

15. A computer program product, characterized in that The computer program product comprises: a computer program code, and when the computer program code is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 6.

Images