Named entity recognition method and apparatus

Abstract

The application relates to a named entity recognition method and device. The method comprises the following steps: splicing a to-be-recognized sentence and a plurality of questions respectively to obtain a plurality of spliced sentences; for any question in the plurality of questions, determining answer information of the question based on the spliced sentence corresponding to the question; processing the answer information of a first question based on the answer information of at least one second question to obtain a target answer of the first question, the first question being any question in the plurality of questions, and the second question being any question in the plurality of questions except the first question; and determining a named entity in the to-be-recognized sentence according to the target answers of the plurality of questions. The named entity recognition method and device provided in the application embodiment can improve the accuracy of recognizing embedded entities.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of entity recognition in the field of terminal artificial intelligence (AI), and in particular to a named entity recognition method and device. BACKGROUND

[0002] A named entity refers to a person name, an organization name, a location name, and other entities identified by a name. As an important step in the natural language processing process, named entity recognition is widely applied to information extraction, information retrieval, information recommendation and other tasks.

[0003] Due to the diversity of natural language, there are nested entities in a large amount of text. Here, the nested entity refers to a case where the whole constitutes an entity, and the inside also contains one or more simple entities. For example, the nested entity "Zhang Zhongjing Pharmacy" contains the person name entity "Zhang Zhongjing", and the nested entity "Nanjing Qixia District" contains the location entity "Nanjing" and the location entity "Qixia District". The result of nested entity recognition can be applied to intent classification, slot extraction and other natural language understanding problems. How to accurately recognize the nested entity is a problem to be solved at present. SUMMARY

[0004] Therefore, a named entity recognition method and device are provided, which can improve the accuracy of recognizing nested entities.

[0005] In a first aspect, an embodiment of the present application provides a named entity recognition method, which comprises: splicing a to-be-recognized sentence with a plurality of questions respectively to obtain a plurality of spliced sentences; for any question in the plurality of questions, determining answer information of the question based on the spliced sentence corresponding to the question, the answer information being used to indicate a start position and an end position of an answer of the question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position; processing the answer information of a first question based on the answer information of at least one second question to obtain a target answer of the first question, the first question being any question in the plurality of questions, and the second question being any question in the plurality of questions except the first question; and determining a named entity in the to-be-recognized sentence according to the target answers of the plurality of questions.

[0006] In the embodiment of the present application, the nested entity recognition can be implemented by splicing multiple questions and obtaining multiple answers; the answer to any question is processed according to the answers to other questions, so that when the accuracy of the answer to a question is uncertain, the answers to other questions can be used to assist in proving the accuracy of the answer to the question, thereby improving the accuracy of the answer to the question and the accuracy of the nested entity recognition.

[0007] According to the first aspect, in a first possible implementation of the named entity recognition method, the processing of the answer information of the first question based on the answer information of at least one second question to obtain the target answer of the first question comprises: determining at least one second answer based on the answer information of the at least one second question; determining a first answer based on the answer information of the first question; in a case where the at least one second answer and the first answer satisfy a positive filtering condition, determining the first answer as the target answer of the first question; or in a case where the at least one second answer and the first answer satisfy a negative filtering condition, verifying the first answer, and in a case where the verification is passed, determining the first answer as the target answer of the first question.

[0008] In the embodiment of the present application, by setting the positive filtering condition and the negative filtering condition, it can be determined whether the first answer can be directly used as the target answer of the first question, thereby improving the accuracy of the target answer and the accuracy of the nested entity recognition.

[0009] According to the first aspect, in a second possible implementation of the named entity recognition method, the processing of the answer information of the first question based on the answer information of at least one second question to obtain the target answer of the first question comprises: obtaining at least one second feature vector according to the answer information of the at least one second question, the second feature vector being used to indicate a start position and an end position of the answer to the second question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position; obtaining a first feature vector according to the answer information of the first question, the first feature vector being used to indicate a start position and an end position of the answer to the first question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position; performing convolution processing on the at least one second feature vector and the first feature vector to obtain a third feature vector; and determining the target answer of the first question according to the third feature vector.

[0010] In the embodiments of the present application, on the basis that a plurality of questions are spliced to identify a plurality of answers, the feature vector of the answer to one question can be adjusted according to the convolution result of the feature vector of the answer to another question and the feature vector of the question, thereby improving the accuracy of the answer to the question and improving the overall recognition accuracy of the to-be-recognized sentence.

[0011] According to the first aspect, in a third possible implementation of the named entity recognition method, the processing of the answer information of the first question based on the answer information of at least one second question to obtain the target answer of the first question comprises: obtaining at least one second feature vector according to the answer information of the at least one second question, the second feature vector being used to indicate the start position and the end position of the answer to the second question in the to-be-recognized sentence, and the first state probability corresponding to the start position and the second state probability corresponding to the end position; obtaining a first feature vector according to the answer information of the first question, the first feature vector being used to indicate the start position and the end position of the answer to the first question in the to-be-recognized sentence, and the first state probability corresponding to the start position and the second state probability corresponding to the end position; splicing the at least one second feature vector and the first feature vector to obtain a fourth feature vector; inputting the fourth feature vector into a pre-trained CRF model to obtain a fifth feature vector; and determining the target answer of the first question according to the fifth feature vector.

[0012] In the embodiments of the present application, on the basis that a plurality of questions are spliced to identify a plurality of answers, the feature vector of the answer to one question can be adjusted according to the convolution result of the feature vector of the answer to another question and the feature vector of the question, thereby improving the accuracy of the answer to the question and improving the overall recognition accuracy of the to-be-recognized sentence.

[0013] According to the first aspect, in a fourth possible implementation of the named entity recognition method, the determining of the answer information of the question based on the spliced sentence corresponding to the question comprises: inputting the spliced sentence corresponding to the question into a pre-trained entity extraction model to output a first vector and a second vector, the first vector being used to indicate the start position of the answer to the question in the to-be-recognized sentence and the third state probability corresponding to the start position, and the second vector being used to indicate the end position of the answer to the question in the to-be-recognized sentence and the fourth state probability corresponding to the end position; and determining the answer information of the question according to the first vector and the second vector.

[0014] In the embodiments of the present application, the pre-trained entity extraction model can be used to identify the answers to different questions, thereby realizing the identification of nested entities.

[0015] In a fifth possible implementation form of the named entity recognition method according to the fourth possible implementation form of the first aspect, the determining the answer information of the question according to the first vector and the second vector comprises: determining the answer information of the question according to the first vector, the second vector and a relative position matrix, wherein the relative position matrix is used to indicate a transition probability of a start position and an end position belonging to the same answer.

[0016] In the embodiments of the present application, the answers to the same question are corrected, so that the answers to each question are more consistent with the expectations, the accuracy of the answers to each question is improved, and the accuracy of identifying the nested named entities is improved.

[0017] In a sixth possible implementation form of the named entity recognition method according to the first possible implementation form of the first aspect, the forward filtering condition comprises: the position of the first answer is not overlapped with the position of the at least one second answer, or the position relationship between the first answer and the at least one second answer matches the range size relationship between the first question and the at least one second question; and the reverse filtering condition comprises: the position of the first answer is overlapped with the position of the at least one second answer, or the position relationship between the first answer and the at least one second answer does not match the range size relationship between the first question and the at least one second question.

[0018] In a seventh possible implementation form of the named entity recognition method according to the sixth possible implementation form of the first aspect, the position relationship between the first answer and the at least one second answer matching the range size relationship between the first question and the at least one second question comprises: the first answer and the at least one second answer are place names, the position of the first answer in the to-be-recognized sentence is before the position of the at least one second answer, the range corresponding to the first question is greater than the range of the at least one second question, or the position of the first answer in the to-be-recognized sentence is after the position of the at least one second answer, and the range corresponding to the first question is smaller than the range of the at least one second question.

[0019] In an eighth possible implementation form of the named entity recognition method according to the sixth possible implementation form of the first aspect, in a case where the at least one second answer and the first answer satisfy the reverse filtering condition, the first answer is verified, and in a case where the verification is passed, the first answer is determined as the target answer of the first question, comprising:

[0020] The first answer and the at least one second answer are multimedia resources, a dictionary check is performed on the majority of the first answers in a case that the at least one second answer overlaps with the position of the first answer, and the first answer is determined as the target answer of the first question in a case that the dictionary check passes.

[0021] According to the first aspect, or any possible implementation mode of the above first aspect, in a ninth possible implementation mode of the named entity recognition method, the plurality of questions belong to a plurality of preset categories, and / or different questions of a same preset category correspond to different granularities.

[0022] According to the second aspect, the embodiments of the present application provide a named entity recognition device, the device comprising: a splicing module configured to splice a to-be-recognized sentence and a plurality of questions respectively to obtain a plurality of spliced sentences; a first determination module configured to determine, for any question in the plurality of questions, answer information of the question based on a spliced sentence corresponding to the question obtained by the splicing module, the answer information being used to indicate a start position and an end position of an answer of the question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position; a processing module configured to process answer information of a first question determined by the first determination module based on answer information of at least one second question determined by the first determination module, to obtain a target answer of the first question, the first question being any question in the plurality of questions, and the second question being any question in the plurality of questions except the first question; and a second determination module configured to determine a named entity in the to-be-recognized sentence according to target answers of the plurality of questions obtained by the processing module.

[0023] According to the second aspect, in a first possible implementation mode of the named entity recognition device, the processing module is further configured to:

[0024] determine at least one second answer based on the answer information of the at least one second question;

[0025] determine a first answer based on the answer information of the first question;

[0026] in a case that the at least one second answer and the first answer satisfy a forward filtering condition, determine the first answer as a target answer of the first question;

[0027] or,

[0028] in a case that the at least one second answer and the first answer satisfy a reverse filtering condition, perform a check on the first answer, and in a case that the check passes, determine the first answer as the target answer of the first question.

[0029] According to a second aspect, in a second possible implementation form of the named entity recognition apparatus according to the second aspect, the processing module is further configured to:

[0030] obtain at least one second feature vector according to the answer information of the at least one second question, the second feature vector being used to indicate a start position and an end position of an answer to the second question in the sentence to be recognized, and a first state probability corresponding to the start position and a second state probability corresponding to the end position;

[0031] obtain a first feature vector according to the answer information of the first question, the first feature vector being used to indicate a start position and an end position of an answer to the first question in the sentence to be recognized, and a first state probability corresponding to the start position and a second state probability corresponding to the end position;

[0032] perform convolution processing on the at least one second feature vector and the first feature vector to obtain a third feature vector;

[0033] determine the target answer to the first question according to the third feature vector.

[0034] According to a second aspect, in a third possible implementation form of the named entity recognition apparatus according to the second aspect, the processing module is further configured to:

[0035] obtain at least one second feature vector according to the answer information of the at least one second question, the second feature vector being used to indicate a start position and an end position of an answer to the second question in the sentence to be recognized, and a first state probability corresponding to the start position and a second state probability corresponding to the end position;

[0036] obtain a first feature vector according to the answer information of the first question, the first feature vector being used to indicate a start position and an end position of an answer to the first question in the sentence to be recognized, and a first state probability corresponding to the start position and a second state probability corresponding to the end position;

[0037] concatenate the at least one second feature vector and the first feature vector to obtain a fourth feature vector;

[0038] input the fourth feature vector into a pre-trained CRF model to obtain a fifth feature vector;

[0039] determine the target answer to the first question according to the fifth feature vector.

[0040] According to a second aspect, in a fourth possible implementation form of the named entity recognition apparatus according to the second aspect, the first determining module is further configured to:

[0041] inputting the spliced sentence corresponding to the question into a pre-trained entity extraction model, outputting a first vector and a second vector, the first vector being used to indicate a start position of an answer to the question in the to-be-identified sentence and a third state probability corresponding to the start position, and the second vector being used to indicate a termination position of the answer to the question in the to-be-identified sentence and a fourth state probability corresponding to the termination position;

[0042] determining answer information of the question according to the first vector and the second vector.

[0043] According to a fourth possible implementation manner of the second aspect, in the fifth possible implementation manner of the named entity recognition device, the determining the answer information of the question according to the first vector and the second vector comprises:

[0044] determining the answer information of the question according to the first vector, the second vector and a relative position matrix, wherein the relative position matrix is used to indicate a transition probability that a start position and a termination position belong to a same answer.

[0045] According to the first possible implementation manner of the second aspect, in a sixth possible implementation manner of the named entity recognition device, the forward filtering condition comprises that a position of the first answer and a position of the at least one second answer are not overlapped, or a position relationship between the first answer and the at least one second answer matches a range size relationship between the first question and the at least one second question.

[0046] The backward filtering condition comprises that the position of the first answer and the position of the at least one second answer are overlapped, or the position relationship between the first answer and the at least one second answer does not match the range size relationship between the first question and the at least one second question.

[0047] According to the sixth possible implementation manner of the second aspect, in a seventh possible implementation manner of the named entity recognition device, the position relationship between the first answer and the at least one second answer matching the range size relationship between the first question and the at least one second question comprises:

[0048] the first answer and the at least one second answer are place names, the position of the first answer in the to-be-identified sentence is before the position of the at least one second answer, the range corresponding to the first question is greater than the range corresponding to the at least one second question, or the position of the first answer in the to-be-identified sentence is after the position of the at least one second answer, and the range corresponding to the first question is smaller than the range corresponding to the at least one second question.

[0049] In a sixth possible implementation of the second aspect, in an eighth possible implementation of the naming entity recognition apparatus, in a case where the at least one second answer and the first answer satisfy a reverse filtering condition, the first answer is verified, and in a case where the verification is passed, the first answer is determined as a target answer of the first question, including: the first answer and the at least one second answer are multimedia resources, in a case where the at least one second answer and the first answer overlap in position, a dictionary verification is performed on the majority of the first answer, and in a case where the dictionary verification is passed, the first answer is determined as the target answer of the first question.

[0050] In a ninth possible implementation of the second aspect, or any possible implementation of the second aspect, the plurality of questions belong to a plurality of preset categories, and / or different questions of a same preset category correspond to different granularities.

[0051] In a third aspect, an embodiment of the present application provides a naming entity recognition apparatus, including: a preprocessing module, configured to concatenate a to-be-recognized sentence with a plurality of questions respectively to obtain a plurality of concatenated sentences;

[0052] an entity extraction module, configured to determine, for any question in the plurality of questions, answer information of the question based on the concatenated sentence corresponding to the question obtained by the preprocessing module, the answer information being used to indicate a start position and an end position of an answer of the question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position;

[0053] a question-to-question reasoning module, configured to process, based on answer information of at least one second question obtained by the entity extraction module, answer information of a first question obtained by the entity extraction module to obtain a target answer of the first question, the first question being any question in the plurality of questions, and the second question being any question in the plurality of questions except the first question;

[0054] an entity determination module, configured to determine a naming entity in the to-be-recognized sentence according to target answers of a plurality of questions obtained by the question-to-question reasoning module.

[0055] In a first possible implementation of the third aspect, the question-to-question reasoning module is further configured to:

[0056] determine at least one second answer based on the answer information of the at least one second question;

[0057] determine a first answer based on the answer information of the first question;

[0058] In a case where the at least one second answer and the first answer satisfy a positive filtering condition, the first answer is determined as a target answer of the first question;

[0059] Or,

[0060] In a case where the at least one second answer and the first answer satisfy a reverse filtering condition, the first answer is verified, and in a case where the verification is passed, the first answer is determined as a target answer of the first question.

[0061] According to a third aspect, in a second possible implementation manner of the named entity recognition apparatus, the inter-question reasoning module is further configured to:

[0062] obtain at least one second feature vector according to answer information of the at least one second question, the second feature vector being used to indicate a start position and an end position of an answer of the second question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position;

[0063] obtain a first feature vector according to answer information of the first question, the first feature vector being used to indicate a start position and an end position of an answer of the first question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position;

[0064] perform convolution processing on the at least one second feature vector and the first feature vector to obtain a third feature vector;

[0065] determine a target answer of the first question according to the third feature vector.

[0066] According to a third aspect, in a third possible implementation manner of the named entity recognition apparatus, the inter-question reasoning module is further configured to:

[0067] obtain at least one second feature vector according to answer information of the at least one second question, the second feature vector being used to indicate a start position and an end position of an answer of the second question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position;

[0068] obtain a first feature vector according to answer information of the first question, the first feature vector being used to indicate a start position and an end position of an answer of the first question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position;

[0069] concatenate the at least one second feature vector and the first feature vector to obtain a fourth feature vector;

[0070] input the fourth feature vector into a pre-trained CRF model to obtain a fifth feature vector;

[0071] determine the target answer of the first question according to the fifth feature vector.

[0072] According to a fourth possible implementation of the third aspect, in a fourth possible implementation of the named entity recognition apparatus, the entity extraction module is further configured to:

[0073] input the concatenated sentence corresponding to the question into a pre-trained entity extraction model to output a first vector and a second vector, the first vector being used to indicate a start position of an answer to the question in the to-be-recognized sentence and a third state probability corresponding to the start position, and the second vector being used to indicate an end position of the answer to the question in the to-be-recognized sentence and a fourth state probability corresponding to the end position;

[0074] The apparatus further includes:

[0075] a question-in reasoning module configured to determine answer information of the question according to the first vector and the second vector;

[0076] The question-in reasoning module is further configured to process the answer information of the first question obtained by the question-in reasoning module based on the answer information of at least one second question obtained by the question-in reasoning module, to obtain a target answer of the first question, the first question being any question in the plurality of questions, and the second question being any question in the plurality of questions except the first question.

[0077] According to the fourth possible implementation of the third aspect, in a fifth possible implementation of the named entity recognition apparatus, the question-in reasoning module is further configured to:

[0078] determine the answer information of the question according to the first vector, the second vector, and a relative position matrix, the relative position matrix being used to indicate a transition probability of a start position and an end position belonging to a same answer.

[0079] According to the first possible implementation of the third aspect, in a sixth possible implementation of the named entity recognition apparatus, the forward filtering condition includes that a position of the first answer and a position of the at least one second answer do not overlap, or a position relationship of the first answer and the at least one second answer matches a range size relationship of the first question and the at least one second question.

[0080] The reverse filtering condition comprises that the position of the first answer overlaps the position of the at least one second answer, or the position relationship between the first answer and the at least one second answer does not match the range size relationship between the first question and the at least one second question.

[0081] According to a sixth possible implementation manner of the third aspect, in a seventh possible implementation manner of the naming entity recognition apparatus, the position relationship between the first answer and the at least one second answer matches the range size relationship between the first question and the at least one second question comprises that:

[0082] The first answer and the at least one second answer are place names, the position of the first answer in the sentence to be recognized is before the position of the at least one second answer, the range corresponding to the first question is greater than the range of the at least one second question, or the position of the first answer in the sentence to be recognized is after the position of the at least one second answer, and the range corresponding to the first question is smaller than the range of the at least one second question.

[0083] According to the sixth possible implementation manner of the third aspect, in an eighth possible implementation manner of the naming entity recognition apparatus, in the case that the at least one second answer and the first answer satisfy the reverse filtering condition, the first answer is verified, and in the case that the verification is passed, the first answer is determined as the target answer of the first question, comprising:

[0084] The first answer and the at least one second answer are multimedia resources, in the case that the position of the at least one second answer overlaps the position of the first answer, the first answer is subjected to a dictionary verification, and in the case that the dictionary verification is passed, the first answer is determined as the target answer of the first question.

[0085] According to the third aspect, or any one of the possible implementation manners of the third aspect, in a ninth possible implementation manner of the naming entity recognition apparatus, the plurality of questions belong to a plurality of preset categories, and / or different questions of the same preset category correspond to different granularities.

[0086] In a fourth aspect, an embodiment of the present application provides a naming entity recognition apparatus, which can execute the naming entity recognition method of the first aspect or one or more of the possible implementation manners of the first aspect.

[0087] The named entity recognition apparatus in the second aspect or one or more of the possible implementation manners of the second aspect, or the named entity recognition apparatus in the third aspect or one or more of the possible implementation manners of the third aspect, or the named entity recognition apparatus in one or more of the possible implementation manners of the fourth aspect can be a terminal device, a processor, a chip (for example, an AI chip), or a model (for example, an AI model).

[0088] In a fifth aspect, an embodiment of the present application provides a computer program product, including computer readable code or a non-volatile computer readable storage medium carrying computer readable code, when the computer readable code is run in an electronic device, a processor in the electronic device performs the named entity recognition method in the first aspect or one or more of the possible implementation manners of the first aspect.

[0089] These and other aspects of the application will become more fully understood from the following (a few) embodiment descriptions. BRIEF DESCRIPTION OF DRAWINGS

[0090] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate example embodiments, features, and aspects of the application and serve to explain the principles of the application.

[0091] Figure 1a An example schematic diagram showing multi-label annotation is shown;

[0092] Figure 1b An example schematic diagram showing reading comprehension annotation is shown;

[0093] Figure 1c Another example schematic diagram showing reading comprehension annotation is shown;

[0094] Figure 1d An example schematic diagram showing multiple named entities is shown;

[0095] Figure 2 An example schematic diagram showing the implementation process of the named entity recognition method provided by an embodiment of the present application is shown;

[0096] Figure 3 An example schematic diagram showing the structure of an electronic device is shown;

[0097] Figure 4 An example schematic diagram showing the flow of the named entity recognition method provided by an embodiment of the present application is shown;

[0098] Figure 5 An example schematic diagram showing the process of the entity extraction model processing the first spliced sentence is shown;

[0099] Figure 6Exemplary schematic diagram showing the relative position matrix of the embodiments of the present application;

[0100] Figure 7 Schematic process diagram showing the process of obtaining the probability corresponding to the first answer in the embodiments of the present application;

[0101] Figure 8 Schematic structural diagram showing the named entity recognition system provided by the embodiments of the present application;

[0102] Figure 9 Schematic structural diagram showing the named entity recognition device provided by the embodiments of the present application;

[0103] Figure 10 Schematic structural diagram showing the named entity recognition device provided by the embodiments of the present application. Detailed implementation manners

[0104] The following will describe in detail various exemplary embodiments, features and aspects of the present application with reference to the accompanying drawings. The same reference numerals in the drawings denote elements having the same or similar functions. Although various aspects of the embodiments are shown in the drawings, the drawings do not have to be drawn to scale unless otherwise specified.

[0105] The term "exemplary" used herein means "serving as an example, an embodiment or an illustration". Any embodiment described as "exemplary" herein does not have to be construed as superior or better than other embodiments.

[0106] In addition, for better illustration of the present application, numerous specific details are given in the following detailed implementation manners. Those skilled in the art should understand that the present application can also be implemented without some specific details. In some instances, methods, means, elements and circuits well known to those skilled in the art are not described in detail so as to highlight the gist of the present application.

[0107] In the related art, a multi-label based annotation method is applied to named entity recognition. First, the named entity is segmented into a combination of multiple minimum phrases. Then, each part of the phrase is segmented and annotated according to its position in the named entity, and each character in the phrase is further word-position annotated based on its position in the phrase; finally, the entire annotation is used as the annotation result of each character in the named entity. This annotation method enables each word in named entity recognition to correspond to multiple labels simultaneously, breaking through the limitation of single label. For example, "Zhang Zhongjing" in "Zhang Zhongjing Pharmacy" can correspond to both a person's name and a location. Figure 1a Show an exemplary schematic diagram of multi-label annotation. As Figure 1aAs shown, there are two labels for "Zhang", in which "B-PER" represents the first character of the person name, and "B-IOC" represents the first character of the place name. There are also two labels for "Zhong", in which "I-PER" represents the non-first character of the person name, and "I-LOC" represents the non-first character of the place name. It should be noted that, Figure 1a The label "O" shown represents a non-named entity.

[0108] In the above method, the label faces a combination explosion, so that the recognized named entity category is uncontrollable, and the annotation interruption is very serious. For example, if "Zhang Zhongjing" is mislabeled as [B-PER | B-LOC, I-PER, I-PER | I-LOC], the middle label is interrupted, resulting in the entire entity not being recognized, thus causing an error.

[0109] In the annotation mode based on reading comprehension, the to-be-recognized sentence is taken as reading material, and the question is concatenated behind, and the start position and the end position of the answer in the to-be-recognized sentence are predicted, so that one word in the to-be-recognized sentence corresponds to one label, avoiding the label combination explosion and the annotation interruption. Figure 1b An exemplary schematic diagram of reading comprehension annotation is shown. As shown in the figure, Figure 1b As shown, the to-be-recognized sentence "Nanjing weather how" is concatenated with the question "city location", and the start position (which can be recorded as Start point) of the answer is "Nan", and the end position (which can be recorded as End point) of the answer is "Jing". It should be noted that, Figure 1b As shown, "[CLS]" represents the beginning of the reading material and is placed at the beginning of the first sentence of the reading material; "[SEP]" is used to separate two sentences, such as separating the to-be-recognized sentence and the question; "[PAD]" is used for padding, which can make the concatenated sentence have a fixed length, for example, 64 characters.

[0110] In the embodiment of the present application, by concatenating different questions after the to-be-recognized sentence, different answers can be obtained, and these answers can be nested, so that the recognition of nested entities can be realized. Figure 1c Another exemplary schematic diagram of reading comprehension annotation is shown. As shown in the figure, Figure 1cAs shown, the question "city location" is concatenated after the to-be-identified sentence "Nanjing Qixia District, how is the weather?", and the start position of the answer is predicted to be "Nanjing", and the end position of the answer is predicted to be "District". The question "county location" is concatenated after the to-be-identified sentence "Nanjing Qixia District, how is the weather?", and the start position of the answer is predicted to be "Qixia", and the end position of the answer is predicted to be "District". The question "complete location" is concatenated after the to-be-identified sentence "Nanjing Qixia District, how is the weather?", and the start position of the answer is predicted to be "Nanjing", and the end position of the answer is predicted to be "District". As can be seen, based on the reading comprehension annotation mode, the nested entity "Nanjing Qixia District" and the simple entities "Nanjing" and "Qixia District" inside it are identified.

[0111] When different questions are concatenated after the to-be-identified sentence, the focus of reading comprehension is different, and some structure information of the to-be-identified sentence is lost. For example, the front-back relationship between the city location and the county location, the front-back relationship between the album name and the song name, and the like. The named entity recognition method provided in the embodiments of the present application identifies the answers to multiple questions by concatenating multiple questions in a reading comprehension manner; after the answer to a question is identified, the answer to the question can be processed according to the answers to other questions, so that when it is not certain whether the answer to the question is accurate, the answers to other questions are used to assist in proving whether the answer is accurate, thereby improving the accuracy of the answer to the question, improving the accuracy of identifying the nested entity, and further improving the overall accuracy of identifying the to-be-identified sentence.

[0112] The named entity recognition method provided in the embodiments of the present application can be applied to an electronic device such as a terminal device or a server, or a virtual machine (VM) or a container on a cloud (public cloud or private cloud). The terminal device (which can refer to a device having a wireless connection function, and the wireless connection function refers to the ability to connect to other terminal devices through wireless connection modes such as wifi and Bluetooth. The terminal device of the present application can also have the function of wired connection for communication. The terminal device of the present application can be touch screen, non-touch screen, or even without a screen. The touch screen can be controlled by clicking, sliding, and the like on the display screen through a finger, a stylus, or the like. The non-touch screen device can be connected to a mouse, a keyboard, a touch panel, and the like. The device without a screen can be a Bluetooth speaker without a screen, and the like. For example, the terminal device of the present application can be a smart phone, a netbook, a tablet computer, a notebook computer, a wearable electronic device (such as a smart bracelet, a smart watch, and the like), a virtual reality device, a smart speaker, and the like. The cloud can be a cluster composed of multiple electronic devices.

[0113] Figure 2 An implementation process schematic diagram of the named entity recognition method provided in the embodiments of the present application is shown. As shown in FIG. 1, the named entity recognition method provided in the embodiments of the present application includes the following steps:Figure 2 As shown, after the electronic device receives the input sentence to be recognized, the electronic device outputs the recognition result of the input sentence to be recognized. In one example, the recognition result can include named entities. For example, "Nanjing" and "Qixia District". In another example, the recognition result can include named entities and categories of the named entities. For example, the city location "Nanjing" and the county location "Qixia District".

[0114] In the embodiments of the present application, taking a smart speaker as an example of the electronic device, the user gives a voice instruction "Nanjing Qixia District weather how", the smart speaker takes "Nanjing Qixia District weather how" as the input sentence to be recognized, and identifies the complete location "Nanjing Qixia District", the city location "Nanjing", and the county location "Qixia District" from the input sentence to be recognized, and then finds the weather of Nanjing Qixia District and feeds back to the user. Taking a server as an example of the electronic device, the user gives a voice instruction "Nanjing Qixia District weather how", the smart speaker takes "Nanjing Qixia District weather how" as the input sentence to be recognized and sends it to the server, the server identifies the complete location "Nanjing Qixia District", the city location "Nanjing", and the county location "Qixia District" from the input sentence to be recognized, and feeds back the above locations to the smart speaker, and the smart speaker finds the weather of Nanjing Qixia District and feeds back to the user.

[0115] Figure 3 A structural schematic diagram of the electronic device is shown. As shown in the figure, Figure 3 The electronic device can include at least one processor 301, a memory 302, an input / output device 303, and a bus 304. The various components of the electronic device will be described in detail below. Figure 3 The various components of the electronic device will be described in detail below.

[0116] The processor 301 is the control center of the electronic device, which can be one processor or a combination of multiple processing elements. For example, the processor 301 is a CPU, or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement one or more embodiments of the present disclosure, such as one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs).

[0117] The processor 301 can perform various functions of the electronic device by running or executing software programs stored in the memory 302 and calling data stored in the memory 302.

[0118] In a specific implementation, as one embodiment, the processor 301 can include one or more CPUs, such as CPU 0 and CPU 1 shown in the figure.

[0119] In particular implementations, as one example, an electronic device can include multiple processors, such as the processor 301 and the processor 305 shown in FIG. 3. Each of these processors can be a single-CPU or a multi-CPU. A processor here can refer to one or more devices, circuits, and / or processing cores for processing data, such as computer program instructions. Figure 3

[0120] The memory 302 can be a Read-Only Memory (ROM) or other type of static storage device that can store static information and instructions, a Random Access Memory (RAM), or other type of dynamic storage device that can store information and instructions, an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, a magnetic disk storage or other magnetic storage devices, or any other medium capable of storing desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto. The memory 302 can exist independently of the processor 301, and can be connected to the processor 301 via the bus 304. The memory 302 can also be integrated with the processor 301.

[0121] The input / output device 303 is configured to communicate with other devices or communication networks, such as an Ethernet, a Radio access network (RAN), a Wireless Local Area Networks (WLAN), and the like. The input / output device 303 can include all or part of a baseband processor, and can optionally include a Radio Frequency (RF) processor. The RF processor is configured to transceive RF signals, and the baseband processor is configured to implement processing of baseband signals converted from or to be converted to the RF signals.

[0122] In particular implementations, as one example, the input / output device 303 can include a transmitter and a receiver. The transmitter is configured to transmit signals to other devices or communication networks, and the receiver is configured to receive signals transmitted by other devices or communication networks. The transmitter and the receiver can exist independently, or can be integrated together.​

[0123] The bus 304 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or other types of bus. Figure 3 In the figure, only one thick line is used to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0124] Figure 3 The device structure shown in the figure does not constitute a limitation on electronic devices, and can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0125] Figure 4 A flowchart of a naming entity recognition method provided by an embodiment of the present application is shown. The method can be applied to Figure 3 The electronic device shown. As Figure 4 shown, the method can include:

[0126] In step S401, the to-be-identified sentence is spliced with a plurality of questions to obtain a plurality of spliced sentences.

[0127] The first question is any question in the plurality of questions, and the second question is any question in the plurality of questions except the first question. There can be one or more second questions.

[0128] The to-be-identified sentence can be used to represent a sentence with a naming entity recognition requirement. The to-be-identified sentence can be any natural language sentence. The to-be-identified sentence can include one or more sentences. For example, "How is the weather in Nanjing?", "How is the weather in Nanjing Xisha District?", "Send a message to the caterpillar, the caterpillar, and the monkey to have dinner at 6 pm tonight", "I am in Los Angeles, how is the weather in Nanjing?", "Navigate to Zhang Zhongjing Pharmacy", etc. can be used as the to-be-identified sentence.

[0129] The first question can represent any question spliced with the to-be-identified sentence. The spliced sentence obtained by splicing the to-be-identified sentence with the first question can be referred to as the first spliced sentence. The second question can represent a question different from the first question. The spliced sentence obtained by splicing the to-be-identified sentence with the second question can be referred to as the second spliced sentence. There can be one or more second questions. Correspondingly, there can be one or more second spliced sentences. It can be understood that the number of second questions is consistent with the number of second spliced sentences.

[0130] In a possible implementation, the plurality of questions can belong to a plurality of preset categories. In the embodiment of the present application, the questions (including the first question and the second question) can be pre-set as needed. For example, to identify the names of persons and places, questions of two preset categories of "person name" and "place name" can be set. In a possible implementation, different questions of the same preset category correspond to different granularities. For example, for the preset category of "place name", six granularity questions of "complete place", "country", "city", "county", and "street" can be set. The above is only an exemplary description of the preset categories and granularities, and the preset categories and granularities can also be other, for example, the preset categories can also include multimedia resources, institutions, dates, etc., and the granularities can also include albums and songs of the "multimedia resource" category, TV series and movies, etc., which are not limited by the present application. In addition, in the implementation, a question can be a word, a declarative sentence, a question, or a label, etc. The embodiment of the present application does not limit the form of the question. For example, the question can be "person name", or "which person?", or "character", etc.

[0131] In step S402, for any question in the plurality of questions, based on the spliced sentence corresponding to the question, the answer information of the question is determined.

[0132] The answer information is used to indicate the start position and the end position of the answer of the question in the to-be-recognized sentence, and the first state probability corresponding to the start position and the second state probability corresponding to the end position.

[0133] In a possible implementation, step S402 can include: inputting the spliced sentence corresponding to the question into a pre-trained entity extraction model, outputting a first vector and a second vector; determining the answer information of the question according to the first vector and the second vector. The first vector is used to indicate the start position of the answer of the question in the to-be-recognized sentence and the third state probability corresponding to the start position, and the second vector is used to indicate the end position of the answer of the question in the to-be-recognized sentence and the fourth state probability corresponding to the end position.

[0134] For the first question, the answer information of the first question can be determined based on the first spliced sentence. For the second question, the answer information of the second question can be determined based on the second spliced sentence. Taking the first question as an example, step S402 can include inputting the first spliced sentence into a pre-trained entity extraction model, and outputting a first vector and a second vector. At this time, the first vector is used to indicate the start position of the answer to the first question in the to-be-recognized sentence and a third state probability corresponding to the start position, and the second vector is used to indicate the end position of the answer to the first question in the to-be-recognized sentence and a fourth state probability corresponding to the end position. Taking the second question as an example, step S402 can include inputting the second spliced sentence into a pre-trained entity extraction model, and outputting a first vector and a second vector corresponding to the second spliced sentence. At this time, the first vector is used to indicate the start position of the answer to the second question in the to-be-recognized sentence and a third state probability corresponding to the start position, and the second vector is used to indicate the end position of the answer to the second question in the to-be-recognized sentence and a fourth state probability corresponding to the end position.

[0135] The processing process of the entity extraction model on the first spliced sentence is described below. The processing process of the entity extraction model on the second spliced sentence can refer to the processing process of the entity extraction model on the first spliced sentence, which will not be described here.

[0136] In the embodiment of the present application, the entity extraction model can be pre-trained. After the first spliced sentence is input into the entity extraction model, the entity extraction model can label the start position and the end position of the answer to the first question in the to-be-recognized sentence based on the reading comprehension mode, and can output the third state probability corresponding to each start position and the fourth state probability corresponding to each end position. Here, the third state probability represents the probability that a position is the start position of the answer. The higher the third state probability of a position, the greater the probability that the position is the start position of the answer. The lower the third state probability of a position, the smaller the probability that the position is the start position of the answer. The fourth state probability represents the probability that a position is the end position of the answer. The higher the fourth state probability of a position, the greater the probability that the position is the end position of the answer. The lower the fourth state probability of a position, the smaller the probability that the position is the end position of the answer. In one example, the state probability (including the first state probability, the second state probability, the third state probability, and the fourth state probability) can be a probability value that has not been normalized, or can be a probability value that has been normalized, which is not limited by the present application.

[0137] In a possible implementation, the entity extraction model can be a bidirectional encoder representation from transformers (BERT) model.Figure 5 A process diagram of the entity extraction model processing the first spliced sentence is shown. As shown in Figure 5 the to-be-recognized sentence is "Nanjing Qixia District, how to add", the first question is "city location", and the first spliced sentence obtained by splicing is "[CLS] Nanjing Qixia District weather how [SEP] city location [SEP]". It should be noted that [CLS] is used to identify the first character of the to-be-recognized sentence, and [SEP] is used to separate two sentences. In actual applications, the input of the bert model is fixed-length. When the length of the first spliced sentence does not reach the input requirement of the bert model, [PAD] needs to be filled after the first spliced sentence. When the length of the first spliced sentence is greater than the input requirement of the bert model, the first spliced sentence needs to be truncated. In an example, the length of the sentence input by the bert model can be 64 characters or 32 characters, etc. In addition, the [CLS], [SEP] and [PAD] marks in the first spliced sentence can be different for different entity extraction models. For the bert model, [CLS], [SEP] and [PAD] marks can be used. For other models capable of entity recognition, other marks can be used, which are not limited by the present application.

[0138] As shown in Figure 5 , the first spliced sentence is input into the bert model, and the bert model obtains the word vector, text vector and position vector of each word in the first spliced sentence. Among them, the word vector represents a unique word. In an example, the word vector of each word can be obtained by Word2Vec. The text vector is used to identify the sentence to which the word belongs. In Figure 5 , "[CLS] Nanjing Qixia District weather how [SEP]" is the first sentence, and "city location [SEP]" is the second sentence. The text vector corresponding to each word in the first sentence is denoted by EA, and the text vector corresponding to each word in the second sentence is denoted by EB. The position vector is used to identify the position of the word in the entire first spliced sentence. Referring to Figure 5, "[CLS]" is represented by E0 to indicate that it is in the 0th position in the first spliced sentence, and "South" is represented by E1 to indicate that it is in the 1st position in the first spliced sentence. As can be seen, based on the word vector corresponding to a word, the content of the word can be determined, based on the text vector corresponding to a word, it can be determined that the word belongs to which sentence in the first spliced sentence, and based on the position vector corresponding to a word, the position of the word in the first spliced sentence can be determined. In the bert model, for each word in the first spliced sentence, the word vector, the text vector and the position vector corresponding to the word are spliced together to obtain the spliced vector corresponding to the word. Then, based on the spliced vector corresponding to each word in the first spliced sentence, the bert model obtains the probability that each word in the first spliced sentence is the starting position of the answer to the first question (i.e. the third state probability) and the probability that each word in the first spliced sentence is the ending position of the answer to the first question (i.e. the fourth state probability). In Figure 5 , P10 represents the probability that the word at position 0 in the first spliced sentence is the starting position of the answer to the first question, P11 represents the probability that the word at position 1 in the first spliced sentence is the starting position of the answer to the first question, P20 represents the probability that the word at position 0 in the first spliced sentence is the ending position of the answer to the first question, P21 represents the probability that the word at position 1 in the first spliced sentence is the ending position of the answer to the first question, and so on. Here, no further description is given.

[0139] In a possible implementation, the first vector can indicate the third state probability corresponding to each word in the sentence to be recognized (for example, P11 to P19 as shown in Figure 5 ), and the second vector can indicate the fourth state probability corresponding to each word in the sentence to be recognized (for example, P21 to P29 as shown in Figure 5 ). On this basis, the word whose third state probability is greater than the first threshold value can be output as the starting position, and the word whose fourth state probability is greater than the second threshold value can be output as the ending position. The first threshold value and the second threshold value can be set as needed, and the first threshold value and the second threshold value can be the same or different. In one example, the first threshold value can be 85%, and the second threshold value can be 90%. The present application does not make any limitation in this regard.

[0140] In a possible implementation, the word whose third state probability is greater than the first threshold value can be determined as the starting position, and the word whose fourth state probability is greater than the second threshold value can be determined as the ending position. The first vector can indicate the third state probability corresponding to each word determined as the starting position in the sentence to be recognized (for example, P11 and P13 as shown in Figure 5 ), and the second vector can indicate the fourth state probability corresponding to each word determined as the ending position in the sentence to be recognized (for example, P22 and P25 as shown in Figure 5 ).

[0141] So far, the first vector and the second vector corresponding to the first question can be obtained. Then, the answer information of the first question can be determined according to the first vector and the second vector corresponding to the first question. Similarly, the answer information of each question can be obtained according to the first vector and the second vector corresponding to each question.

[0142] In a possible implementation, determining the answer information of the question according to the first vector and the second vector can include: determining the start position indicated by the first vector as the start position indicated in the answer information, and determining the third state probability indicated by the first vector as the first state probability indicated in the answer information; and determining the end position indicated by the second vector as the end position indicated in the answer information, and determining the fourth state probability indicated by the second vector as the second state probability indicated in the answer information.

[0143] Considering that there can be multiple named entities in a to-be-recognized sentence, for example, multiple personal names, multiple place names, multiple cities, or multiple counties, and the like. Therefore, there can be multiple start positions and end positions for one question. Figure 1d An exemplary schematic diagram of multiple named entities is shown. As shown in Figure 1d The to-be-recognized sentence "send a message to the caterpillar, the caterpillar worm, and the monkey to have dinner at 6 o'clock tomorrow night" includes multiple personal names, namely "caterpillar", "caterpillar worm", and "monkey". In this scenario, as shown in Figure 1d If the question "personal name" is concatenated behind the to-be-recognized sentence based on the annotation manner of reading comprehension, multiple start positions (the first character of "caterpillar", the first character of "caterpillar worm", and the first character of "monkey") and multiple end positions (the last character of "caterpillar", the last character of "caterpillar worm", and the last character of "monkey") can be output. At this time, if the longest answer (i.e., the distance between the start position and the end position is the largest) "caterpillar, caterpillar worm, and monkey" is output, the situation that the nested entity is not accurately identified exists, the rationality is poor, it does not meet the user's expectation, and the user experience is poor. Therefore, the named entity recognition method provided in this embodiment can correct each answer of the same question based on the annotation manner of reading comprehension, so that each answer of the question is more in line with the expectation, the accuracy of each answer of the question is improved, and the accuracy of identifying the nested named entity is further improved.

[0144] In a possible implementation, determining the answer information of the question according to the first vector and the second vector can include: determining the answer information of the question according to the first vector, the second vector, and the relative position matrix.

[0145] The relative position matrix indicates the transition probability that a starting position and an ending position belong to the same answer. In the relative position matrix, an eigenvalue corresponds to a pair of starting and ending positions, representing the probability and likelihood that the corresponding starting and ending positions belong to the same answer, i.e., the transition probability. In one example, the row numbers of the relative position matrix correspond to the starting positions, and the column numbers correspond to the ending positions. In another example, the row numbers of the relative position matrix correspond to the ending positions, and the column numbers correspond to the starting positions.

[0146] Taking the first question and the first concatenated statement as an example, the process of determining the answer information of the question based on the first vector, the second vector, and the relative position matrix is ​​explained. Figure 6 An exemplary schematic diagram illustrating the relative position matrix of an embodiment of this application is shown. Figure 6 As shown, the relative position matrix has dimensions N*N, where N is a positive integer. N can be the same as the length of the first concatenated statement; for example, N can be 64 or 32, etc. This application does not impose any restrictions on this. Taking the relative position matrix where row numbers correspond to the starting position and column numbers correspond to the ending position as an example, refer to... Figure 6 The feature value 0 in row 0, column 0 represents that the probability of the 0th character and the 0th character in the first concatenated sentence forming an answer is 0; the feature value 1 in row 1, column 2 represents that the probability of the 1st character and the 2nd character in the first concatenated sentence forming an answer is 1; the feature value 0 in row 4, column 3 represents that the probability of the 4th character and the 3rd character in the first concatenated sentence forming an answer is 0.

[0147] In one possible implementation, the eigenvalues ​​in the relative position matrix can be given numerical values. For example, statistically, the average length of a person's name is 2.6 characters, and the average length of a place's name is 3.3 characters; very few city or place names exceed eight characters. Therefore, in the relative position matrix, when the difference between the row number and column number of an eigenvalue exceeds eight characters, it can be set to 0.001. Additionally, the starting position of the same answer will precede the ending position; therefore, in the relative position matrix, when the row number of an eigenvalue is less than the column number, it can be set to 0. In another possible implementation, the relative position matrix can also be obtained through model learning.

[0148] In implementation, the first vector can be used to determine a start position of an answer to the first question in the to-be-identified sentence, and a third state probability corresponding to each start position. The second vector can be used to determine a termination position of the answer to the first question in the to-be-identified sentence, and a fourth state probability corresponding to each termination position. The relative position matrix can be used to determine a transition probability corresponding to a combination of each start position and termination position. For each combination of start position and termination position, a probability that the combination is correct, i.e., a probability that the start position and termination position in the combination constitute a correct answer, can be determined according to the third state probability corresponding to the start position in the combination, the fourth state probability corresponding to the termination position in the combination, and the transition probability corresponding to the combination.

[0149] In a possible implementation, the third state probability corresponding to the first start position can be obtained from the first vector; the fourth state probability corresponding to the first termination position can be obtained from the second vector; the transition probability corresponding to the first start position and the first termination position can be obtained from the relative position matrix; a probability corresponding to a candidate answer can be determined based on the third state probability corresponding to the first start position, the fourth state probability corresponding to the first termination position, and the transition probability corresponding to the first start position and the first termination position; in a case where the probability of the candidate answer is greater than a probability threshold, the start position indicated by the first vector is determined as the start position indicated in the answer information, the third state probability indicated by the first vector is determined as the first state probability indicated in the answer information; the termination position indicated by the second vector is determined as the termination position indicated in the answer information, and the fourth state probability indicated by the second vector is determined as the second state probability indicated in the answer information.

[0150] In the formula, the first start position can represent any one of the start positions indicated by the first vector, the first termination position can represent any one of the termination positions indicated by the second vector, and the candidate answer can represent an answer with the start position being the first start position and the termination position being the first termination position.

[0151] In an example, determining a probability corresponding to the first answer based on the first state probability corresponding to the first start position, the second state probability corresponding to the first termination position, and the transition probability corresponding to the first start position and the first termination position can include multiplying the first state probability corresponding to the first start position, the second state probability corresponding to the first termination position, and the transition probability corresponding to the first start position and the first termination position to obtain the probability corresponding to the candidate answer.

[0152] When the probability of the candidate answer is greater than the probability threshold, it indicates that the accuracy of the candidate answer is relatively high. Therefore, the candidate answer can be determined as the first answer to the first question in the statement to be recognized. When the probability of the candidate answer is less than or equal to the probability threshold, it indicates that the accuracy of the candidate answer is relatively low. Therefore, the candidate answer will not be determined as the first answer to the first question in the statement to be recognized.

[0153] Figure 7 The process schematic diagram shows the process of obtaining the probability corresponding to the candidate answer in the embodiment of the present application. As Figure 7 shown, assume Figure 5 that the first spliced statement shown is input into a pre-trained entity extraction model (in Figure 5 it is the bert model), and two starting positions and two ending positions are obtained. The two starting positions are the 1st character and the 3rd character in the first spliced statement, namely "南" and "栖". The two ending positions are the 2nd character and the 5th character in the first spliced statement, namely "京" and "区". Among them, the third state probabilities of "南" and "栖" are P11 and P13 respectively, and the fourth state probabilities of "京" and "区" are P22 and P25 respectively. It can be seen that the possible combinations of starting positions and ending positions include: the combination of "南" and "京" (corresponding to the answer "南京"), the combination of "南" and "区" (corresponding to the answer "南京栖霞区"), the combination of "栖" and "京" (corresponding to the answer "栖京"), and the combination of "栖" and "区" (corresponding to the answer "栖霞区").

[0154] For the combination of "南" and "京", the third state probability corresponding to the starting position "南" is P11, and the fourth state probability corresponding to the ending position "京" is P22. The transfer probability corresponding to the combination is: the eigenvalue of the 0th row and 1st column of the relative position matrix (for example, Figure 6 shown as 0.8). Therefore, the probability of the answer "南京" is: P11 * P22 * the eigenvalue of the 0th row and 1st column of the relative position matrix. The combinations of "南" and "区", "栖" and "京", and "栖" and "区" can be referred to the combination of "南" and "京", which will not be elaborated here.

[0155] It can be understood that the combination of "栖" and "京" is unreasonable. In the embodiment of the present application, referring to Figure 6 (the eigenvalue of the 3rd row and 2nd column of the relative position matrix is 0), it can be obtained that the transfer probability value corresponding to the combination of "栖" and "京" is 0. Therefore, the probability of the answer "栖京" is 0, eliminating unreasonable candidate answers and improving the accuracy of the answer to the first question.

[0156] Thus, the answer information of each question is obtained. For example, the answer information of the first question, the answer information of the second question. In the embodiment of the present application, different answers can be obtained by concatenating different questions after the to-be-recognized sentence, and the answers can be nested, so that the recognition of nested named entities can be implemented; the answer of each question is more consistent with the expectation by correcting the answer of the same question, and the accuracy of each question answer itself is improved, and thus the accuracy of recognizing nested named entities is improved.

[0157] In step S403, the answer information of the first question is processed based on the answer information of at least one second question to obtain a target answer of the first question.

[0158] In the embodiment of the present application, based on the annotation mode of reading comprehension, after the answer of a question is recognized, the answer of the question can be processed according to the answers of other questions, so that when it is uncertain whether the answer of the question is accurate, the answers of other questions are used to assist in proving whether the answer is accurate, thereby improving the accuracy of the answer of the question and the overall accuracy of the to-be-recognized sentence.

[0159] In a possible implementation, step S403 can include: determining at least one second answer based on the answer information of the at least one second question; determining a first answer based on the answer information of the first question; in a case where the at least one second answer and the first answer satisfy a positive filtering condition, determining the first answer as the target answer of the first question; in a case where the at least one second answer and the first answer satisfy a negative filtering condition, verifying the first answer, and in a case where the verification is passed, determining the first answer as the target answer of the first question.

[0160] For each second question, in a case where the sum of the first state probability and the second state probability indicated by the answer information of the second question is greater than a certain threshold, or the product of the first state probability and the second state probability is greater than a certain threshold, or the first state probability is greater than a certain threshold and the second state probability is greater than a certain threshold, the starting position corresponding to the first state probability and the ending position corresponding to the second state probability can be determined as the starting position and the ending position of the second answer, so as to obtain the second answer. Similarly, the first answer can be obtained based on the answer information of the first question. For example, the first answer of the first question "city location" is "Nanjing", the second answer of the second question "county location" is "Qixia District", and the second answer of the second question "complete location" is "Nanjing Qixia District".

[0161] The forward filtering condition represents a condition under which the first answer can be directly the target answer. In one example, the forward filtering condition includes that the position of the first answer does not overlap with the position of the at least one second answer, or the positional relationship between the first answer and the at least one second answer matches the range size relationship between the first question and the at least one second question.

[0162] For example, in a case where the first answer and the second answer belong to different categories (e.g., the first answer is a person name and the at least one second answer is a place name, or the first answer is a place name and the at least one second answer is a person name), if the position of the first answer does not overlap with the position of each second answer, the first answer can be determined as the target answer of the first question. In one example, the to-be-recognized sentence is "Xiaoming is in New York, he wants to go to London, how is the weather in London", the first question is "person name", the second question is "place name", the determined first answer is "Xiaoming", and the second answers are "New York" and "London". Since the position of "Xiaoming" in the to-be-recognized sentence does not overlap with the positions of "New York" and "London" in the to-be-recognized sentence, it can be determined that "Xiaoming" is the target answer of the first question.

[0163] For example, in a case where the first answer and the second answer belong to the same category (e.g., the first answer is a province name, and the second answer is a city name, both of which belong to place names), if the positional relationship between the first answer and each second answer matches the range size relationship between the first question and the at least one second question, the first answer can be determined as the target answer of the first question. In one example, the to-be-recognized sentence is "navigate to Guangdong Province Guangzhou Baiyun Road", the first question is "province name", the second question is "city name", "street name", and "complete place", the determined first answer is "Guangdong Province", and the second answers are "Guangzhou", "Baiyun Road", and "Guangdong Province Guangzhou Baiyun Road". Since "Guangdong Province" in the to-be-recognized sentence is located before "Guangzhou" and "Baiyun Road", and the range corresponding to the "province name" is larger than the ranges corresponding to the "city name" and the "street name", and "Guangdong Province" in the to-be-recognized sentence is located within "Guangdong Province Guangzhou Baiyun Road", and the range corresponding to the "province name" is included in the range corresponding to the "complete place", it can be determined that "Guangdong Province" is the target answer of the first question.

[0164] The reverse filtering condition represents a condition under which the first answer cannot be directly the target answer. In one example, the reverse filtering condition includes that the position of the first answer overlaps with the position of the at least one second answer, or the positional relationship between the first answer and the at least one second answer does not match the range size relationship between the first question and the at least one second question.

[0165] In one example, the position relationship between the first answer and the at least one second answer matches the range size relationship between the first question and the at least one second question includes that the first answer and the at least one second answer are place names, the position of the first answer in the sentence to be recognized is before the at least one second answer, the range corresponding to the first question is greater than the range of the at least one second question, or the position of the first answer in the sentence to be recognized is after the at least one second answer, and the range corresponding to the first question is less than the range of the at least one second question.

[0166] In one example, in a case where the at least one second answer and the first answer satisfy the reverse filtering condition, the first answer is verified, and in a case where the verification is passed, the first answer is determined as the target answer of the first question, including that the first answer and the at least one second answer are multimedia resources, in a case where the positions of the at least one second answer and the first answer overlap, the first answer is subjected to a dictionary verification, and in a case where the dictionary verification is passed, the first answer is determined as the target answer of the first question. For example, the sentence to be recognized is "play seven miles of fragrance", the first question is "song", the second question is "album", the corresponding first answer is "seven miles of fragrance", the second answer is also "seven miles of fragrance", and the positions of the first answer and the second answer overlap, at this time, the first answer can be subjected to a dictionary verification to determine whether the song "seven miles of fragrance" really exists. In a case where the dictionary verification is passed, it is indicated that the song "seven miles of fragrance" really exists, and "seven miles of fragrance" can be determined as the target answer of the first question.

[0167] The above are only exemplary descriptions of the forward filtering condition and the reverse filtering condition, and the forward filtering condition and the reverse filtering condition can further include other conditions. The forward filtering condition and the reverse filtering condition can be set and adjusted as needed, and the embodiments of the present application are not limited thereto.

[0168] In a possible implementation, the step S403 can include: obtaining at least one second feature vector according to the answer information of the at least one second question, the second feature vector being used to indicate a start position and an end position of an answer to the second question in the to-be-identified sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position; obtaining a first feature vector according to the answer information of the first question, the first feature vector being used to indicate a start position and an end position of an answer to the first question in the to-be-identified sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position; performing convolution processing on the at least one second feature vector and the first feature vector to obtain a third feature vector; and determining the target answer of the first question according to the third feature vector.

[0169] In an example, the first vector and the second vector corresponding to the second question can be spliced or combined together to obtain a second feature vector, and the first vector and the second vector corresponding to the first question can be spliced or combined together to obtain a first feature vector.

[0170] In implementation, the convolution processing can be implemented by a non-full connection layer of a pre-trained CNN model, which will not be described herein. Determining the target answer of the first question according to the third feature vector includes inputting the third feature vector into an activation function to obtain a probability value, and in a case where the probability value is greater than a certain threshold, determining the first answer as the target answer of the first question.

[0171] In a possible implementation, the step S403 can include: obtaining at least one second feature vector according to the answer information of the at least one second question, the second feature vector being used to indicate a start position and an end position of an answer to the second question in the to-be-identified sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position; obtaining a first feature vector according to the answer information of the first question, the first feature vector being used to indicate a start position and an end position of an answer to the first question in the to-be-identified sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position; splicing the at least one second feature vector and the first feature vector to obtain a fourth feature vector; inputting the fourth feature vector into a pre-trained CRF model to obtain a fifth feature vector; and determining the target answer of the first question according to the fifth feature vector.

[0172] Determining the target answer of the first question according to the fifth feature vector can refer to determining the target answer of the first question according to the third feature vector, which will not be described herein.

[0173] Different problems represent different labels. For some unpopular labels, the recognition accuracy of the entity extraction model can be low, thereby causing the overall recognition effect of the to-be-recognized sentence to be unstable. For example, county locations are less common in the corpus used to train the entity extraction model, which causes the trained entity extraction model to be prone to errors when recognizing county locations. Considering different problems concatenated after the to-be-recognized sentence, the entity extraction model analyzes different focuses, thereby losing some structural information of the to-be-recognized sentence, for example, the front-back relationship between city locations and county locations. In the embodiments of the present application, the contribution information between labels can be learned, thereby recovering the lost structural information, and correcting the answers of unpopular labels according to the structural information, thereby improving the overall recognition accuracy of the to-be-recognized sentence. For example, city locations are prone to be followed by county locations, and therefore, the confidence of the structure of "city location + county location" is high, while the confidence of the structure of "county location + city location" is low. In this way, for unpopular county locations, when it is uncertain whether "Qixia District" is a county, the structure information is used to recognize that "Qixia District" is preceded by a city location "Nanjing", and at this time, it can be more confirmed that "Qixia District" is a county.

[0174] In step S404, the named entity in the to-be-recognized sentence is determined according to the target answers of the plurality of problems.

[0175] After obtaining the target answers of the problems, the target answers of the problems can be determined as the named entities in the to-be-recognized sentence. For example, the target answer "Nanjing" of the problem "city location", the target answer "Qixia District" of the problem "county location", and the target answer "Nanjing Qixia District" of the problem "complete location" are determined as the named entities in the to-be-recognized sentence. Here, "Nanjing Qixia District" is a nested entity, which is accurately recognized, and the simple entities "Nanjing" and "Qixia District" therein are also accurately recognized.

[0176] Figure 8 The structure of the named entity recognition system provided by the embodiments of the present application is shown. As shown in FIG. 1, the named entity recognition system includes a problem identification module 101, a target answer determination module 102, and a named entity recognition module 103. Figure 8As shown, the named entity recognition system includes a preprocessing module and an identification module. The preprocessing module is configured to perform splicing processing on the to-be-identified sentence to obtain a spliced sentence. The identification module is configured to perform labeling on the spliced sentence based on a reading comprehension labeling manner. The identification module includes an entity extraction model, an intra-question verification model, and an inter-question reasoning model. The entity extraction model is configured to identify a start position and an end position in the spliced sentence, and a third state probability corresponding to the start position and a fourth state probability corresponding to the end position. The intra-question verification model is configured to adjust the extraction result (the third state probability corresponding to the start position and the fourth state probability corresponding to the end position) of each spliced sentence output by the entity extraction model based on a relative position matrix to obtain an answer to each entity question and a probability (including a first state probability and a second state probability) of each answer to each entity question. The inter-question reasoning model is configured to process answer information of a first question based on answer information of at least one second question to obtain a target answer to the first question, the first question being any one of the plurality of questions, and the second question being any one of the plurality of questions other than the first question. In the embodiment of the present application, the entity question to which the to-be-identified sentence is spliced can be pre-set. The input of the preprocessing module is the to-be-identified sentence, and the output of the preprocessing module is a plurality of spliced sentences (for example, a first spliced sentence and at least one second spliced sentence). The input of the identification module is the plurality of spliced sentences, and the output of the identification module is a named entity in the to-be-identified sentence and a label (for example, a city location, a county location, and a complete location) of the named entity.

[0177] In one example, the entity extraction model can be a bert model, the intra-question verification model can be a CNN model, and the inter-question reasoning model can be a CRF model or a CNN model. The embodiments of the present application do not limit the models. In implementation, the entity extraction model, the intra-question verification model, and the inter-question reasoning model in the identification module can be trained in an end-to-end manner, that is, the labeled spliced sentence is input into the identification module, and the parameters of the entity extraction model, the intra-question verification model, and the inter-question reasoning model are adjusted based on the output of the identification module and the labeling.

[0178] Figure 9 A structure schematic diagram of the named entity recognition device provided by the embodiments of the present application is shown. As shown in FIG. 6, the device 60 can include: Figure 6

[0179] A splicing module 61, configured to splice the to-be-identified sentence with a plurality of questions respectively to obtain a plurality of spliced sentences;

[0180] ​The first determining module 62 is configured to determine, for any question in the plurality of questions, answer information of the question based on the spliced sentence corresponding to the question obtained by the splicing module 61, the answer information being used to indicate a start position and an end position of an answer to the question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position.

[0181] The processing module 63 is configured to process the answer information of the first question determined by the first determining module 62 based on the answer information of at least one second question determined by the first determining module 62, to obtain a target answer of the first question, the first question being any question in the plurality of questions, and the second question being any question in the plurality of questions except the first question.

[0182] The second determining module 64 is configured to determine a named entity in the to-be-recognized sentence according to the target answers of the plurality of questions obtained by the processing module 63.

[0183] In a possible implementation, the processing module is further configured to: determine at least one second answer based on the answer information of the at least one second question; determine a first answer based on the answer information of the first question; in a case where the at least one second answer and the first answer satisfy a positive filtering condition, determine the first answer as the target answer of the first question; or in a case where the at least one second answer and the first answer satisfy a negative filtering condition, verify the first answer, and in a case where the verification is passed, determine the first answer as the target answer of the first question.

[0184] In a possible implementation, the processing module is further configured to: obtain at least one second feature vector according to the answer information of the at least one second question, the second feature vector being used to indicate the start position and the end position of the answer to the second question in the to-be-recognized sentence, and the first state probability corresponding to the start position and the second state probability corresponding to the end position; obtain a first feature vector according to the answer information of the first question, the first feature vector being used to indicate the start position and the end position of the answer to the first question in the to-be-recognized sentence, and the first state probability corresponding to the start position and the second state probability corresponding to the end position; perform convolution processing on the at least one second feature vector and the first feature vector to obtain a third feature vector; and determine the target answer of the first question according to the third feature vector.

[0185] In a possible implementation, the processing module is further configured to: obtain at least one second feature vector according to the answer information of the at least one second question, the second feature vector being used to indicate a start position and an end position of an answer to the second question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position; obtain a first feature vector according to the answer information of the first question, the first feature vector being used to indicate a start position and an end position of an answer to the first question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position; splice the at least one second feature vector and the first feature vector to obtain a fourth feature vector; input the fourth feature vector into a pre-trained CRF model to obtain a fifth feature vector; and determine the target answer of the first question according to the fifth feature vector.

[0186] In a possible implementation, the first determining module is further configured to: input the spliced sentence corresponding to the question into a pre-trained entity extraction model, and output a first vector and a second vector, the first vector being used to indicate a start position of an answer to the question in the to-be-recognized sentence and a third state probability corresponding to the start position, and the second vector being used to indicate an end position of the answer to the question in the to-be-recognized sentence and a fourth state probability corresponding to the end position; and determine the answer information of the question according to the first vector and the second vector.

[0187] In a possible implementation, the determining the answer information of the question according to the first vector and the second vector includes: determining the answer information of the question according to the first vector, the second vector, and a relative position matrix, the relative position matrix being used to indicate a transition probability of a start position and an end position belonging to the same answer.

[0188] In a possible implementation, the forward filtering condition includes that a position of the first answer and a position of the at least one second answer do not overlap, or a position relationship between the first answer and the at least one second answer matches a range size relationship between the first question and the at least one second question; and the reverse filtering condition includes that the position of the first answer and the position of the at least one second answer overlap, or the position relationship between the first answer and the at least one second answer does not match the range size relationship between the first question and the at least one second question.

[0189] In a possible implementation, the position relationship between the first answer and the at least one second answer matches the range size relationship between the first question and the at least one second question, including that the first answer and the at least one second answer are geographical names, the position of the first answer in the to-be-recognized sentence is before the at least one second answer, and the range corresponding to the first question is greater than the range of the at least one second question, or the position of the first answer in the to-be-recognized sentence is after the at least one second answer, and the range corresponding to the first question is less than the range of the at least one second question.

[0190] In a possible implementation, in the case where the at least one second answer and the first answer satisfy the reverse filtering condition, the first answer is verified, and in the case where the verification is passed, the first answer is determined as the target answer of the first question, including that the first answer and the at least one second answer are multimedia resources, in the case where the positions of the at least one second answer and the first answer overlap, the majority of the first answers are subjected to dictionary verification, and in the case where the dictionary verification is passed, the first answer is determined as the target answer of the first question.

[0191] In a possible implementation, the plurality of questions belong to a plurality of preset categories, and / or different questions of the same preset category correspond to different granularities.

[0192] In the embodiments of the present application, by splicing a plurality of questions and obtaining a plurality of answers, the recognition of nested entities can be implemented; for the answer of any question, the answer of the question is processed according to the answers of other questions, so that when it is not determined whether the answer of a question is accurate, the answers of other questions can be used to assist in proving whether the answer is accurate, thereby improving the accuracy of the answer of the question and the accuracy of recognizing the nested entity.

[0193] Figure 10 A structure schematic diagram of a named entity recognition device provided by an embodiment of the present application is shown. As shown in the figure, the device 70 includes: Figure 10

[0194] The preprocessing module 71 is configured to splice the to-be-recognized sentence and a plurality of questions respectively to obtain a plurality of spliced sentences.

[0195] The entity extraction module 72 is configured to, for any question in the plurality of questions, determine answer information of the question based on the spliced sentence corresponding to the question obtained by the preprocessing module 71, the answer information being used to indicate a start position and an end position of the answer of the question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position. ​

[0196] The inter-question reasoning module 73 is configured to process answer information of a first question obtained by the entity extraction module 72 based on answer information of at least one second question obtained by the entity extraction module 72, to obtain a target answer of the first question, the first question being any one of the plurality of questions, and the second question being any one of the plurality of questions except the first question.

[0197] The entity determination module 74 is configured to determine a named entity in the to-be-recognized sentence according to the target answers of the plurality of questions obtained by the inter-question reasoning module 73.

[0198] In a possible implementation, the inter-question reasoning module is further configured to:

[0199] determine at least one second answer based on the answer information of the at least one second question;

[0200] determine a first answer based on the answer information of the first question;

[0201] in a case where the at least one second answer and the first answer satisfy a positive filtering condition, determine the first answer as the target answer of the first question;

[0202] or,

[0203] in a case where the at least one second answer and the first answer satisfy a negative filtering condition, verify the first answer, and in a case where the verification is passed, determine the first answer as the target answer of the first question.

[0204] In a possible implementation, the inter-question reasoning module is further configured to:

[0205] obtain at least one second feature vector according to the answer information of the at least one second question, the second feature vector being used to indicate a start position and an end position of an answer of the second question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position;

[0206] obtain a first feature vector according to the answer information of the first question, the first feature vector being used to indicate a start position and an end position of an answer of the first question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position;

[0207] perform convolution processing on the at least one second feature vector and the first feature vector to obtain a third feature vector;

[0208] According to the third feature vector, a target answer of the first question is determined.

[0209] In a possible implementation, the inter-question reasoning module is further configured to:

[0210] According to the answer information of the at least one second question, at least one second feature vector is obtained, the second feature vector being used to indicate a start position and an end position of an answer of the second question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position;

[0211] According to the answer information of the first question, a first feature vector is obtained, the first feature vector being used to indicate a start position and an end position of an answer of the first question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position;

[0212] The at least one second feature vector and the first feature vector are spliced to obtain a fourth feature vector;

[0213] The fourth feature vector is input into a pre-trained CRF model to obtain a fifth feature vector;

[0214] According to the fifth feature vector, a target answer of the first question is determined.

[0215] In a possible implementation, the entity extraction module is further configured to:

[0216] The spliced sentence corresponding to the question is input into a pre-trained entity extraction model, and a first vector and a second vector are output, the first vector being used to indicate a start position of an answer of the question in the to-be-recognized sentence and a third state probability corresponding to the start position, and the second vector being used to indicate an end position of the answer of the question in the to-be-recognized sentence and a fourth state probability corresponding to the end position;

[0217] The apparatus further includes:

[0218] An intra-question reasoning module configured to determine answer information of the question according to the first vector and the second vector obtained by the entity extraction module;

[0219] The inter-question reasoning module is further configured to process the answer information of the first question obtained by the intra-question reasoning module based on the answer information of the at least one second question obtained by the intra-question reasoning module, to obtain a target answer of the first question, the first question being any question in the plurality of questions, and the second question being any question in the plurality of questions except the first question.

[0220] In a possible implementation, the question internal reasoning module is further configured to:

[0221] According to the first vector, the second vector, and a relative position matrix, determine answer information of the question, where the relative position matrix is used to indicate a transition probability of a start position and an end position belonging to a same answer.

[0222] In a possible implementation, the forward filtering condition comprises: a position of the first answer and a position of the at least one second answer do not overlap, or a position relationship between the first answer and the at least one second answer matches a range size relationship between the first question and the at least one second question.

[0223] The reverse filtering condition comprises: the position of the first answer and the position of the at least one second answer overlap, or the position relationship between the first answer and the at least one second answer does not match the range size relationship between the first question and the at least one second question.

[0224] In a possible implementation, the position relationship between the first answer and the at least one second answer matching the range size relationship between the first question and the at least one second question comprises:

[0225] The first answer and the at least one second answer are place names, a position of the first answer in the to-be-recognized sentence is before a position of the at least one second answer, a range corresponding to the first question is greater than a range of the at least one second question, or the position of the first answer in the to-be-recognized sentence is after the position of the at least one second answer, and the range corresponding to the first question is smaller than the range of the at least one second question.

[0226] In a possible implementation, in a case where the at least one second answer and the first answer satisfy the reverse filtering condition, the first answer is verified, and in a case where the verification is passed, the first answer is determined as a target answer of the first question, comprising:

[0227] The first answer and the at least one second answer are multimedia resources, in a case where the position of the at least one second answer and the position of the first answer overlap, a majority of the first answers are subjected to a dictionary verification, and in a case where the dictionary verification is passed, the first answer is determined as the target answer of the first question.

[0228] In a possible implementation, the plurality of questions belong to a plurality of preset categories, and / or different questions corresponding to a same preset category correspond to different granularities.

[0229] The embodiment of the present application provides a named entity recognition device, including a processor and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions.

[0230] The embodiment of the present application provides a named entity recognition device, for example Figure 9 The named entity recognition device shown in the embodiment of the present application, Figure 10 The named entity recognition device shown in the embodiment of the present application can be a terminal device, a processor, a chip (for example, an AI chip) or a model (for example, an AI model).

[0231] The embodiment of the present application provides an electronic device, including a processor and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions.

[0232] The embodiment of the present application provides a nonvolatile computer readable storage medium, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the above method.

[0233] The embodiment of the present application provides a computer program product, including computer readable code or a nonvolatile computer readable storage medium carrying computer readable code, when the computer readable code is run in the processor of an electronic device, the processor in the electronic device executes the above method.

[0234] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any appropriate combination of the above. More specific examples (a non-exhaustive list) of the computer readable storage medium include a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an electrically programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital video disc (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a hole in a raised structure, and any appropriate combination of the above.

[0235] Computer readable program instructions or code for carry out the operations described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adaptation card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing / processing device.

[0236] Computer readable program instructions for carrying out operations of the present application can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or any combination of source code or object code in any combination of one or more programming languages including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on a user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider. In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0237] Various aspects of the present application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0238] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer readable storage medium having no data, programs, program modules, and / or computer readable program instructions presently, and / or not yet, stored thereon. The instructions can be stored in a computer readable storage medium that can be implemented in any method or technology for storage of information, including magnetic storage media, optical storage media, solid-state storage media, and others.

[0239] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable data processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0240] The flow diagrams and block diagrams in the accompanying drawings show archi tectures, functional and operational architectures of possible implementations of apparatuses, systems, methods and computer program products according to the present application. In this regard, each block in the flow diagrams and block diagrams can represent a module, a segment, or a portion of instructions, which comprises one or more executable instructions for implementing the specified logical functions. In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in some cases, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. These and other alternatives are possible.

[0241] It also is noted that each of the blocks of the flowchart, and / or combinations of blocks, can be implemented by hardware, by software including one or more computer program instructions, by firmware, or by a combination of hardware, software, and / or firmware. Also, the flowcharts and / or block diagrams can be implemented in hardware, in software, or in a combination of hardware and software.

[0242] Although the application has been described in connection with various embodiments, it will be understood that the application is capable of further modifications. These and other changes, along with the apparent alternatives and equivalents, fall within the scope of the claimed application. The description herein is intended to be illustrative only and is presented to enable any person skilled in the art to make and use the application. Numerous modifications and adaptations will be apparent to those skilled in the art without departing from the scope of the described application. The scope of the described application is not to be limited by the specific illustrative embodiments contained herein but only by the scope of the appended claims, which follow this disclosure.

[0243] Various embodiments of the application have been described in connection with the embodiments described above. The description is intended to be illustrative only and not exhaustive of all possible embodiments. Numerous modifications and adaptations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The scope of the described embodiments is not to be limited by the specific illustrative embodiments contained herein but only by the scope of the appended claims, which follow this disclosure.

Claims

1. A method of named entity recognition, characterized by, The method comprises: Splicing the to-be-recognized sentence with a plurality of questions respectively to obtain a plurality of spliced sentences; For any question in the plurality of questions, determining answer information of the question based on the spliced sentence corresponding to the question, the answer information being used to indicate a start position and an end position of an answer of the question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position; Processing the answer information of a first question based on answer information of at least one second question to obtain a target answer of the first question, the first question being any question in the plurality of questions, and the second question being any question in the plurality of questions except the first question; Determining a named entity in the to-be-recognized sentence according to the target answers of the plurality of questions, the questions being pre-set according to a to-be-recognized named entity, a category of the question being associated with a category of the to-be-recognized named entity, and the first question and the second question being one word, one statement, one question or one label.

2. The method of claim 1, wherein, The processing of the answer information of the first question based on the answer information of the at least one second question to obtain the target answer of the first question comprises: Determining at least one second answer based on the answer information of the at least one second question; Determining a first answer based on the answer information of the first question; In a case where the at least one second answer and the first answer satisfy a forward filtering condition, determining the first answer as the target answer of the first question; Or, In a case where the at least one second answer and the first answer satisfy a reverse filtering condition, verifying the first answer, and in a case where the verification is passed, determining the first answer as the target answer of the first question.

3. The method of claim 1, wherein, The processing of the answer information of the first question based on the answer information of the at least one second question to obtain the target answer of the first question comprises: Obtaining at least one second feature vector according to the answer information of the at least one second question, the second feature vector being used to indicate the start position and the end position of the answer of the second question in the to-be-recognized sentence, and the first state probability corresponding to the start position and the second state probability corresponding to the end position; Obtaining a first feature vector according to the answer information of the first question, the first feature vector being used to indicate the start position and the end position of the answer of the first question in the to-be-recognized sentence, and the first state probability corresponding to the start position and the second state probability corresponding to the end position; Performing convolution processing on the at least one second feature vector and the first feature vector to obtain a third feature vector; Determining the target answer of the first question according to the third feature vector.

4. The method of claim 1, wherein, The processing of the answer information of the first question based on the answer information of the at least one second question to obtain the target answer of the first question comprises: According to the answer information of the at least one second question, at least one second feature vector is obtained, the second feature vector being used to indicate a start position and an end position of the answer to the second question in the to-be-identified sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position; According to the answer information of the first question, a first feature vector is obtained, the first feature vector being used to indicate a start position and an end position of the answer to the first question in the to-be-identified sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position; The at least one second feature vector and the first feature vector are spliced to obtain a fourth feature vector; The fourth feature vector is input into a pre-trained CRF model to obtain a fifth feature vector; According to the fifth feature vector, the target answer of the first question is determined.

5. The method of claim 1, wherein, The answer information of the question is determined based on the spliced sentence corresponding to the question, and the method comprises: The spliced sentence corresponding to the question is input into a pre-trained entity extraction model to output a first vector and a second vector, the first vector being used to indicate a start position of the answer to the question in the to-be-identified sentence and a third state probability corresponding to the start position, and the second vector being used to indicate an end position of the answer to the question in the to-be-identified sentence and a fourth state probability corresponding to the end position; The answer information of the question is determined according to the first vector and the second vector.

6. The method of claim 5, wherein, The answer information of the question is determined according to the first vector, the second vector, and a relative position matrix, wherein the relative position matrix is used to indicate a transition probability of a start position and an end position belonging to the same answer.

7. The method of claim 2, wherein: the forward filtering condition comprises that the position of the first answer and the position of the at least one second answer do not overlap, or the position relationship between the first answer and the at least one second answer matches the range size relationship between the first question and the at least one second question; the reverse filtering condition comprises that the position of the first answer and the position of the at least one second answer overlap, or the position relationship between the first answer and the at least one second answer does not match the range size relationship between the first question and the at least one second question. the position relationship between the first answer and the at least one second answer matches the range size relationship between the first question and the at least one second question comprises:

8. The method of claim 7, wherein, the first answer and the at least one second answer are place names, the position of the first answer in the to-be-identified sentence is before the position of the at least one second answer, the range corresponding to the first question is greater than the range corresponding to the at least one second question, or the position of the first answer in the to-be-identified sentence is after the position of the at least one second answer, and the range corresponding to the first question is smaller than the range corresponding to the at least one second question. ​ 9. The method of claim 7, wherein, In a case where the at least one second answer and the first answer satisfy a reverse filtering condition, the first answer is verified, and in a case where the verification is passed, the first answer is determined as a target answer of the first question, comprising: In a case where the at least one second answer and the first answer satisfy a reverse filtering condition, the first answer is verified, and in a case where the verification is passed, the first answer is determined as a target answer of the first question, comprising:

10. The method according to any one of claims 1 to 9, characterized in that, The plurality of questions belong to a plurality of preset categories, and / or different questions of a same preset category correspond to different granularities.

11. A named entity recognition apparatus, characterized by, The device comprises: A preprocessing module configured to splice the to-be-recognized sentence with a plurality of questions respectively to obtain a plurality of spliced sentences; An entity extraction module configured to, for any question in the plurality of questions, determine answer information of the question based on the spliced sentence corresponding to the question obtained by the preprocessing module, the answer information being used to indicate a start position and an end position of an answer of the question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position; A question-to-question reasoning module configured to, based on answer information of at least one second question obtained by the entity extraction module, process answer information of a first question obtained by the entity extraction module to obtain a target answer of the first question, the first question being any question in the plurality of questions, and the second question being any question in the plurality of questions except the first question; An entity determination module configured to determine a named entity in the to-be-recognized sentence according to target answers of a plurality of questions obtained by the question-to-question reasoning module, the questions being pre-set according to a to-be-recognized named entity, a category of the questions being associated with a category of the to-be-recognized named entity, and the first question and the second question being one word, one declarative sentence, one question, or one label.

12. The apparatus of claim 11, wherein, The question-to-question reasoning module is further configured to: determine at least one second answer based on the answer information of the at least one second question; determine a first answer based on the answer information of the first question; In a case where the at least one second answer and the first answer satisfy a reverse filtering condition, the first answer is verified, and in a case where the verification is passed, the first answer is determined as a target answer of the first question, comprising: Or, In a case where the at least one second answer and the first answer satisfy a reverse filtering condition, the first answer is verified, and in a case where the verification is passed, the first answer is determined as a target answer of the first question.

13. The apparatus of claim 11, wherein, The question-to-question reasoning module is further configured to: obtain at least one second feature vector according to the answer information of the at least one second question, the second feature vector being used to indicate a start position and an end position of an answer of the second question in the to-be-recognized sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position; obtain a first feature vector according to the answer information of the first question, the first feature vector being used to indicate a start position and an end position of the answer to the first question in the to-be-identified sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position; perform convolution processing on the at least one second feature vector and the first feature vector to obtain a third feature vector; determine the target answer to the first question according to the third feature vector.

14. The apparatus of claim 11, wherein, The inter-question reasoning module is further configured to: obtain at least one second feature vector according to the answer information of the at least one second question, the second feature vector being used to indicate a start position and an end position of the answer to the second question in the to-be-identified sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position; obtain a first feature vector according to the answer information of the first question, the first feature vector being used to indicate a start position and an end position of the answer to the first question in the to-be-identified sentence, and a first state probability corresponding to the start position and a second state probability corresponding to the end position; splice the at least one second feature vector and the first feature vector to obtain a fourth feature vector; input the fourth feature vector into a pre-trained CRF model to obtain a fifth feature vector; determine the target answer to the first question according to the fifth feature vector.

15. The apparatus of claim 11, wherein, The entity extraction module is further configured to: input the spliced sentence corresponding to the question into a pre-trained entity extraction model, and output a first vector and a second vector, the first vector being used to indicate a start position of the answer to the question in the to-be-identified sentence and a third state probability corresponding to the start position, and the second vector being used to indicate an end position of the answer to the question in the to-be-identified sentence and a fourth state probability corresponding to the end position; The apparatus further includes: an intra-question reasoning module configured to determine answer information of the question according to the first vector and the second vector; The inter-question reasoning module is further configured to process the answer information of the first question obtained by the intra-question reasoning module based on the answer information of the at least one second question obtained by the intra-question reasoning module, to obtain a target answer to the first question, the first question being any question in the plurality of questions, and the second question being any question in the plurality of questions except the first question.

16. The apparatus of claim 15, wherein, The intra-question reasoning module is further configured to: determine the answer information of the question according to the first vector, the second vector, and a relative position matrix, wherein the relative position matrix is used to indicate a transition probability of a start position and an end position belonging to a same answer.

17. The apparatus of claim 12, wherein the forward filtering condition includes that the position of the first answer and the position of the at least one second answer do not overlap, or a position relationship between the first answer and the at least one second answer matches a range size relationship between the first question and the at least one second question. The reverse filtering condition comprises that the position of the first answer overlaps with the position of the at least one second answer, or the position relationship between the first answer and the at least one second answer does not match the range size relationship between the first question and the at least one second question.

18. The apparatus of claim 17, wherein, The position relationship between the first answer and the at least one second answer matches the range size relationship between the first question and the at least one second question comprises that: The first answer and the at least one second answer are place names, the position of the first answer in the to-be-recognized sentence is before the at least one second answer, the range corresponding to the first question is greater than the range of the at least one second question, or the position of the first answer in the to-be-recognized sentence is after the at least one second answer, and the range corresponding to the first question is smaller than the range of the at least one second question.

19. The apparatus of claim 17, wherein, In a case where the at least one second answer and the first answer satisfy the reverse filtering condition, the first answer is verified, and in a case where the verification is passed, the first answer is determined as the target answer of the first question, comprising: The first answer and the at least one second answer are multimedia resources, in a case where the position of the at least one second answer overlaps with the position of the first answer, the first answer is subjected to dictionary verification, and in a case where the dictionary verification is passed, the first answer is determined as the target answer of the first question.

20. The apparatus of any one of claims 11-19, wherein, The plurality of questions belong to a plurality of preset categories, and / or different questions corresponding to different granularities in a same preset category.

21. An apparatus for named entity recognition, the apparatus comprising: Comprise: A processor; A memory for storing processor-executable instructions; Wherein the processor is configured to implement the method of any one of claims 1 to 10 when executing the instructions.

22. A non-transitory computer readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions are executed by the processor to implement the method of any one of claims 1 to 10.

23. A computer program product comprising computer readable code, or a non-transitory computer readable storage medium carrying computer readable code, which when run in an electronic device, a processor in the electronic device implements the method of any one of claims 1 to 10.

Images