Named entity recognition method, electronic device, storage medium and product

Abstract

The present invention relates to the field of computer technology, and provides a method, device, electronic device, storage medium and product for named entity recognition, the method comprising: obtaining a target text to be recognized; based on a named entity recognition model, recognizing the target text to obtain the emission probability of the target text and the abstract transition probability matrix of the named entity recognition model, recognizing the emission probability and the abstract transition probability matrix to obtain the named entity recognition result of the target text. The present invention utilizes a small amount of entity recognition annotation data in a target field to perform entity extraction in the target field, and only needs to annotate a small amount or even a single piece of data to obtain a good entity extraction effect, effectively solving the problem of poor entity extraction effect caused by insufficient data annotation.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a named entity recognition method, electronic equipment, storage medium and product. Background Art

[0002] Named entity recognition, that is, extracting corresponding text from a specified text, is a basic task in natural language processing. It is widely used in various sub-fields of natural language processing, such as information extraction, word slot filling in conversations, etc., and has important application value.

[0003] In traditional named entity recognition tasks, high-quality entity extraction is accomplished through a large amount of labeled data and the use of machine learning and other methods.

[0004] However, in actual industrial applications, we often face problems such as diverse scenarios, complex labeling systems, and lack of labeled data, which makes the efficiency and accuracy of entity extraction low. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems existing in the related art. To this end, the present invention proposes a method for named entity recognition, which uses a small amount of annotated data to achieve cross-domain and cross-label named entity recognition.

[0006] The invention also provides an electronic device.

[0007] The present invention also provides a non-transitory computer-readable storage medium.

[0008] The present invention also provides a computer program product.

[0009] According to a first aspect of the present invention, a method for named entity recognition includes:

[0010] Obtain the target text to be recognized;

[0011] Based on a named entity recognition model, the target text is recognized to obtain an emission probability of the target text and an abstract transition probability matrix of the named entity recognition model, and the emission probability and the abstract transition probability matrix are recognized to obtain a named entity recognition result of the target text;

[0012] Among them, the named entity recognition model is first pre-trained and then obtained based on support set samples and corresponding support set sample labels, the emission probability is determined based on the word embedding vector of the target text and the word embedding vector of the support set sample label, and the abstract transition probability matrix is ​​determined based on the transition probability matrix of the pre-trained named entity recognition model and the support set sample label.

[0013] According to the named entity recognition method of the embodiment of the present invention, the entities in the target document are recognized by constructing a named entity recognition model. Since the constructed named entity model is obtained through pre-training and a support set containing a small number of labeled samples, and the constructed named entity model improves the model structure, and the ability of the pre-trained language model can be effectively utilized, the obtained named entity recognition model can perform entity extraction work in the target field across the categories of the general field during pre-training. Only a small amount or even a single piece of data needs to be labeled to obtain a good entity extraction effect, which effectively solves the problem of poor entity extraction effect caused by insufficient data labeling.

[0014] According to some embodiments of the present invention, the named entity recognition model includes a vector layer, a label layer and a CRF layer, the vector layer is used to obtain the word embedding vector of the target text and the word embedding vector of the support set sample label, the label layer is used to obtain the emission probability of the target text based on the distance between the word embedding vector of the target text and the word embedding vector of the support set sample label, and the CRF layer is used to obtain the named entity recognition result of the target text based on the emission probability and the abstract transition probability matrix.

[0015] According to the named entity recognition method of an embodiment of the present invention, the named entity recognition model adopts the technical architecture of vector layer + label layer + CRF layer, which solves the problem that the CRF layer cannot be directly used across labels, and better utilizes the CRF layer to improve the overall effect of entity recognition.

[0016] According to some embodiments of the present invention, obtaining the emission probability of the target text according to the distance between the word embedding vector of the target text and the word embedding vector of the support set sample label includes:

[0017] Calculating the minimum distance between the word embedding vector of the target text and the word embedding vector of the support set sample label;

[0018] The emission probability of the target text is obtained based on the minimum distance.

[0019] According to some embodiments of the present invention, the CRF layer includes extraction of named entities of multiple categories, and the method further includes:

[0020] Based on the CRF layer, the named entities of each category are extracted and recognized respectively to obtain the named entity recognition results of each category.

[0021] According to some embodiments of the present invention, obtaining a named entity recognition result of the target text according to the emission probability and the abstract transition probability matrix includes:

[0022] The named entity recognition result of the target text is obtained by decoding the emission probability and the abstract transition probability matrix through the Viterbi algorithm.

[0023] According to some embodiments of the present invention, the pre-training of the named entity recognition model includes: pre-training the named entity recognition model based on training set samples and corresponding training set sample labels;

[0024] The training set samples include sample data in a general domain, and the support set samples include sample data in a target domain.

[0025] According to the named entity recognition method of the embodiment of the present invention, since the training set sample labels and the support set sample labels belong to different fields, the present invention can achieve cross-field recognition.

[0026] According to some embodiments of the present invention, the abstract transition probability matrix is ​​determined based on the transition probability matrix of the pre-trained named entity recognition model and the support set sample labels, including:

[0027] The abstract transition probability matrix is ​​determined by reconstructing and averaging the rows and columns of the transition probability matrix of the pre-trained named entity recognition model based on the types of the support set sample labels.

[0028] According to an embodiment of the second aspect of the present invention, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, it implements any one of the above-mentioned named entity recognition methods.

[0029] According to the non-transitory computer-readable storage medium of the third aspect of the present invention, a computer program is stored thereon, and when the computer program is executed by a processor, the named entity recognition method as described in any one of the above is implemented.

[0030] A computer program product according to an embodiment of the fourth aspect of the present invention includes a computer program, and when the computer program is executed by a processor, it implements any of the above-mentioned named entity recognition methods.

[0031] The above one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: a named entity recognition model is obtained by training with a small amount of entity recognition and annotation data in the target field, and the obtained named entity recognition model can perform entity extraction in the target field. The user only needs to annotate a small amount or even one piece of data to obtain a good entity extraction effect, which effectively solves the problem of poor entity extraction effect caused by insufficient data annotation.

[0032] Furthermore, the constructed named entity recognition model proposed a technical architecture of vector layer + label layer + CRF layer, which solved the problem that the CRF layer could not be used directly across labels, and better utilized the CRF layer to improve the overall effect of entity recognition.

[0033] Furthermore, the model training process does not adopt the traditional meta-learning training method. Instead, it uses a general domain training set for pre-training and then uses a support set for pseudo-training, thus avoiding the cumbersome training process of the meta-learning method.

[0034] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 It is a flowchart of a method for named entity recognition provided by an embodiment of the present invention;

[0037] Figure 2 Schematic diagram of a framework of a named entity recognition model provided by an embodiment of the present invention;

[0038] Figure 3 is a schematic diagram of a process for obtaining an abstract transition probability matrix provided by an embodiment of the present invention;

[0039] Figure 4 is a schematic diagram of a flow chart of obtaining emission probability provided by an embodiment of the present invention;

[0040] Figure 5 is a schematic diagram of the structure of a named entity recognition device provided by an embodiment of the present invention;

[0041] Figure 6 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] The following is a further detailed description of the implementation of the present application in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application but cannot be used to limit the scope of the present application.

[0044] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0045] Combine the following Figure 1-Figure 4 A named entity recognition method according to an embodiment of the present invention is described.

[0046] The executor of the named entity recognition method can be a named entity recognition device, or a server, or a user's terminal, including but not limited to mobile phones, tablet computers, PCs, vehicle-mounted terminals, and household smart appliances.

[0047] In some embodiments, the named entity recognition method includes:

[0048] Step 101: Obtain target text to be recognized.

[0049] It should be noted that the target text is usually a sentence, and the sentence is composed of multiple words and characters. In some embodiments, the target text includes named entity information to be identified. For example, the target text can be "Does the rice cooker have a heating function?", wherein "rice cooker" and "heating function" can be understood as named entity information to be identified.

[0050] Step 102: Based on the named entity recognition model, the target text is recognized to obtain the emission probability of the target text and the abstract transition probability matrix of the named entity recognition model.

[0051] Step 103: Identify the emission probability and the abstract transition probability matrix to obtain the named entity recognition result of the target text.

[0052] It should be noted that the named entity recognition result of the target text can be understood as the named entities in the recognized target text. For example, if the target text is "Does the rice cooker have a heating function?", the named entity recognition result of the target text can be understood as the recognized "rice cooker" and "heating function".

[0053] In some embodiments, the support set sample includes a pre-labeled support set sample label, and the named entity recognition result obtained by identifying the target text can belong to the pre-labeled support set sample label. For example, assuming that the support set sample includes "the refrigerator has a defrost function", the named entities therein are identified to obtain "refrigerator" and "defrost function", and the extracted named entities are labeled to obtain support set sample labels. For example, the named entity "refrigerator" is labeled as the "electrical appliance" label, and the named entity "defrost function" is labeled as the "function" label. In a specific implementation, the entire sentence of the target text (for example, whether the rice cooker has a heating function) is input into the trained named entity recognition model, and the extracted "rice cooker" can correspond to the "electrical appliance" label in the support set sample label, and the extracted "heating function" can correspond to the "function" label in the support set sample label.

[0054] Among them, the named entity recognition model is first pre-trained and then obtained based on the support set samples and the corresponding support set sample labels.

[0055] It should be noted that the training set samples include data in the general field, the support set samples are data containing entities in the target field, and the support set samples may specifically include one or more texts in the target field, and the support set sample labels are used to mark the types of entities in the target field. For example, when the target field corresponds to the electrical appliance field, it is assumed that a text included in the support set sample is "the refrigerator has a defrosting function", where the target field entity may include "refrigerator", and the label corresponding to the target field entity "refrigerator" may be "electrical appliance", and "electrical appliance" may be understood as the type to which "refrigerator" belongs.

[0056] In some embodiments, in the pre-training stage, the model is trained using data from the general domain, which may not include data from the target domain, to obtain a trained base model, and then pseudo-training is continued using support set data including data from the target domain.

[0057] Since the support set data includes the data of the target domain and sample labels, the pre-trained base model can be further constructed into a flat NER layer corresponding to each support set label, i.e., a label layer, after training on the support set.

[0058] In the above embodiment, a base model is obtained by performing model training based on a large number of training sample sets in general fields, and a label layer corresponding to the label of the target field is obtained by continuing pseudo training with a small amount of data in the target field, so that the model can accurately identify entities in the target field.

[0059] In the above embodiment, although the model pre-training uses data from other general fields, the label layer can still accurately identify entities in the target field. The overall training method is simpler than the traditional meta-learning method and can also solve cross-domain problems.

[0060] In some embodiments, the target field in this embodiment is the field of home appliances, and specific tasks that can be achieved include knowledge graph platform construction, recipe information extraction, word slot filling of the dialogue system, and the like.

[0061] In some embodiments, the pre-training of the named entity recognition model includes: pre-training the named entity recognition model based on training set samples and corresponding training set sample labels, wherein the training set samples include sample data in a general field, the support set samples include sample data in a target field, and the training set sample labels and the support set sample labels belong to different fields.

[0062] It should be noted that the training set can use the Chinese Fine-Grained Named Entity Recognition Dataset (CLUENER), which includes named entity recognition data covering ten label categories including addresses and book titles. The CLUENER Chinese dataset is a dataset that annotates entity words in general fields, including ten general entity words such as names of people, places, organization names, and book titles. In actual landing scene applications, data from several other fields can also be added to achieve better results.

[0063] In the above embodiment, the target domain to which the support set sample labels belong is the electrical appliance domain, and the training set sample labels belong to the general domain and do not include electrical appliances.

[0064] In some embodiments, the named entity recognition model is used to obtain a named entity recognition result of the target text based on the emission probability of the target text and an abstract transition probability matrix of the named entity recognition model.

[0065] Among them, the emission probability is determined based on the word embedding vector of the target text and the word embedding vector of the support set sample label, and the abstract transition probability matrix is ​​determined based on the transition probability matrix of the pre-trained named entity recognition model and the support set sample label.

[0066] In the above embodiment, the emission probability is determined based on the word embedding vector of the target text and the word embedding vector of the support set sample label. Specifically, it can be that the distance between any word embedding vector in the target text and each support set sample label word embedding vector is calculated, and the emission probabilities of multiple candidates are determined based on the calculated distance, and the support set sample label closest to the word embedding vector of the target text is selected, and the candidate emission probability corresponding to the word embedding vector and the nearest neighbor support set sample label is obtained as the final emission probability of the word embedding vector. For example, the target text is "Does the rice cooker have a heating function?", and the support set sample label includes "electrical appliances", and the "electrical appliances" label contains two word embedding vectors of "ice" and "box". Randomly select a word embedding vector in the target text, such as "electricity", then for "electricity", it is necessary to calculate its distance from all word embedding vectors under "electricity" respectively, and finally obtain the emission probability of the word embedding vector "electricity" and "electricity" based on the calculated distance and input it into the CRF layer.

[0067] It should be noted that each word embedding vector of the target text needs to calculate its emission probability and then be input into the CRF layer.

[0068] In the above embodiment, the abstract transition probability matrix is ​​determined based on the transition probability matrix of the pre-trained named entity recognition model and the support set sample labels. Specifically, the transition probability matrix, i.e., the transition probability, of the pre-trained named entity recognition model is obtained, and the categories of the transition probability matrix are added or deleted according to the types of the support set labels to obtain the abstract transition probability matrix. Among them, the categories included in the abstract transition probability matrix are consistent with the support set sample labels, and the initial values ​​in the abstract transition probability matrix are obtained by averaging the transition probability matrix. For example, there is only the "electrical appliance" entity category in the support set, so the transition probability matrix between "electrical appliances" and "non-electrical appliances" can be constructed.

[0069] In some embodiments, the named entity recognition model includes a vector layer, a label layer and a CRF layer. The vector layer is used to obtain the word embedding vector of the target text and the word embedding vector of the support set sample label. The label layer is used to obtain the emission probability of the target text based on the distance between the word embedding vector of the target text and the word embedding vector of the support set sample label. The CRF layer is used to obtain the named entity recognition result of the target text based on the emission probability and the abstract transition probability matrix.

[0070] In the above embodiment, the emission probability and the abstract transition probability matrix are respectively the input and output of the CFR layer. The CFR layer refers to the Conditional Random Field (CRF). The Conditional Random Field is a basic model for natural language processing and is widely used in annotation scenarios such as Chinese word segmentation, named entity recognition, and part-of-speech tagging.

[0071] It should be noted that the emission probability of the target text is obtained according to the distance between the word embedding vector of the target text and the word embedding vector of the support set sample label. Specifically, it can be that the distance between any word embedding vector in the target text and each support set sample label word embedding vector is calculated to obtain multiple candidate distances, and the candidate emission probability corresponding to the support set sample label with the smallest candidate distance between the word embedding vector and the candidate is selected as the final emission probability of the word embedding vector. The smallest candidate distance conforms to the nearest neighbor idea, that is, the smallest candidate distance indicates that the word embedding vector is the nearest neighbor to the corresponding support set sample label. In the above embodiment, the CRF layer is used to obtain the named entity recognition result of the target text according to the emission probability and the abstract transition probability matrix. The input of the CRF layer is the emission probability of the word embedding vector of the target text and the abstract transition probability matrix established after pre-training. The output of the CRF layer is the category and probability of the entity extracted from the target text, and the abstract transition probability matrix is ​​the constraint of the CRF layer.

[0072] In some embodiments, the CRF layer includes extraction of named entities of multiple categories, and each category of named entities is extracted and recognized based on the CRF layer to obtain a named entity recognition result of each category. For example, in a specific implementation, a pipeline mode can be used to extract and recognize each category in turn, so that the problem of nested named entity recognition can be effectively solved.

[0073] For example, the nested named entity "Peking University" belongs to the entity of the organization name type, and the nested "Beijing" is also an entity of the place name type. Through the CRF layer provided by the above embodiment, multiple categories of named entities can be extracted respectively, and named entity recognition results of different categories can be obtained respectively, thereby effectively solving the problem of nested named entity recognition.

[0074] The named entity recognition method of the embodiment of the present invention, the named entity recognition model adopts the technical architecture of vector layer + label layer + CRF layer, and obtains the abstract transfer probability matrix by reconstructing the original transfer probability matrix. Since the categories included in the abstract transfer probability matrix are consistent with the support set sample labels, the embodiment of the present invention ensures that the constraints of the CRF layer are related to the target domain under the premise of using data from a general domain for pre-training, solves the problem that the CRF layer cannot be directly used across labels, and better utilizes the CRF layer to improve the overall effect of entity recognition.

[0075] In some embodiments, the process of the vector layer acquiring the word embedding vector of the support set sample label can be that the named entity recognition model includes a label layer of the "electrical appliance" class, the support set data is input into the named entity recognition model, and through the label layer of the "electrical appliance" class in the model, the vector layer of the model can extract the entity words belonging to the "electrical appliance" class in the support set, and obtain the word embedding vector of the "electrical appliance" class. For example, "the refrigerator has a defrost function" can extract the "refrigerator" belonging to the "electrical appliance" class. Alternatively, the named entity recognition model includes a label layer of the "function" class, the support set data is input into the named entity recognition model, and through the label layer of the "function" class in the model, the vector layer of the model can extract the entity words belonging to the "function" class in the support set, and obtain the word embedding vector of the "function" class.

[0076] In the above embodiment, the process of the vector layer obtaining the word embedding vector of the target text can be: the named entity recognition model includes a label layer of the "electrical appliance" class, then through the label layer of the "electrical appliance" class, the entity words belonging to the "electrical appliance" class in the target text can be extracted to obtain the word embedding vector of the "electrical appliance" class, such as "does the rice cooker have a heating function" can extract the three words "electric cooker".

[0077] It should be noted that the label layer is constructed using a given support set, i.e., a small amount of annotated data. When extracting and identifying entities from the target text, if the target text contains multiple support set sample labels, it is necessary to calculate the emission probability of the target text label by label. For each label layer, the text to be extracted is measured and calculated to obtain the probability of belonging to each category label as the emission probability of the CRF layer. For example, the target text is "Does the rice cooker have a heating function?", the support set samples include "Does the refrigerator have a defrost function?", the support set sample labels include "Electrical appliances" and "Functions", the "Electrical appliances" label contains two word embedding vectors of "ice" and "box", and the "Function" label contains four word embedding vectors of "defrost", "function", "function" and "energy". Randomly select a word embedding vector in the target text as "electricity", then for the "electrical appliance" label layer, it is necessary to calculate the candidate emission probabilities of the word embedding vectors "ice" and "box" under "electrical appliance" respectively. For the "function" label layer, it is necessary to calculate the candidate emission probabilities of "defrost", "power" and "energy" under "function" respectively. All candidate emission probabilities are measured and calculated, and finally the emission probabilities of the word embedding vectors "electricity" and "electrical appliance" are obtained and input into the CRF layer.

[0078] The named entity recognition method of the embodiment of the present invention calculates the emission probability by extracting each word embedding vector in the target text and each word embedding vector under a certain type of label. The calculation of the two word embedding vectors avoids the misjudgment problem that occurs when extracting word vectors to calculate the emission probability for labels (generally word vectors) in the existing methods. Therefore, the label layer system constructed in the embodiment of the present invention can effectively solve the problems of nested NER and label similarity.

[0079] Specifically, the vector layer can use the Bert model or the BiLSTM model.

[0080] The named entity recognition method of the embodiment of the present invention can achieve good results in cross-domain and cross-label named entity recognition tasks by using only a small amount of annotated data of the support set sample data. The present invention uses named entity data in a general domain for pre-training to obtain a transfer probability matrix based on general domain training, and then converts the transfer probability matrix into an abstract transfer probability matrix through a small amount of sample annotated data and trains the label layer to achieve domain and label migration. When the target text is identified to obtain the emission probability of the target text, the nearest neighbor idea is used to obtain the emission probability corresponding to the minimum distance between the two word embedding vectors, avoiding the input of the emission probability of all word embedding vectors in the target text into the CRF layer, effectively solving the problem of few-sample named entity recognition.

[0081] like Figure 2 As shown in FIG. 1 , a schematic diagram of a framework of a named entity recognition model provided by an embodiment of the present invention is shown in FIG. Figure 2 On the left is the pre-training process of the Bert+CRF model architecture selected in this embodiment. The Bert model is combined with the label layer to output the emission probability CRF layer. Since the constraint of the CFR layer is an improved abstract transfer probability matrix, the CFR layer can output the optimal path for the category recognition of the target field. Therefore, the CRF layer can limit unreasonable outputs and improve the accuracy of entity recognition. Figure 2 The right side shows the framework for recognizing the target text after pseudo-training the model with the support set.

[0082] In some embodiments of the present invention, the abstract transition probability matrix is ​​determined based on the transition probability matrix of the pre-trained named entity recognition model and the support set sample labels, and includes:

[0083] The abstract transition probability matrix is ​​determined by reconstructing and averaging the rows and columns of the transition probability matrix of the pre-trained named entity recognition model based on the types of support set sample labels.

[0084] It should be noted that the abstract transfer probability matrix is ​​the degradation of the original transfer probability matrix. The essence of the degradation is to reconstruct the rows and columns of the transfer probability matrix according to the number of types of support set sample labels. The reconstructed rows and columns are all types of support set sample labels, and then the probability of each type after reconstruction is averaged to finally obtain the abstract transfer probability matrix.

[0085] like Figure 3 As shown, the transition probability matrix of the pre-trained named entity recognition model in the above embodiment is as follows: Figure 3 As shown in the matrix on the left, the pre-training uses general domain data, such as "name" + "place name" + "organization structure name". Figure 3 The corresponding pre-training data is "Xiao Ming goes to school", Figure 3 The meaning of the transition probability matrix on the left side can be that I-PER represents "person's name", I-LOC represents "place name", I-ORG represents "organization name", and O represents others. Each element in the transition probability matrix represents the transition probability between two labels. For the CRF layer of the pre-trained named entity recognition model, the transition probability matrix is ​​extracted and degenerated to obtain the abstract transition probability containing only I and I-other. I represents electrical appliances, O represents non-electrical appliances, and I-other represents other categories of non-electrical appliances.

[0086] from Figure 3 The transition probability matrix on the left is transformed to Figure 3 The process of the abstract transition probability matrix on the right can be:

[0087] The number of rows and columns of the matrix is ​​obtained according to the number of categories of the support set sample labels. In this embodiment, the support set sample labels can be "electrical appliances" and "functions", which correspond to Figure 3 In I and I-other, O is a non-electrical device, which is consistent with O in the original matrix.

[0088] The abstract transfer probability matrix is ​​initialized according to the value of the pre-trained transfer probability matrix. In this embodiment, the initialization may be: directly migrating the value of the initialized abstract transfer probability matrix (O, O) to 0.7, adding (0, I-PER), (0, I-LOC) and (0, I-ORG) to obtain the value of (0, I), that is, 0.15+0.1+0.05=0.3, and adding (I-PER, I-PER), (I-LOC, I-LOC) and (I-ORG, I-ORG) to obtain an average value of The value of (I,I) is (0.55+0.35+0.6) / 3=0.5, and the value of (I-PER,0), (I-LOC,0) and (I-ORG,0) are added and averaged to obtain the value of (I,0), which is (0.3+0.35+0.25) / 3=0.3. The value of the remaining unused transfer probability matrices is added and averaged (divided by 3) to obtain the value of (I,I-other), which is ((0.05+0.1)+(0.1+0.2)+(0.1+0.05)) / 3=0.2.

[0089] In at least some embodiments of the present invention, obtaining the emission probability of the target text according to the distance between the word embedding vector of the target text and the word embedding vector of the support set sample label includes:

[0090] Calculate the minimum distance between the word embedding vector of the target text and the word embedding vector of the support set sample label;

[0091] It should be noted that, in this embodiment, the minimum distance between the word embedding vector of the target text and the word embedding vector of the support set sample label is calculated. In fact, each word in the text to be recognized is compared with each word of all entity categories in the support set text belonging to the target domain. The similarity of two words can be determined by calculating the distance between the word embedding vectors corresponding to the two words. If the distance value is greater than a preset threshold, it means that the two words are not similar. If the distance value is less than the preset threshold, the two words are similar.

[0092] The emission probability of the target text is obtained based on the minimum distance, and the emission probability is the emission probability of the input CRF layer.

[0093] like Figure 4As shown, the embodiment of the present invention illustrates the acquisition process of the emission probability through a specific example. This embodiment hopes to extract two entity categories of "function" and "electrical appliance". The support set data support_set contains text about "function" and "electrical appliance" and the corresponding entities. Assume that the support set text is "the refrigerator has a defrost function", that is, the "text" in support_set is: "the refrigerator has a defrost function", and the "refrigerator" and "defrost function" therein are labeled, that is, the "label" in support_set is: "electrical appliance": [[0,2]] and "function" [[4,8]], where [[,]] represents the serial number of the character corresponding to the label in the text. The whole sentence is input into the base model, and for each category, such as "electrical appliance", the word-level vector representation corresponding to "refrigerator" is extracted, that is, the embedding of "ice" and the embedding of "box". For the sentences that need to be predicted in the query set query_set, such as "Does the rice cooker have a heating function", that is, "text" in query_set is "Does the rice cooker have a heating function", it is also input into the base model, and the vector representation of each word is calculated, such as the embedding of "electricity". For the three words "electric rice cooker", the minimum distances with all word vectors in the "electrical appliances" category and the "non-electrical appliances" category of the support set text are calculated respectively. The distance between the embedding of "electricity" and the embedding of "ice" is distance1, and the distance between the embedding of "electricity" and the embedding of "box" is distance2. Since distance1 and distance1 are smaller than all other distances and "ice" and "box" both belong to the electrical appliance category, the "I-electric appliance" emission probability of "electricity" is used as the emission probability of the input CRF layer. The result obtained after the named entity recognition model of the embodiment of the present invention extracts "Does the rice cooker have a heating function" is:

[0094] "Electric appliances": "rice cooker",

[0095] "Function": "Heating function",

[0096] “sequence labeling”: [“I-electrical”, “I-electrical”, “I-electrical”, “O”, “O”, “O”, “I-function”, “I-function”, “I-function”, “I-function”].

[0097] Among them, sequence labeling outputs the corresponding category according to the character sequence of the target text.

[0098] In some embodiments of the present invention, obtaining a named entity recognition result of a target text according to the emission probability and the abstract transition probability matrix includes:

[0099] The optimal sequence annotation path is obtained by decoding through the Viterbi algorithm according to the emission probability and the abstract transition probability matrix, thereby obtaining the named entity recognition result of the target text. The named entity recognition result of the target text includes the target entity belonging to the support set sample label in the target text. In this embodiment, the optimal sequence annotation path is obtained by decoding using the Viterbi algorithm, which can improve the accuracy of entity recognition.

[0100] In some embodiments of the present invention, a method for named entity recognition is disclosed, comprising:

[0101] Step 201: pre-train a named entity recognition model to obtain a transition probability matrix of the pre-trained named entity recognition model.

[0102] Step 202: Pseudo-train the pre-trained named entity recognition model based on the support set samples and the corresponding support set sample labels.

[0103] Step 203: Obtain an abstract transition probability matrix according to the transition probability matrix and the support set sample labels.

[0104] Step 204: Obtain a target text to be recognized, and recognize the target text based on the named entity recognition model to obtain the emission probability of the target text and the abstract transition probability matrix of the named entity recognition model, wherein the emission probability is determined based on the word embedding vector of the target text and the word embedding vector of the support set sample label.

[0105] Step 205: Based on a named entity recognition model, the emission probability and the abstract transition probability matrix are identified to obtain a named entity recognition result of the target text.

[0106] The named entity recognition method provided by the embodiment of the present invention is first pre-trained based on data from a general field, and then a named entity recognition model is obtained through pseudo-training on a small amount of entity recognition annotated data in a target field. Since an abstract transition probability matrix is ​​constructed based on the support set sample labels, the obtained named entity recognition model can be used to implement entity extraction in the target field across domains. The user only needs to annotate a small amount or even one piece of data to obtain a good entity extraction effect, which effectively solves the problem of poor entity extraction effect caused by insufficient data annotation.

[0107] The following is a description of the named entity recognition device provided by the present invention. The named entity recognition device described below and the named entity recognition method described above can be referred to in correspondence with each other. Figure 5 As shown, the embodiment of the present invention also discloses a named entity recognition device, including:

[0108] The input module 501 is used to obtain the target text to be recognized.

[0109] The recognition module 502 is used to recognize the target text based on the named entity recognition model to obtain the emission probability of the target text and the abstract transition probability matrix of the named entity recognition model, and recognize the emission probability and the abstract transition probability matrix to obtain the named entity recognition result of the target text.

[0110] Among them, the named entity recognition model is first pre-trained and then pseudo-trained based on the support set samples and the corresponding support set sample labels. The emission probability is determined based on the word embedding vector of the target text and the word embedding vector of the support set sample label. The abstract transition probability matrix is ​​determined based on the transition probability matrix of the pre-trained named entity recognition model and the support set sample labels.

[0111] The named entity recognition device provided by the embodiment of the present invention obtains a named entity recognition model through a small amount of entity recognition and annotation data in the target field. The obtained named entity recognition model can perform entity extraction work in the target field. The user only needs to annotate a small amount or even one piece of data to obtain a good entity extraction effect, which effectively solves the problem of poor entity extraction effect caused by insufficient data annotation.

[0112] In some embodiments of the present invention, the named entity recognition model includes a vector layer, a label layer and a CRF layer. The vector layer is used to obtain the word embedding vector of the target text and the word embedding vector of the support set sample label. The label layer is used to obtain the emission probability of the target text based on the distance between the word embedding vector of the target text and the word embedding vector of the support set sample label. The CRF layer is used to obtain the named entity recognition result of the target text based on the emission probability and the abstract transition probability matrix.

[0113] In some embodiments of the present invention, obtaining the emission probability of the target text according to the distance between the word embedding vector of the target text and the word embedding vector of the support set sample label includes:

[0114] Calculate the minimum distance between the word embedding vector of the target text and the word embedding vector of the support set sample label.

[0115] The emission probability of the target text is obtained based on the minimum distance.

[0116] In some embodiments of the present invention, a named entity recognition result of a target text is obtained according to an emission probability and an abstract transition probability matrix, wherein the named entity recognition result of the target text is obtained by decoding the emission probability and the abstract transition probability matrix through a Viterbi algorithm.

[0117] In some embodiments of the present invention, pre-training of a named entity recognition model includes: pre-training the named entity recognition model based on training set samples and corresponding training set sample labels, wherein the training set samples include sample data of a general domain, the support set samples include sample data of a target domain, and the training set sample labels and the support set sample labels belong to different domains.

[0118] In some embodiments of the present invention, the abstract transition probability matrix is ​​determined based on the transition probability matrix of the pre-trained named entity recognition model and the support set sample labels, wherein the abstract transition probability matrix is ​​determined by reconstructing and averaging the rows and columns of the transition probability matrix of the pre-trained named entity recognition model based on the types of the support set sample labels.

[0119] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the named entity recognition method, which includes:

[0120] Get the target text to be recognized.

[0121] Based on the named entity recognition model, the target text is recognized to obtain the emission probability of the target text and the abstract transition probability matrix of the named entity recognition model, and the emission probability and the abstract transition probability matrix are recognized to obtain the named entity recognition result of the target text.

[0122] Among them, the named entity recognition model is first pre-trained and then obtained based on the support set samples and the corresponding support set sample labels. The emission probability is determined based on the word embedding vector of the target text and the word embedding vector of the support set sample label. The abstract transition probability matrix is ​​determined based on the transition probability matrix of the pre-trained named entity recognition model and the support set sample labels.

[0123] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0124] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the named entity recognition method provided by the above methods, the method includes:

[0125] Get the target text to be recognized.

[0126] Based on the named entity recognition model, the target text is recognized to obtain the emission probability of the target text and the abstract transition probability matrix of the named entity recognition model, and the emission probability and the abstract transition probability matrix are recognized to obtain the named entity recognition result of the target text.

[0127] Among them, the named entity recognition model is first pre-trained and then obtained based on the support set samples and the corresponding support set sample labels. The emission probability is determined based on the word embedding vector of the target text and the word embedding vector of the support set sample label. The abstract transition probability matrix is ​​determined based on the transition probability matrix of the pre-trained named entity recognition model and the support set sample labels.

[0128] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the named entity recognition method provided by the above methods, the method comprising:

[0129] Get the target text to be recognized.

[0130] Based on the named entity recognition model, the target text is recognized to obtain the emission probability of the target text and the abstract transition probability matrix of the named entity recognition model, and the emission probability and the abstract transition probability matrix are recognized to obtain the named entity recognition result of the target text.

[0131] Among them, the named entity recognition model is first pre-trained and then obtained based on the support set samples and the corresponding support set sample labels. The emission probability is determined based on the word embedding vector of the target text and the word embedding vector of the support set sample label. The abstract transition probability matrix is ​​determined based on the transition probability matrix of the pre-trained named entity recognition model and the support set sample labels.

[0132] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0133] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for named entity recognition, characterized in that: include: Obtain the target text to be recognized; Based on the named entity recognition model, the word embedding vector of the target text and the word embedding vector of the support set sample label are identified to obtain the emission probability of the target text and the abstract transition probability matrix of the named entity recognition model; Identifying the emission probability and the abstract transition probability matrix to obtain a named entity recognition result of the target text; Wherein, the named entity recognition model is first pre-trained and then obtained based on support set samples and corresponding support set sample labels, and the abstract transition probability matrix is ​​determined based on the transition probability matrix of the pre-trained named entity recognition model and the support set sample labels; The step of identifying the word embedding vector of the target text and the word embedding vector of the support set sample label based on the named entity recognition model to obtain the emission probability of the target text includes: Calculating the minimum distance between the word embedding vector of the target text and the word embedding vector of the support set sample label; Obtaining a transmission probability of the target text based on the minimum distance; The abstract transition probability matrix is ​​determined based on the transition probability matrix of the pre-trained named entity recognition model and the support set sample labels, and includes: The abstract transition probability matrix is ​​determined by reconstructing and averaging the rows and columns of the transition probability matrix of the pre-trained named entity recognition model based on the types of the support set sample labels; The step of identifying the emission probability and the abstract transition probability matrix to obtain a named entity recognition result of the target text includes: The named entity recognition result of the target text is obtained by decoding the emission probability and the abstract transition probability matrix through the Viterbi algorithm.

2. The method for named entity recognition according to claim 1, characterized in that: The named entity recognition model includes a vector layer, a label layer and a CRF layer; The vector layer is used to obtain the word embedding vector of the target text and the word embedding vector of the support set sample label; The label layer is used to obtain the emission probability of the target text according to the distance between the word embedding vector of the target text and the word embedding vector of the support set sample label; The CRF layer is used to obtain a named entity recognition result of the target text according to the emission probability and the abstract transition probability matrix.

3. The method for named entity recognition according to claim 2, characterized in that: The CRF layer includes extraction of named entities of multiple categories, and the method further includes: Based on the CRF layer, the named entities of each category are extracted and recognized respectively to obtain the named entity recognition results of each category.

4. The method for named entity recognition according to any one of claims 1 to 3, characterized in that: The pre-training of the named entity recognition model includes: Pre-training the named entity recognition model based on training set samples and corresponding training set sample labels; The training set samples include sample data in a general domain, and the support set samples include sample data in a target domain.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for named entity recognition according to any one of claims 1 to 4 is implemented.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for named entity recognition as claimed in any one of claims 1 to 4 is implemented.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for named entity recognition according to any one of claims 1 to 4 is implemented.

Images