A method, device and electronic device for extracting entity relations from text

Abstract

The present application discloses a method, device and electronic device for extracting entity relationships from text, which obtains a target text to be processed; calls a target recognition model to recognize the target text, and determines the entity relationship recognition result corresponding to the target text. The model structure corresponding to the target recognition model includes a distillation loss module and a category increment module. The distillation loss from the basic recognition model to the target recognition model can be obtained through the distillation loss module, and the classification loss of the newly added category can be determined through the category increment module. The problem of increased labor costs caused by the need to re-label data after adding new entity and relationship categories is solved based on the model structure, and continuous learning of the model can be achieved, saving the time cost of re-learning and improving the efficiency of entity relationship extraction.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and more specifically to a method, device and electronic device for extracting entity relationships from text. Background Art

[0002] Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. Natural language processing usually includes technologies such as text processing, semantic understanding, intelligent question answering, and knowledge graphs. Among them, entity recognition and relationship extraction are the most basic parts of natural language processing applications.

[0003] Entities and relationships in text are usually extracted through machine learning models. However, in actual application scenarios, the categories of entities and / or relationships may be redefined in business scenarios every once in a while. The increase in entity or relationship categories will lead to a large amount of manual labeling, that is, the training samples of the machine learning model need to be relabeled and then retrained, which consumes huge manpower costs and reduces the efficiency of entity relationship extraction. Summary of the invention

[0004] In view of this, this application provides the following technical solutions:

[0005] A text entity relationship extraction method, comprising:

[0006] Get the target text to be processed;

[0007] Calling a target recognition model to recognize the target text and determining an entity relationship recognition result corresponding to the target text;

[0008] Among them, the model structure corresponding to the target recognition model includes a knowledge distillation module and a category increment module. The knowledge distillation module is used to obtain the distillation loss when the basic recognition model is distilled to the target recognition model, and the category increment module is used to obtain the new category classification loss corresponding to the new entity relationship category relative to the basic recognition model, so that the target recognition model is determined by the basic recognition model, the distillation loss and the new category classification loss, and the recognition tasks corresponding to the target recognition model and the basic recognition model are the same.

[0009] Optionally, the method further comprises:

[0010] Acquire target training sample data, wherein the target training sample data is annotated with first entity relationship category annotation data for the newly added entity relationship category;

[0011] Identify the target training sample data based on the basic recognition model to obtain second entity relationship category labeling data;

[0012] Generate a word vector matrix corresponding to the first entity relationship category labeling data and the second entity category labeling data through a matrix filling module in the model structure corresponding to the target recognition model;

[0013] Processing the word vector matrix through the knowledge distillation module to obtain a distillation loss;

[0014] The word vector matrix is ​​processed by the category increment module to obtain a new category classification loss;

[0015] The target recognition model is determined based on the distillation loss, the newly added category classification loss and the target training sample data.

[0016] Optionally, the method further comprises:

[0017] Acquire first text data and entity relationship category information matching the first text data;

[0018] Performing entity relationship annotation on the first text data based on the entity relationship category information to obtain a first training sample;

[0019] Performing word segmentation on the first training sample, and performing word vector representation based on the word segmentation result to obtain a word vector matrix;

[0020] Determine a loss function based on the word vector matrix;

[0021] Training is performed based on the loss function and the first training sample to obtain a basic recognition model.

[0022] Optionally, determining a loss function based on the word vector matrix includes:

[0023] Using the basic recognition model to predict each word in the word segmentation result, and obtaining predicted entities and relationships corresponding to each word;

[0024] Based on the predicted entity, determining an entity cross entropy loss between the entity annotated with each of the words;

[0025] Determining a relational cross entropy loss for a relation with each of the word tags based on the predicted relation;

[0026] A loss function is determined according to the entity cross entropy loss and the relationship cross entropy loss.

[0027] Optionally, the distillation loss includes entity distillation loss and relationship distillation loss, the newly added category classification loss includes newly added entity category classification loss and newly added relationship classification loss, and the method further includes:

[0028] Determine the category information corresponding to each newly added node in the word vector matrix in the matrix filling module;

[0029] If the category information indicates that the newly added node is an entity category, determining the overall loss of the node based on the entity distillation loss and the newly added entity category classification loss;

[0030] If the category information indicates that the newly added node is a relationship category, determining the overall loss of the node based on the relationship distillation loss and the newly added relationship category classification loss;

[0031] Based on the overall loss of each newly added node, a loss function of the target recognition model is determined.

[0032] Optionally, obtaining a new category classification loss corresponding to the new entity relationship category relative to the basic recognition model includes:

[0033] Predicting data samples corresponding to newly added entity relationship categories relative to the basic recognition model using the target recognition model to obtain predicted entity relationship data;

[0034] Based on the predicted entity relationship data and the first entity relationship annotation data, a new category classification loss of entity relationship extraction corresponding to the new entity relationship category information is determined.

[0035] Optionally, obtaining the distillation loss when the base recognition model is distilled to the target recognition model includes:

[0036] Determining a first entity relationship classification probability distribution corresponding to the basic recognition model;

[0037] Obtaining a second entity relationship classification probability distribution in a training phase corresponding to the target recognition model;

[0038] A distillation loss is determined according to the first entity relationship classification probability distribution and the second entity relationship classification probability distribution.

[0039] Optionally, the method further comprises:

[0040] If newly added entity categories and / or relationship categories for the target recognition model are obtained;

[0041] Based on the target recognition model and the training samples corresponding to the newly added entity categories and / or relationship categories, the target recognition model is updated to obtain an updated target recognition model. The updated target recognition model can extract entity relationships from the text of the newly added entity categories and / or relationship categories and the original entity categories corresponding to the target recognition model.

[0042] A text entity relationship extraction device, comprising:

[0043] An acquisition unit, used for acquiring a target text to be processed;

[0044] A calling unit, used for calling a target recognition model to recognize the target text and determine an entity relationship recognition result corresponding to the target text;

[0045] Among them, the model structure corresponding to the target recognition model includes a knowledge distillation module and a category increment module. The knowledge distillation module is used to obtain the distillation loss when the basic recognition model is distilled to the target recognition model, and the category increment module is used to obtain the new category classification loss corresponding to the new entity relationship category relative to the basic recognition model, so that the target recognition model is determined by the basic recognition model, the distillation loss and the new category classification loss, and the recognition tasks corresponding to the target recognition model and the basic recognition model are the same.

[0046] An electronic device, comprising:

[0047] A memory, used to store applications and data generated by the operation of the applications;

[0048] A processor is used to execute the application program to implement the entity relationship extraction method of the text as described in any one of the above.

[0049] Through the above technical solutions, it can be known that the present application discloses a method, device and electronic device for extracting entity relationships from text, obtains a target text to be processed; calls a target recognition model to recognize the target text, and determines the entity relationship recognition result corresponding to the target text. The model structure corresponding to the target recognition model includes a distillation loss module and a category increment module. The distillation loss from the basic recognition model to the target recognition model can be obtained through the distillation loss module, and the classification loss of the newly added category can be determined through the category increment module. The problem of increased labor costs caused by the need to re-label data after adding new entity and relationship categories is solved based on the model structure, and the continuous learning of the model can be realized, saving the time cost of re-learning and improving the efficiency of entity relationship extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0051] Figure 1 A flowchart of a method for extracting entity relationships from text provided in an embodiment of the present application;

[0052] Figure 2 A schematic diagram of the structure of a target recognition model provided in an embodiment of the present application;

[0053] Figure 3 A schematic diagram of matrix filling provided in an embodiment of the present application;

[0054] Figure 4 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0055] Figure 5 A schematic diagram of the structure of a text entity relationship extraction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are described as follows:

[0058] An entity is a thing or concept that can be defined within a system, such as a person's name, a role, a place name, an event, etc. Correspondingly, the entity recognition task is to identify the entities and categories that exist in a text sentence.

[0059] Relation extraction: A relation is defined as the connection between two or more entities. Relation extraction is to identify the relationship between multiple entities by learning the semantic connection between them in the text. The input of relation extraction is a piece of text, and the output is usually a triple: (entity 1, relation, entity 2). For example, the input text "Li Si's address is Heping Street" is output as a triple after relation extraction (Li Si, address, Heping Street). The relation extraction task is to identify the relationship between any two entities in a sentence, which can be modeled as a text classification task.

[0060] Unstructured information extraction refers to the joint entity relationship extraction task, which aims to extract triplets from text, such as (head entity, relationship, tail entity).

[0061] See also Figure 1 , is a flow chart of a method for extracting entity relationships from text provided in an embodiment of the present application, the method may include the following steps:

[0062] S101, obtaining a target text to be processed.

[0063] The target text to be processed refers to the text that needs to be subjected to entity relationship extraction, which can be a paragraph of text content or a sentence. The target text can be the original text information obtained, that is, the text that has not been processed in any way, or the text that has been preprocessed, such as text content format unification, duplicate content removal, or data cleaning.

[0064] S102: calling a target recognition model to recognize a target text, and determining an entity relationship recognition result corresponding to the target text.

[0065] The entity relationship recognition result obtained by the target recognition model is mainly reflected in the form of triples, that is, the entity relationship recognition result is [entity 1, relationship, entity 2].

[0066] The target recognition model in the embodiment of the present application does not adopt the model structure of a general recognition model, that is, it is not obtained by annotating all sample data of the recognition task corresponding to the target recognition model to obtain the target training sample, and then training it through a neural network. Instead, on the basis of the existing entity relationship recognition model, it is only necessary to annotate the entity relationship of the incremental samples to obtain a neural network model. Specifically, the model structure corresponding to the target recognition model includes a knowledge distillation module and a category increment module, the knowledge distillation module is used to obtain the distillation loss when the basic recognition model is distilled to the target recognition model, and the category increment module is used to obtain the newly added category classification loss corresponding to the newly added entity relationship category relative to the basic recognition model, so that the target recognition model is determined by the basic recognition model, the distillation loss and the newly added category classification loss. Among them, the recognition tasks corresponding to the target recognition model and the basic recognition model are the same.

[0067] The application scenario corresponding to the present application is a scenario for extracting entity relationships from text after adding new entity and / or relationship categories on the basis of an existing recognition model. Among them, the existing recognition model is a basic recognition model, which is used to extract existing entity relationships before adding entity and / or relationship categories. The target recognition model can recognize the text corresponding to the newly added entity and / or relationship categories, but the recognition tasks of the two are the same. For example, both the basic recognition model and the target recognition model extract entity relationships in the user information field. For example, when the basic recognition model is trained, the existing entity categories include the first entity category and the second entity category, and the relationship category includes the first relationship category and the second relationship category. At this time, the result that the basic recognition model can recognize is the entity relationship triple corresponding to the first entity category, the second entity category, the first relationship category and the second relationship category. If a third entity category is added, the target recognition model can identify the entity relationship triple corresponding to the first entity category, the second entity category, the third entity category and the first relationship category and the second relationship category. When obtaining the target recognition model, the embodiment of the present application does not need to re-label the entity relationships of all sample data after adding the third entity category, but only labels the entity relationships related to the third entity category. The original entity relationship labeling data is obtained through the basic recognition model.

[0068] Specifically, in an embodiment of the present application, the target recognition model has a specific model structure, which includes a knowledge distillation module and a category increment module. The knowledge distillation module can transfer the knowledge of the basic recognition model to the target recognition model, that is, to obtain the distillation loss of the basic recognition model distilled to the target recognition model. The newly added category recognition model learns the knowledge of the newly added category based on the general classification problem, that is, to obtain the classification loss of the newly added category, so that the loss function of the target recognition model can be determined according to the distillation loss and the newly added classification loss. The target recognition model is obtained based on the loss function and the distilled knowledge of the original basic recognition model. Thereby, the problem of increasing entity and relationship categories at the same time can be solved, and the cost of data annotation and model retraining can be reduced, and the catastrophic forgetting of the model can be reduced.

[0069] In an embodiment of the present application, a method for constructing a target recognition model is also provided, and the method may include the following steps:

[0070] S201: Obtain target training samples.

[0071] The target training sample data is annotated with first entity relationship category annotation data for a newly added entity relationship category.

[0072] S202: Identify the target training sample data based on the basic recognition model to obtain second entity relationship category labeling data.

[0073] S203. Generate a word vector matrix corresponding to the first entity relationship category annotation data and the second entity category annotation data through a matrix filling module in a model structure corresponding to the target recognition model.

[0074] S204. Process the word vector matrix through the knowledge distillation module to obtain the distillation loss.

[0075] S205. Process the word vector matrix through the category increment module to obtain the new category classification loss.

[0076] S206: Determine a target recognition model based on the distillation loss, the newly added category classification loss, and the target training sample data.

[0077] The target training sample data may be text data including only the newly added entity relationship categories, or text data including the existing entity relationship categories and the newly added entity relationship categories. It should be noted that in the embodiment of the present application, only the corresponding triple data of the newly added entity relationship categories need to be annotated (i.e., the first entity relationship category annotated data is obtained), and there is no need to annotate the triple data corresponding to all entity relationship categories.

[0078] For the existing entity relationship categories, the corresponding basic recognition model can be used to identify the target training sample data to obtain the second entity relationship category annotation data corresponding to the existing entity relationship category. The first entity relationship category annotation data and the second entity relationship category annotation data represent the entity relationship triples in the text corresponding to the current target training sample data, such as [entity 1, relationship 1, entity 2], [entity 3, relationship 2, entity 4], etc.

[0079] In the embodiment of the present application, entity and relationship extraction are defined as a joint modeling problem of matrix filling, and matrix knowledge distillation is constructed to jointly solve the catastrophic forgetting caused by the simultaneous increase of entity and relationship categories. A category increment module is built to realize new category learning. Through the configured model structure of the target recognition model, when the categories of entities and relationships are increased at the same time, the ability to recognize new categories is increased while maintaining the old category recognition capabilities of entities and relationships. Therefore, the model structure of the target recognition model also includes a matrix filling module, which is used to generate a word vector matrix corresponding to the first entity relationship category annotation data and the second entity category annotation data. The knowledge of the basic recognition model can be obtained through the knowledge distillation module, and the distillation loss can be further obtained. The newly added entity relationship categories can be learned through the category increment module, and the classification loss of the newly added categories can be further obtained.

[0080] See also Figure 2, which is a structural diagram of a target recognition model provided in an embodiment of the present application, the model structure of the target recognition model includes a matrix filling module 301, a knowledge distillation module 302 and a category increment module 303, wherein the knowledge distillation module 302 and the category increment module 303 constitute the continuous learning part of the target recognition model.

[0081] In an embodiment of the present application, a method for generating a basic recognition model is also provided, and the method may include the following steps:

[0082] S401: Acquire first text data and entity relationship category information matching the first text data.

[0083] S402: Annotate the first text data with entity relationships based on the entity relationship category information to obtain a first training sample.

[0084] S403: Segment the first training sample, and perform word vector representation based on the word segmentation result to obtain a word vector matrix.

[0085] S404: Determine a loss function based on the word vector matrix.

[0086] S405: Perform training based on the loss function and the first training sample to obtain a basic recognition model.

[0087] Among them, the first text data is the text data used to train the basic recognition model. When the first text data is annotated with entity relationships, the entity relationship category corresponding to the first text data is first determined, so that when new entities and / or relationship categories are added later, the specific new entity and / or relationship category information relative to the basic recognition model can be clearly identified. Then the entity relationships in the text data are annotated to obtain the first training sample. The first training sample is segmented and represented by word vectors to obtain this vector matrix. The neural network model is trained based on the word vector matrix. During the training process, the loss function is determined by comparing the predicted results with the actual annotation results. The model parameters are adjusted based on the loss function to obtain the final basic recognition model.

[0088] Furthermore, the loss function is determined based on the word vector matrix, including: using the basic recognition model to predict each word in the word segmentation result to obtain predicted entities and relationships corresponding to each word; based on the predicted entities, determining the entity cross entropy loss between the entities annotated with each of the words; based on the predicted relationships, determining the relationship cross entropy loss between the relationships annotated with each of the words; and determining the loss function based on the entity cross entropy loss and the relationship cross entropy loss.

[0089] In this application, the recognition problem of the newly added category is constructed into a matrix filling model, that is, the word vector matrix is ​​generated by the matrix filling module 301. Figure 3 , which is a schematic diagram of a matrix filling provided in an embodiment of the present application. First, the sentence is input into the information extraction model to obtain the vector representation of each word. If the sentence has N words, the word is represented as (N*H), where H is the length of the word representation. According to the relationship between words, a word vector matrix of (N*N*H) is constructed. Figure 3 In the table, the horizontal axis is the entity, the vertical axis is also the entity, and the superposition between entities is the relationship between the corresponding entities, such as [Zhang San, lives in City A]. Figure 3 The diagonal words represent the loss used to construct the real entity (expressed as NER loss), and the off-diagonal words represent the loss used to construct the real relationship (expressed as REloss). The information extraction model can be trained by back propagation of NER loss + RE loss. In this way, the corresponding classification loss can be obtained through the matrix filling module. When adding new entities or relationships, they can be supplemented in the matrix filling module. For example, when adding entities, you can supplement Figure 3 The horizontal and vertical axis positions in the figure can be added in the corresponding entity superimposed grid if a relationship is added.

[0090] In another implementation of the embodiment of the present application, the distillation loss includes entity distillation loss and relationship distillation loss, the newly added category classification loss includes newly added entity category classification loss and newly added relationship classification loss, and the method further includes:

[0091] Determine the category information corresponding to each newly added node in the word vector matrix in the matrix filling module; if the category information characterizes that the newly added node is an entity category, determine the overall loss of the node based on the entity distillation loss and the newly added entity category classification loss; if the category information characterizes that the newly added node is a relationship category, determine the overall loss of the node based on the relationship distillation loss and the newly added relationship category classification loss; based on the overall loss of each newly added node, determine the loss function of the target recognition model.

[0092] After adding entity and / or relationship categories, the matrix filling module can be used to determine the classification loss corresponding to each newly added node, thereby determining the overall loss of the node. Figure 3 Nodes on the diagonal or off-diagonal lines of .

[0093] In one implementation of the embodiments of the present application, the classification loss of the newly added category is determined by comparing the predicted result obtained by extracting entity relationships from text using a trained model with the actual annotation result. Specifically, the classification loss of the newly added category corresponding to the newly added entity relationship category relative to the basic recognition model includes: predicting the data samples corresponding to the newly added entity relationship category relative to the basic recognition model using the target recognition model to obtain predicted entity relationship data; and determining the classification loss of the newly added category of entity relationship extraction corresponding to the newly added entity relationship category information based on the predicted entity relationship data and the first entity relationship annotation data.

[0094] In another embodiment, obtaining the distillation loss when the basic recognition model is distilled to the target recognition model includes: determining a first entity relationship classification probability distribution corresponding to the basic recognition model; obtaining a second entity relationship classification probability distribution in a training phase corresponding to the target recognition model; and determining the distillation loss based on the first entity relationship classification probability distribution and the second entity relationship classification probability distribution.

[0095] The distillation loss is the loss obtained by evaluating the knowledge distillation in combination with the corresponding basic knowledge model. A sample data can be selected and input into the basic recognition model and the target recognition model respectively, so as to obtain the corresponding entity relationship recognition results through the basic recognition model and the target recognition model respectively, and the probability distribution of the triples corresponding to each entity relationship recognized by the basic recognition model is recorded as the first entity relationship probability distribution, and the probability distribution of the triples corresponding to each entity relationship recognized by the target recognition model is recorded as the second entity relationship probability distribution. Both the first entity relationship probability distribution and the second entity relationship probability distribution are used to reflect the distribution of the probability or score of the selected sample data belonging to various candidate triples.

[0096] After obtaining the recognition probability distribution for the same sample data based on the basic recognition model and the target recognition model, the distillation loss of the corresponding entity relationship recognition task can be determined based on the gap between the first entity relationship probability distribution and the second entity relationship probability distribution, and the distillation loss can be used as a parameter iteration of the target recognition model.

[0097] The continuous learning part of the model structure of the target recognition model in the embodiment of the present application includes a knowledge distillation module and a category increment module. In actual application scenarios, if new categories of entities and relationships appear, Figure 3 The matrix filling model in cannot be used because Figure 3 The category outputs of entities and relations in the matrix filling model are predefined. Therefore, a new model structure needs to be proposed to adapt to the scenario where the number of categories of unstructured information extraction increases.

[0098] For details, see Figure 4 , is a schematic diagram of an application scenario provided by an embodiment of the present application. Among them, encoder 1 is a basic recognition model, that is, a model that can perform entity relationship recognition on existing entity and relationship categories. Figure 4 In the figure, encoder 1 identifies an information extraction model with N entity categories (N=2) and M relationship categories (M=3), and encoder 2 represents a target recognition model, that is, an information extraction model with one more category for entity and relationship respectively.

[0099] When building encoder 2, based on Figure 3 The matrix filling model in the matrix filling module sets the output neuron of each diagonal node in the extraction matrix corresponding to encoder 2 to N+1, and the output neuron of each non-diagonal node to M+1. Then encoder 2 is initialized, and the parameters other than the newly added categories are used during initialization, and the other parameters are initialized with encoder 1.

[0100] Knowledge distillation module: Only consider the N entity categories and M relationship categories in encoder 1 to determine the distillation loss from encoder 1 (i.e., the base recognition model) to encoder 2 (i.e., the target recognition model). Figure 3 The diagonal output nodes in the upper and lower matrices correspond one-to-one to construct the entity distillation loss (expressed as NER distillation loss); the relationship nodes correspond one-to-one to the non-diagonal output nodes in the upper and lower matrices to construct the relationship distillation loss (expressed as RE distillation loss).

[0101] The category increment module includes the category increment of relations and the category increment of entities. Figure 4 The hollow circles represent the original entities and relations, and the solid circles represent the newly added entities and relations. The category increment of relations refers to the classification loss calculated by the nodes of the newly added relations and the new category labels in the off-diagonal output nodes, which is recorded as RE classification loss. The category increment of entities refers to the classification loss calculated by the new relation nodes and the new category labels in the diagonal output nodes, which is recorded as NER classification loss.

[0102] For each node in the upper and lower matrices, the distillation loss and classification loss are calculated one by one, and the losses are summed and averaged. After back propagation, an information extraction model that adds both entities and categories can be obtained, that is, a target recognition model.

[0103] exist Figure 4 Existing entities may include names of people and places, and existing relationships may include "does not exist", "lives in", and "is located in". New entities may include names of organizations, and new relationships may include "works in". Figure 4In the encoder 2, only the relevant information of the newly added entities and relationships needs to be annotated, and other information can be obtained by distillation through the encoder 1. For details, please refer to the detailed description process above.

[0104] Since the newly added categories of entities and relationships can be continuously increased, that is, the newly added entity and / or relationship categories can continue to be learned on the basis of the target recognition model, so that the updated recognition model can extract entity relationships for the newly added types, that is, when new categories are added, the above-mentioned iterative process is repeated based on the model structure in the embodiment of the present application, and lifelong learning of information extraction can be achieved. Specifically, if new entity categories and / or relationship categories are obtained for the target recognition model; based on the target recognition model and the training samples corresponding to the new entity categories and / or relationship categories, the target recognition model is updated to obtain an updated target recognition model, and the updated target recognition model can extract entity relationships for the text of the new entity categories and / or relationship categories and the original entity categories corresponding to the target recognition model.

[0105] In a text entity relationship extraction method provided in an embodiment of the present application, it is not necessary to re-label all entity relationships in the sample data each time an entity or relationship category is added. The entity relationship extraction problem is constructed into a matrix filling model, and the knowledge distillation module and the category increment module in the model structure are used to learn the knowledge of the old categories through knowledge distillation between matrices, and learn the knowledge of the new categories through ordinary classification problems. The problem of adding entity and relationship categories at the same time is defined and solved, and the cost of data annotation and model retraining is reduced, and the catastrophic forgetting of the model is reduced. Through the interactive cooperation of the knowledge distillation module and the category increment module, it is possible to simultaneously increase entity and relationship categories while avoiding catastrophic forgetting as much as possible, minimize the annotation cost, retain the model performance, and greatly improve the overall adaptability of downstream tasks of information extraction (KBQA, query, recommendation, search).

[0106] In another embodiment of the present application, a text entity relationship extraction device is also provided. Figure 5 ,include:

[0107] An acquisition unit 501 is used to acquire a target text to be processed;

[0108] A calling unit 502 is used to call a target recognition model to recognize the target text and determine an entity relationship recognition result corresponding to the target text;

[0109] Among them, the model structure corresponding to the target recognition model includes a knowledge distillation module and a category increment module. The knowledge distillation module is used to obtain the distillation loss when the basic recognition model is distilled to the target recognition model, and the category increment module is used to obtain the new category classification loss corresponding to the new entity relationship category relative to the basic recognition model, so that the target recognition model is determined by the basic recognition model, the distillation loss and the new category classification loss, and the recognition tasks corresponding to the target recognition model and the basic recognition model are the same.

[0110] The present application discloses a text entity relationship extraction device, wherein an acquisition unit acquires a target text to be processed; a calling unit calls a target recognition model to recognize the target text, and determines an entity relationship recognition result corresponding to the target text. The model structure corresponding to the target recognition model includes a distillation loss module and a category increment module, and the distillation loss from the basic recognition model to the target recognition model can be obtained through the distillation loss module, and the classification loss of the newly added category can be determined through the category increment module. The problem of increased labor costs caused by the need to re-label data after adding new entity and relationship categories is solved based on the model structure, and continuous learning of the model can be achieved, saving the time cost of re-learning and improving the efficiency of entity relationship extraction.

[0111] Optionally, the device further comprises:

[0112] A first acquisition unit, configured to acquire target training sample data, wherein the target training sample data is annotated with first entity relationship category annotation data for the newly added entity relationship category;

[0113] A first recognition unit, configured to recognize the target training sample data based on the basic recognition model to obtain second entity relationship category labeling data;

[0114] A first generating unit, configured to generate a word vector matrix corresponding to the first entity relationship category labeling data and the second entity category labeling data through a matrix filling module in a model structure corresponding to the target recognition model;

[0115] A first processing unit, configured to process the word vector matrix through the knowledge distillation module to obtain a distillation loss;

[0116] A second processing unit, configured to process the word vector matrix through the category increment module to obtain a new category classification loss;

[0117] The first determining unit is used to determine the target recognition model based on the distillation loss, the newly added category classification loss and the target training sample data.

[0118] Optionally, the device further comprises:

[0119] A second acquisition unit, used to acquire first text data and entity relationship category information matching the first text data;

[0120] A first labeling unit, configured to label the first text data with entity relationships based on the entity relationship category information to obtain a first training sample;

[0121] A word segmentation unit, used to segment the first training sample, and perform word vector representation based on the word segmentation result to obtain a word vector matrix;

[0122] A second determining unit, configured to determine a loss function based on the word vector matrix;

[0123] A training unit is used to perform training based on the loss function and the first training sample to obtain a basic recognition model.

[0124] Further, the second determining unit includes:

[0125] A first prediction subunit, configured to predict each word in the word segmentation result by using the basic recognition model, and obtain a predicted entity and relationship corresponding to each word;

[0126] A first determination subunit, configured to determine, based on the predicted entity, an entity cross entropy loss between the entity annotated with each of the words;

[0127] A second determination subunit is used to determine the relationship cross entropy loss between the relationship with each of the word tags based on the predicted relationship;

[0128] The third determination subunit is used to determine the loss function according to the entity cross entropy loss and the relationship cross entropy loss.

[0129] Optionally, the distillation loss includes entity distillation loss and relationship distillation loss, the newly added category classification loss includes newly added entity category classification loss and newly added relationship classification loss, and the device further includes:

[0130] A third determining unit is used to determine the category information corresponding to each newly added node in the word vector matrix in the matrix filling module;

[0131] a fourth determining unit, configured to determine, if the category information indicates that the newly added node is an entity category, an overall loss of the node based on the entity distillation loss and the newly added entity category classification loss;

[0132] a fifth determining unit, configured to determine, if the category information indicates that the newly added node is a relationship category, an overall loss of the node based on the relationship distillation loss and the newly added relationship category classification loss;

[0133] The sixth determination unit is used to determine the loss function of the target recognition model based on the overall loss of each newly added node.

[0134] Further, the step of obtaining a new category classification loss corresponding to the new entity relationship category relative to the basic recognition model includes:

[0135] Predicting data samples corresponding to newly added entity relationship categories relative to the basic recognition model using the target recognition model to obtain predicted entity relationship data;

[0136] Based on the predicted entity relationship data and the first entity relationship annotation data, a new category classification loss of entity relationship extraction corresponding to the new entity relationship category information is determined.

[0137] Optionally, obtaining the distillation loss when the base recognition model is distilled to the target recognition model includes:

[0138] Determining a first entity relationship classification probability distribution corresponding to the basic recognition model;

[0139] Obtaining a second entity relationship classification probability distribution in a training phase corresponding to the target recognition model;

[0140] A distillation loss is determined according to the first entity relationship classification probability distribution and the second entity relationship classification probability distribution.

[0141] Optionally, the device further comprises:

[0142] A third acquisition unit is used to acquire a newly added entity category and / or relationship category for the target recognition model;

[0143] An updating unit is used to update the target recognition model based on the target recognition model and the training samples corresponding to the newly added entity categories and / or relationship categories to obtain an updated target recognition model. The updated target recognition model can extract entity relationships from the text of the newly added entity categories and / or relationship categories and the original entity categories corresponding to the target recognition model.

[0144] It should be noted that the specific implementation of each unit and sub-unit in this embodiment can refer to the corresponding content in the previous text and will not be described in detail here.

[0145] In another embodiment of the present application, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, each step of the method for extracting entity relationships from text as described in any one of the above items is implemented.

[0146] In another embodiment of the present application, an electronic device is further provided, and the electronic device may include:

[0147] A memory, used to store applications and data generated by the operation of the applications;

[0148] A processor is used to execute the application program to implement the entity relationship extraction method of the text as described in any one of the above.

[0149] It should be noted that the specific implementation of the processor in this embodiment can refer to the corresponding content in the previous text and will not be described in detail here.

[0150] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0151] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0152] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0153] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for extracting entity relations from text, comprising: Get the target text to be processed; Calling a target recognition model to recognize the target text and determining an entity relationship recognition result corresponding to the target text; Wherein, the model structure corresponding to the target recognition model includes a knowledge distillation module and a category increment module, the knowledge distillation module is used to obtain the distillation loss when the basic recognition model is distilled to the target recognition model, and the category increment module is used to obtain the newly added category classification loss corresponding to the newly added entity relationship category relative to the basic recognition model, so that the target recognition model is determined by the basic recognition model, the distillation loss and the newly added category classification loss, and the recognition tasks corresponding to the target recognition model and the basic recognition model are the same; Among them, the distillation loss is obtained by processing the word vector matrix through the knowledge distillation module, the new category classification loss is obtained by processing the word vector matrix through the category increment module, and the word vector is a word vector matrix corresponding to the first entity relationship category annotation data and the second entity category annotation data generated by the matrix filling module in the model structure corresponding to the target recognition model. The horizontal axis and vertical axis of the word vector matrix are both entities, and the superposition between entities is the corresponding relationship between entities.

2. The method according to claim 1, further comprising: Acquire target training sample data, wherein the target training sample data is annotated with first entity relationship category annotation data for the newly added entity relationship category; Identify the target training sample data based on the basic recognition model to obtain second entity relationship category labeling data; Generate a word vector matrix corresponding to the first entity relationship category labeling data and the second entity category labeling data through a matrix filling module in the model structure corresponding to the target recognition model; Processing the word vector matrix through the knowledge distillation module to obtain a distillation loss; The word vector matrix is ​​processed by the category increment module to obtain a new category classification loss; The target recognition model is determined based on the distillation loss, the newly added category classification loss and the target training sample data.

3. The method according to claim 1, further comprising: Acquire first text data and entity relationship category information matching the first text data; Performing entity relationship annotation on the first text data based on the entity relationship category information to obtain a first training sample; Performing word segmentation on the first training sample, and performing word vector representation based on the word segmentation result to obtain a word vector matrix; Determine a loss function based on the word vector matrix; Training is performed based on the loss function and the first training sample to obtain a basic recognition model.

4. The method according to claim 3, wherein determining the loss function based on the word vector matrix comprises: Using the basic recognition model to predict each word in the word segmentation result, and obtaining predicted entities and relationships corresponding to each word; Based on the predicted entity, determining an entity cross entropy loss between the entity annotated with each of the words; Determining a relational cross entropy loss for a relation with each of the word tags based on the predicted relation; A loss function is determined according to the entity cross entropy loss and the relationship cross entropy loss.

5. The method according to claim 2, wherein the distillation loss includes entity distillation loss and relationship distillation loss, the newly added category classification loss includes newly added entity category classification loss and newly added relationship classification loss, and the method further comprises: Determine the category information corresponding to each newly added node in the word vector matrix in the matrix filling module; If the category information indicates that the newly added node is an entity category, determining the overall loss of the node based on the entity distillation loss and the newly added entity category classification loss; If the category information indicates that the newly added node is a relationship category, determining the overall loss of the node based on the relationship distillation loss and the newly added relationship category classification loss; Based on the overall loss of each newly added node, a loss function of the target recognition model is determined.

6. According to the method of claim 2, the step of obtaining the classification loss of the newly added category corresponding to the newly added entity relationship category relative to the basic recognition model comprises: Predicting data samples corresponding to newly added entity relationship categories relative to the basic recognition model using the target recognition model to obtain predicted entity relationship data; Based on the predicted entity relationship data and the first entity relationship annotation data, a new category classification loss for entity relationship extraction corresponding to the new entity relationship category information is determined.

7. The method according to claim 5, wherein: Obtaining the distillation loss when the base recognition model is distilled to the target recognition model, including: Determining a first entity relationship classification probability distribution corresponding to the basic recognition model; Obtaining a second entity relationship classification probability distribution in a training phase corresponding to the target recognition model; A distillation loss is determined according to the first entity relationship classification probability distribution and the second entity relationship classification probability distribution.

8. The method according to claim 1, further comprising: If newly added entity categories and / or relationship categories for the target recognition model are obtained; Based on the target recognition model and the training samples corresponding to the newly added entity categories and / or relationship categories, the target recognition model is updated to obtain an updated target recognition model. The updated target recognition model can extract entity relationships from the text of the newly added entity categories and / or relationship categories and the original entity categories corresponding to the target recognition model.

9. A text entity relationship extraction device, comprising: An acquisition unit, used for acquiring a target text to be processed; A calling unit, used for calling a target recognition model to recognize the target text and determine an entity relationship recognition result corresponding to the target text; Wherein, the model structure corresponding to the target recognition model includes a knowledge distillation module and a category increment module, the knowledge distillation module is used to obtain the distillation loss when the basic recognition model is distilled to the target recognition model, and the category increment module is used to obtain the newly added category classification loss corresponding to the newly added entity relationship category relative to the basic recognition model, so that the target recognition model is determined by the basic recognition model, the distillation loss and the newly added category classification loss, and the recognition tasks corresponding to the target recognition model and the basic recognition model are the same; Among them, the distillation loss is obtained by processing the word vector matrix through the knowledge distillation module, the new category classification loss is obtained by processing the word vector matrix through the category increment module, and the word vector is a word vector matrix corresponding to the first entity relationship category annotation data and the second entity category annotation data generated by the matrix filling module in the model structure corresponding to the target recognition model. The horizontal axis and vertical axis of the word vector matrix are both entities, and the superposition between entities is the corresponding relationship between entities.

10. An electronic device comprising: A memory, used to store applications and data generated by the operation of the applications; A processor is used to execute the application program to implement the entity relationship extraction method of text as described in any one of claims 1-8.

Images