Relationship extraction method, apparatus, device, storage medium and computer program product

Abstract

The present application relates to a relationship extraction method, apparatus, device, storage medium, and computer program product. The method comprises: obtaining relationship features corresponding to each instance in an instance set; the instance set includes instances of known relationships and instances of unknown relationships; based on the relationship features, identifying instances of unknown relationships from the instance set as instances to be labeled; the relationships between entities in the instances to be labeled are unknown; selecting a target instance from the instances to be labeled, and obtaining relationship information labeled for the target instance; the target instance contains more information than non-target instances other than the target instance in the instances to be labeled; training a relationship classifier based on the target instance after labeling the relationship information; the relationship information represents the relationship between entities in the target instance; and labeling the relationship between entities in the non-target instance based on the trained relationship classifier. The use of this method can save labor costs and time costs.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a relationship extraction method, apparatus, device, storage medium, and computer program product. Background Art

[0002] With the development of computer technology, relationship extraction has been widely used in many business scenarios. For example, relationship extraction can be applied to the construction of knowledge graphs. Knowledge graphs, also known as knowledge domain visualizations or knowledge domain mapping maps, are a series of various graphs that display the development process and structural relationships of knowledge. Knowledge graphs use visualization techniques to describe knowledge resources and their carriers, and to mine, analyze, construct, map, and display knowledge and its interconnections. Knowledge graphs are widely applicable in various business fields.

[0003] Traditionally, the relationships between entities have been manually labeled. However, this manual labeling of relationships between entities requires a lot of manpower and time. Summary of the Invention

[0004] Based on this, it is necessary to provide a relationship extraction method, device, equipment, storage medium and computer program product that can save manpower costs and time costs in response to the above technical problems.

[0005] A relationship extraction method, comprising:

[0006] Obtaining a relationship feature corresponding to each instance in an instance set; the instance set includes instances with known relationships and instances with unknown relationships;

[0007] According to the relationship features, identifying instances with unknown relationships from the instance set as instances to be labeled; the relationships between entities in the instances to be labeled are unknown;

[0008] Selecting a target instance from the instances to be labeled, and obtaining relationship information labeled for the target instance; the target instance contains more information than non-target instances other than the target instance in the instances to be labeled;

[0009] Training a relation classifier based on the target instance after annotating relation information; the relation information represents the relationship between entities in the target instance;

[0010] Based on the trained relationship classifier, the relationship between the entities in the non-target instance is labeled.

[0011] A relationship extraction device, comprising:

[0012] An acquisition module, configured to acquire a relationship feature corresponding to each instance in an instance set; the instance set includes instances with known relationships and instances with unknown relationships;

[0013] an identification module, configured to identify instances of unknown relationships from the instance set based on the relationship features, as instances to be labeled; wherein the relationship between entities in the instances to be labeled is unknown;

[0014] a selection module, configured to select a target instance from the instances to be labeled and obtain relationship information labeled for the target instance; the target instance contains more information than the non-target instances other than the target instance in the instances to be labeled;

[0015] A training module, configured to train a relation classifier based on the target instance annotated with relation information; the relation information represents the relationship between entities in the target instance;

[0016] The labeling module is used to label the relationship between the entities in the non-target instance based on the trained relationship classifier.

[0017] In one embodiment, each instance includes at least two entities; the acquisition module is also used to input each instance in the instance set into a trained language model respectively, so as to identify the relationship between the entities in each instance based on the trained language model, and obtain the relationship features corresponding to each instance.

[0018] In one embodiment, the training module is also used to obtain a language model to be trained; obtain a sample instance set; the sample instance set includes sample instances of customized known relationships; and optimize and train the language model to be trained through the sample instance set to obtain a trained language model.

[0019] In one embodiment, the training module is further used to input the sample instance set into the language model to be trained, obtain the predicted relationship features output by the language model to be trained for the sample instances in the sample instance set; determine the loss value according to the first objective function and the second objective function in the language model to be trained; the first objective function is used to measure the difference between the predicted relationship features and the known relationship features corresponding to the same sample instance; the known relationship features are used to characterize the known relationships of the sample instances; the second objective function is used to constrain the distribution of the predicted relationship features corresponding to each of the sample instances; and adjust the model parameters of the language model to be trained based on the loss value to continue iterative training until the iteration stop condition is met to obtain a trained language model.

[0020] In one embodiment, every two entities form an entity pair; the entity pair includes a head entity and a tail entity; the acquisition module is also used to, for each entity pair in each instance of the instance set, insert a head entity starting position identifier and a head entity ending position identifier before and after the head entity in the entity pair, and insert a tail entity starting position identifier and a tail entity ending position identifier before and after the tail entity in the entity pair, to obtain each preprocessed instance; input each preprocessed instance into the trained language model, and perform feature extraction in sequence through each hidden layer in the trained language model; for each preprocessed instance, obtain the first hidden layer feature vector corresponding to the head entity starting position identifier in the preprocessed instance and the second hidden layer feature vector corresponding to the tail entity starting position identifier output by the last hidden layer; perform vector splicing on the first hidden layer feature vector and the second hidden layer feature vector to obtain the relational feature corresponding to the instance.

[0021] In one embodiment, the recognition module is also used to cluster each instance in the instance set in the semantic space according to the semantic similarity between the relationship features; determine the local outlier factor corresponding to each clustered instance; and determine the instance whose local outlier factor is greater than or equal to a preset threshold as an instance of an unknown relationship.

[0022] In one embodiment, the identification module is also used to determine, for each instance after clustering, the reachability distance between the instance and each reference instance in the neighborhood; determine the first average reachable distance from the instance to each reference instance in the neighborhood based on each reachable distance; determine, for each reference instance, the second average reachable distance from the reference instance to each instance in the neighborhood of the reference instance; and determine the local outlier factor of the instance based on the first average reachable distance and the second average reachable distance corresponding to each reference instance.

[0023] In one embodiment, the selection module is further used to select target instances for the first round of adversarial training from the instances to be labeled, and obtain the relationship information labeled for the selected target instances; the first round is determined as this round, and the encoder and the discriminator are adversarially trained in this round based on the target instances labeled with relationship information, and the current instance to be labeled is encoded by the encoder after this round of adversarial training to obtain the relationship features corresponding to the current instance to be labeled; the discriminator after this round of adversarial training selects the target instance from the current instance to be labeled based on the relationship features corresponding to the current instance to be labeled, and obtains the relationship information labeled for the selected target instance; the remaining unlabeled instances to be labeled in the current instance to be labeled are used as the current instances to be labeled in the next round, and the next round is used as this round, and the step of performing adversarial training on the encoder and the discriminator in this round based on the target instances labeled with relationship information is returned to continue execution until the adversarial training is stopped when the iterative stopping condition is met; the target instances selected in each round of adversarial training are obtained, as well as the relationship information labeled for the selected target instances.

[0024] In one embodiment, the selection module is also used to determine the confidence corresponding to each current instance to be labeled based on the relationship features corresponding to the current instance to be labeled through the discriminator after this round of adversarial training; the confidence is used to characterize the richness of the information contained in the current instance to be labeled; based on the order of the confidence from high to low, a preset number of instances are selected from each current instance to be labeled as target instances, and the relationship information labeled for the selected target instances is obtained.

[0025] In one embodiment, the training module is also used to input the target instance after the relationship information is labeled into the relationship classifier to be trained; predict the predicted relationship information corresponding to the target instance through the relationship classifier to be trained; and adjust the model parameters of the relationship classifier to be trained based on the difference between the predicted relationship information and the labeled relationship information corresponding to the target instance.

[0026] In one embodiment, each instance in the instance set includes at least two entities; each two entities form an entity pair; the entity pair includes a head entity and a tail entity; and the apparatus further includes:

[0027] A generation module is used to determine a relationship triple based on the relationship between the entities in each instance in the instance set; the relationship triple includes a head entity, a tail entity, and the relationship between the head entity and the tail entity; based on the relationship triple, a knowledge graph is generated in the business scenario corresponding to the instance set.

[0028] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0029] Obtaining a relationship feature corresponding to each instance in an instance set; the instance set includes instances with known relationships and instances with unknown relationships;

[0030] According to the relationship features, identifying instances with unknown relationships from the instance set as instances to be labeled; the relationships between entities in the instances to be labeled are unknown;

[0031] Selecting a target instance from the instances to be labeled, and obtaining relationship information labeled for the target instance; the target instance contains more information than non-target instances other than the target instance in the instances to be labeled;

[0032] Training a relation classifier based on the target instance after annotating relation information; the relation information represents the relationship between entities in the target instance;

[0033] Based on the trained relationship classifier, the relationship between the entities in the non-target instance is labeled.

[0034] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0035] Obtaining a relationship feature corresponding to each instance in an instance set; the instance set includes instances with known relationships and instances with unknown relationships;

[0036] According to the relationship features, identifying instances with unknown relationships from the instance set as instances to be labeled; the relationships between entities in the instances to be labeled are unknown;

[0037] Selecting a target instance from the instances to be labeled, and obtaining relationship information labeled for the target instance; the target instance contains more information than non-target instances other than the target instance in the instances to be labeled;

[0038] Training a relation classifier based on the target instance after annotating relation information; the relation information represents the relationship between entities in the target instance;

[0039] Based on the trained relationship classifier, the relationship between the entities in the non-target instance is labeled.

[0040] A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the following steps:

[0041] Obtaining a relationship feature corresponding to each instance in an instance set; the instance set includes instances with known relationships and instances with unknown relationships;

[0042] According to the relationship features, identifying instances with unknown relationships from the instance set as instances to be labeled; the relationships between entities in the instances to be labeled are unknown;

[0043] Selecting a target instance from the instances to be labeled, and obtaining relationship information labeled for the target instance; the target instance contains more information than non-target instances other than the target instance in the instances to be labeled;

[0044] Training a relation classifier based on the target instance after annotating relation information; the relation information represents the relationship between entities in the target instance;

[0045] Based on the trained relationship classifier, the relationship between the entities in the non-target instance is labeled.

[0046] The aforementioned relationship extraction method, apparatus, device, storage medium, and computer program product obtain relationship features corresponding to each instance in a set of instances, where the set includes instances with known relationships and instances with unknown relationships. Based on the relationship features corresponding to each instance in the set, instances with unknown relationships can be identified from the set as unlabeled instances, where the relationships between entities in the unlabeled instances are unknown. A small number of target instances are selected from the unlabeled instances, where the amount of information contained in the target instances is greater than the amount of information contained in the non-target instances in the candidate instances other than the target instances. After manually annotating the relationship information for this small number of target instances, a relationship classifier can be trained based on this small number of target instances with the annotated relationship information. The relationship information represents the relationships between entities in the target instances. Because the selected small number of target instances contains a wealth of information, the trained relationship classifier has the ability to automatically annotate relationships. Furthermore, based on the trained relationship classifier, the relationships between entities in the remaining large number of non-target instances can be automatically annotated, eliminating the need to manually annotate relationships corresponding to a large number of instances and significantly saving labor and time costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is an application environment diagram of a relationship extraction method in one embodiment;

[0048] Figure 2 1 is a flow chart of a relationship extraction method in one embodiment;

[0049] Figure 3 A flowchart of the steps of selecting a target instance from instances to be annotated and annotating relationship information in one embodiment;

[0050] Figure 4 is a flowchart of a relationship extraction method in another embodiment;

[0051] Figure 5 is a flowchart of a relationship extraction method in another embodiment;

[0052] Figure 6 A schematic diagram showing the results of comparing the language model of the present application with a traditional feature extraction model in one embodiment;

[0053] Figure 7 Schematic diagram of the results of comparing the relationship extraction method of the present application with a traditional relationship extraction method in one embodiment;

[0054] Figure 8 is a structural block diagram of a relationship extraction device in one embodiment;

[0055] Figure 9 is a structural block diagram of a relationship extraction device in another embodiment;

[0056] Figure 10 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0058] The relationship extraction method provided in this application can be applied to Figure 1 In the application environment shown. The terminal 102 communicates with the server 104 through the network. The terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, portable wearable devices and vehicle-mounted terminals. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal 102 and the server 104 can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0059] The server 104 can obtain the relationship features corresponding to each instance in the instance set; the instance set includes instances with known relationships and instances with unknown relationships. The server 104 can identify instances with unknown relationships from the instance set based on the relationship features as instances to be labeled; the relationship between the entities in the instances to be labeled is unknown. The server 104 can select a target instance from the instances to be labeled and obtain the relationship information labeled for the target instance. The amount of information contained in the target instance is greater than the amount of information contained in the non-target instances other than the target instance in the instances to be labeled. The server 104 can train a relationship classifier based on the target instance after labeling the relationship information. The server 104 can use the relationship information to characterize the relationship between the entities in the target instance, and based on the trained relationship classifier, label the relationship between the entities in the non-target instance, and based on the relationship between the entities in each instance in the instance set, generate a knowledge graph for the business scenario corresponding to the instance set, and send the knowledge graph to the terminal 102 for display.

[0060] In one embodiment, Figure 2 As shown, a relationship extraction method is provided, which can be applied to the server 104 and can also be applied to the interaction process between the terminal and the server. Figure 1 Taking the server 104 in the example as an example, the following steps are included:

[0061] Step 202: Obtain the relationship feature corresponding to each instance in the instance set; the instance set includes instances with known relationships and instances with unknown relationships.

[0062] An instance set is a set of instances. A relationship feature is a feature used to characterize the relationships between entities in an instance. An instance with a known relationship is an instance where the relationships between entities are known. An instance with an unknown relationship is an instance where the relationships between entities are unknown.

[0063] Specifically, the server may obtain an instance set, which includes instances with known relationships and instances with unknown relationships. Furthermore, the server may obtain a relationship feature corresponding to each instance in the instance set.

[0064] In one embodiment, the server may input each instance in the instance set into a neural network model, and then the server may identify the relationship between the entities in each instance based on the neural network model to obtain the relationship features corresponding to each instance.

[0065] In one embodiment, the neural network model may include a language model, that is, a BERT (Bidirectional Encoder Representations from Transformers, autoencoding language model) model, and any other model with relational feature extraction capabilities.

[0066] Step 204 : Based on the relationship features, instances with unknown relationships are identified from the instance set as instances to be labeled; the relationships between entities in the instances to be labeled are unknown.

[0067] The instances to be labeled refer to instances where the relationships between entities are to be labeled.

[0068] Specifically, the server may identify instances of unknown relationships from the instance set according to the relationship features corresponding to each instance in the instance set, and use the identified instances of unknown relationships as instances to be labeled.

[0069] In one embodiment, in the semantic space, the relationship features corresponding to each instance in the instance set correspond to their own semantics. The server can cluster each instance in the instance set according to the semantic similarity between the relationship features corresponding to each instance in the instance set. Furthermore, the server can identify instances of unknown relationships from the instance set based on the clustering results, and use the identified instances of unknown relationships as instances to be labeled. Among them, semantic similarity is the similarity between the semantics of the relationship features corresponding to each instance. The first semantics corresponding to the instances of known relationships in the instance set are those that already exist in the semantic space; the first semantics are the semantics of the relationship features corresponding to the instances of known relationships; the second semantics corresponding to the instances of unknown relationships in the instance set are those that do not exist in the semantic space; the second semantics are the semantics of the relationship features corresponding to the instances of unknown relationships.

[0070] It can be understood that in the semantic space, for instances of known relationships, since the first semantics corresponding to the instances of known relationships in the instance set already exist in the semantic space, if the semantic similarity between the relationship features corresponding to each instance is high, the instances tend to be densely clustered to form independent clusters; if the semantic similarity between the relationship features corresponding to each instance is low, the instances tend to be discrete. For instances of unknown relationships, since the second semantics corresponding to the instances of unknown relationships in the instance set do not exist in the semantic space, these instances of unknown relationships are not mapped to any cluster. Based on this, the server can identify instances of unknown relationships from the instance set based on the clustering results, and use the identified instances of unknown relationships as instances to be labeled.

[0071] Step 206 , selecting a target instance from the instances to be labeled, and obtaining relationship information labeled for the target instance; the amount of information contained in the target instance is greater than the amount of information contained in non-target instances other than the target instance in the instances to be labeled.

[0072] The target instance is the instance that is the target of the annotation. The non-target instance is the instance that is not the target of the annotation. The relationship information is the information that represents the relationship between the entities in the target instance.

[0073] Specifically, the server can select a small number of target instances with the largest amount of information from all the instances to be annotated, and annotate the corresponding relationship information for the selected small number of target instances through manual annotation. Then, the server can obtain the annotated relationship information for the target instances.

[0074] In one embodiment, experts can leverage their extensive experience to directly select a small number of target instances containing the most information from the instances to be annotated. The server can obtain the target instances selected by the experts from the instances to be annotated and, through manual annotation, annotate the selected small number of target instances with corresponding relationship information. Furthermore, the server can obtain the annotated relationship information for the target instances.

[0075] In one embodiment, the server can input the instances to be annotated into a trained instance selection model. The trained instance selection model then selects a small number of target instances with the greatest information content from the instances to be annotated. Furthermore, the server manually annotates the selected small number of target instances with corresponding relationship information. The server can then obtain the annotated relationship information for the target instances.

[0076] Step 208 : training a relation classifier based on the target instance annotated with relation information; the relation information represents the relationship between entities in the target instance.

[0077] A classifier is a model used to automatically classify unknown data. For example, by inputting unknown data into a classifier, the classifier can automatically classify the unknown data. A relationship classifier is a classifier used to label the relationships between entities in an instance. For example, in a real application scenario, data sources are open and diverse, which means that the relationships between entities in most instances are unknown. By inputting instances with unknown relationships into a relationship classifier, the relationship classifier can automatically label the relationships between entities in the instance.

[0078] Specifically, the server may input the target instances with the relationship information annotated thereon into the relationship classifiers to be trained, respectively, so as to train the relationship classifiers to be trained based on the target instances with the relationship information annotated thereon.

[0079] In one embodiment, the server may input the target instances annotated with the relationship information into the relationship classifier to be trained. The server may adjust the model parameters of the relationship classifier to be trained based on the target instances annotated with the relationship information using the loss function in the relationship classifier to be trained.

[0080] Step 210 : Based on the trained relationship classifier, the relationship between entities in the non-target instance is labeled.

[0081] Specifically, the server may input a large number of non-target instances into a trained relational classifier, so as to automatically label the relationships between entities in these large number of non-target instances based on the trained relational classifier.

[0082] In the above-mentioned relationship extraction method, relationship features corresponding to each instance in a set of instances are obtained. The set of instances includes instances with known relationships and instances with unknown relationships. Based on the relationship features corresponding to each instance in the set of instances, instances with unknown relationships can be identified from the set of instances as unlabeled instances. The relationships between entities in the unlabeled instances are unknown. A small number of target instances are selected from the unlabeled instances. The target instances contain more information than the non-target instances in the candidate instances. After manually annotating the relationship information for this small number of target instances, a relationship classifier is trained based on this small number of annotated target instances. The relationship information represents the relationships between entities in the target instances. Because the selected small number of target instances contains rich information, the trained relationship classifier has the ability to automatically annotate relationships. Furthermore, based on the trained relationship classifier, the relationships between entities in the remaining large number of non-target instances can be automatically annotated. This avoids the need to manually annotate relationships corresponding to a large number of instances, significantly saving labor and time.

[0083] In one embodiment, each instance includes at least two entities; step 202, that is, the step of obtaining the relationship features corresponding to each instance in the instance set, specifically includes: inputting each instance in the instance set into the trained language model to identify the relationship between the entities in each instance based on the trained language model, and obtaining the relationship features corresponding to each instance.

[0084] Specifically, each instance includes at least two entities, each of which has a corresponding relationship. The server can input each instance in the instance set into a trained language model and identify the relationship between the entities in each instance based on the trained language model to obtain the relationship features corresponding to each instance.

[0085] In one embodiment, every two entities form an entity pair. For each entity pair in each instance of the instance set, the server may obtain a location identifier for each entity pair and input each instance after obtaining the location identifier into the trained language model to obtain a feature vector indicated by the location identifier of each entity pair. Based on the feature vector corresponding to each entity pair in each instance, the server may determine the relationship features corresponding to each instance. An entity pair includes two entities. The location identifier of the entity pair is used to indicate the position of the entity pair in the corresponding instance.

[0086] In the above embodiment, each instance in the instance set is input into a trained language model, and the relationship features corresponding to each instance are extracted through the trained language model, so that more accurate relationship features can be obtained.

[0087] In one embodiment, the trained language model is obtained through a language model training step, which includes: obtaining a language model to be trained; obtaining a set of sample instances; the set of sample instances includes sample instances of customized known relationships; and optimizing and training the language model to be trained through the set of sample instances to obtain a trained language model.

[0088] A sample instance set is a collection of sample instances. A sample instance is an instance used to train the language model to be trained. A customized sample instance with known relationships is a sample instance that is customized and annotated with relationship information for a specific business scenario.

[0089] Specifically, the server may obtain a language model to be trained and a set of sample instances. It should be noted that the set of sample instances includes custom sample instances with known relationships and sample instances with known relationships from public datasets. The server may then optimize and train the language model to be trained using the set of sample instances to obtain a trained language model.

[0090] In one embodiment, the language model to be trained may be a language model directly obtained by the server and that has undergone initial training, or a language model that is initially trained by the server based on sample instances with known relationships in a public dataset.

[0091] In one embodiment, the language model to be trained is obtained based on a traditional objective function. The server can further optimize and train the language model to be trained using a set of sample instances, the traditional objective function, and at least one custom objective function to obtain a trained language model.

[0092] In the above embodiment, by optimizing and training the language model to be trained through customized sample instances of known relationships, the trained language model can be made more suitable for the corresponding application scenario, and more accurate relationship features in the application scenario can be obtained through the trained language model.

[0093] In one embodiment, the steps of optimizing and training a language model to be trained by a set of sample instances to obtain a trained language model specifically include: inputting the set of sample instances into the language model to be trained, and obtaining the predicted relationship features output by the language model to be trained for the sample instances in the set of sample instances; determining a loss value based on a first objective function and a second objective function in the language model to be trained; the first objective function is used to measure the difference between the predicted relationship features and the known relationship features corresponding to the same sample instance; the known relationship features are used to characterize the known relationships of the sample instances; the second objective function is used to constrain the distribution of the predicted relationship features corresponding to each sample instance; and adjusting the model parameters of the language model to be trained based on the loss value to continue iterative training until an iteration stop condition is met to obtain a trained language model.

[0094] The predicted relational features are the relational features predicted by the language model to be trained based on each input sample instance, and the known relational features are the features that characterize the known relations of the sample instances.

[0095] Specifically, the server may input the acquired sample instance set into the language model to be trained. The server may predict the predicted relationship features corresponding to each sample instance based on the sample instances in the sample instance set through the language model to be trained. The language model to be trained is pre-constructed with a first objective function for measuring the difference between the predicted relationship features corresponding to the same sample instance and the known relationship features, and a second objective function for constraining the distribution of the predicted relationship features corresponding to each sample instance. The server may determine the loss value in the training process based on the first objective function and the second objective function in the language model to be trained, and adjust the model parameters of the language model to be trained based on the loss value to continue iterative training until the iteration stop condition is met to obtain a trained language model.

[0096] In one embodiment, the first objective function may be a traditional objective function, such as a cross entropy loss function, and the second objective function may be a custom objective function, such as a loss function in contrastive learning.

[0097] In the above embodiment, the loss value is determined together by the first objective function and the second objective function in the language model to be trained, and the model parameters of the language model to be trained are adjusted by the loss value, so as to improve the accuracy of the language model in extracting relational features.

[0098] In one embodiment, every two entities form an entity pair; the entity pair includes a head entity and a tail entity; each instance in the instance set is input into the trained language model respectively to identify the relationship between the entities in each instance based on the trained language model, and obtain the relationship features corresponding to each instance, including: for each entity pair in each instance in the instance set, inserting the head entity starting position identifier and the head entity ending position identifier before and after the head entity in the entity pair, respectively, and inserting the tail entity starting position identifier and the tail entity ending position identifier before and after the tail entity in the entity pair, respectively, to obtain each preprocessed instance; inputting each preprocessed instance into the trained language model, and performing feature extraction in turn through each hidden layer in the trained language model; for each preprocessed instance, obtaining the first hidden layer feature vector corresponding to the head entity starting position identifier in the preprocessed instance and the second hidden layer feature vector corresponding to the tail entity starting position identifier output by the last hidden layer; performing vector concatenation of the first hidden layer feature vector and the second hidden layer feature vector to obtain the relationship features corresponding to the instance.

[0099] Among them, the head entity is the entity at the starting position in the entity pair. The tail entity is the entity at the ending position in the entity pair. The head entity starting position identifier is an identifier used to mark the position of the first entity element in the head entity in the instance. The head entity ending position identifier is an identifier used to mark the position of the last entity element in the head entity in the instance. The tail entity starting position identifier is an identifier used to mark the position of the first entity element in the tail entity in the instance. The tail entity ending position identifier is an identifier used to mark the position of the last entity element in the tail entity in the instance. Entity elements are elements in each entity, and an entity includes at least one entity element. The hidden layer is the intermediate layer other than the input layer and the output layer in the trained language model. The first hidden layer feature vector is the feature vector corresponding to the head entity starting position identifier output by the last hidden layer. The second hidden layer feature vector is the feature vector corresponding to the tail entity starting position identifier output by the last hidden layer.

[0100] Specifically, for each entity pair in each instance of the instance set, the server may insert the head entity starting position identifier before the head entity in the entity pair, and insert the head entity ending position identifier after the head entity in the entity pair. At the same time, the server may also insert the tail entity starting position identifier before the tail entity in the entity pair, and insert the tail entity ending position identifier after the tail entity in the entity pair, to obtain each preprocessed instance. The trained language model includes multiple hidden layers, and the server may input each preprocessed instance into the trained language model, and perform feature extraction through each hidden layer in the trained language model in turn. For each preprocessed instance, the server may obtain the first hidden layer feature vector corresponding to the head entity starting position identifier in the preprocessed instance, and the second hidden layer feature vector corresponding to the tail entity starting position identifier, output by the last hidden layer, and perform vector splicing of the first hidden layer feature vector and the second hidden layer feature vector to obtain the relational feature corresponding to the instance.

[0101] In one embodiment, instance x is a word sequence, denoted as {w1,w2,...,w n}, where w1, w2, ..., w n Represents each word in the word sequence, and n represents the number of words in the word sequence. Assume that there are two entities in the word sequence, denoted as e h and e t , e h and e t can form an entity pair, then we can use the relation triple (e h ,r,e t ) to represent the entity pair e h and e t In addition, the head entity starting position identifier can be defined as <e h >, the end position identifier of the head entity is < / e h >, the tail entity starting position identifier is <e t >, the end position identifier of the tail entity is < / e t >, then the initialized instance x can be expressed as:

[0102] x=...,<e h >,w START(h) +1,...,w END(h) ,< / e h >,...,

[0103] <e t >,w START(t) +1,...,w END(t) ,< / e t >,...

[0104] Where START(h) means <e h > In the example, END(h) means < / e h > In the example, START(t) means <e t > In the example, END(t) means < / e t > Position in the instance.

[0105] The server can input the initialized instance x into the trained language model (BERT) to obtain the hidden layer feature vector corresponding to each word output by the last hidden layer in the trained language model. The specific formula can be illustrated as follows:

[0106] h1,...,h m =BERT(w1,w2,...,w m )

[0107] Among them, BERT (w1,w2,...,w m ) means inputting the initialized instance x into the trained language model (BERT), h1,...,h m Indicates that the last hidden layer outputs the hidden feature vector corresponding to each word.

[0108] The server can h >The corresponding first hidden layer feature vector h START(h) , and <e t >The corresponding second hidden layer feature vector h START(t) Perform vector splicing to obtain the relational feature vector h corresponding to instance x r (x), which can be specifically illustrated by the following formula:

[0109] h r (x)=[h START(h) ,h START(t) ]

[0110] In the above embodiment, by inserting the head entity start position identifier and the head entity end position identifier before and after the head entity in the entity pair, and inserting the tail entity start position identifier and the tail entity end position identifier before and after the tail entity in the entity pair, respectively, the first hidden layer feature vector corresponding to the head entity start position identifier and the second hidden layer feature vector corresponding to the tail entity start position identifier output by the last hidden layer are concatenated, thereby quickly and accurately obtaining the relationship features corresponding to each instance.

[0111] In one embodiment, the step of identifying instances of unknown relationships from the instance set based on the relationship features in step 204 specifically includes: clustering the instances in the instance set based on the semantic similarity between the relationship features in the semantic space; determining the local outlier factor corresponding to each clustered instance; and determining the instances whose local outlier factors are greater than or equal to a preset threshold as instances of unknown relationships.

[0112] Among them, the local outlier factor is an evaluation parameter used to determine whether an instance is an instance of an unknown relationship. If the local outlier factor of an instance is larger, it means that the instance is more likely to be an instance of an unknown relationship. Conversely, if the local outlier factor of an instance is smaller, it means that the instance is more likely to be an instance of a known relationship.

[0113] Specifically, the relationship features corresponding to each instance have their own semantic meaning. In the semantic space, the server can cluster the instances in the instance set based on the semantic similarity between the relationship features. For the clustered instances, the server can calculate the local outlier factor corresponding to each clustered instance and compare the local outlier factor with a preset threshold. Based on the comparison results, the server can determine that instances with a local outlier factor greater than or equal to the preset threshold are instances of unknown relationships.

[0114] In one embodiment, for the clustered instances, the server may calculate the local density corresponding to each clustered instance and, based on the local density corresponding to each instance, calculate the local outlier factor corresponding to each instance. The local density is the density of each instance in the corresponding local space.

[0115] In the above embodiment, by clustering the instances in the instance set according to the semantic similarity between the relationship features in the semantic space, and determining the instances of unknown relationships based on the local outlier factors corresponding to each clustered instance, the instances of unknown relationships can be quickly screened out from the instances in the instance set, thereby improving the screening efficiency of the instances of unknown relationships.

[0116] In one embodiment, the step of determining the local outlier factor corresponding to each instance after clustering includes: for each instance after clustering, determining the reachability distance between the instance and each reference instance in the neighborhood; based on each reachability distance, determining the first average reachable distance from the instance to each reference instance in the neighborhood; for each reference instance, determining the second average reachable distance from the reference instance to each instance in the neighborhood of the reference instance; and determining the local outlier factor of the instance based on the first average reachable distance and the second average reachable distance corresponding to each reference instance.

[0117] The reference instance is the instance in the domain, and the reachability distance is the Euclidean distance between the instance and the reference instance.

[0118] Specifically, for each instance after clustering, the server may determine the neighborhood of the instance, and determine the reachability distance between the instance and each reference instance in the neighborhood. The server may determine the first average reachable distance from the instance to each reference instance in the neighborhood based on each reachability distance and the number of reference instances in the neighborhood of the instance. For each reference instance, the server may determine the neighborhood of the reference instance, and determine the second average reachable distance from the reference instance to each instance in the neighborhood of the reference instance. Furthermore, the server may calculate the local outlier factor of the instance based on the first average reachable distance and the second average reachable distance corresponding to each reference instance.

[0119] In one embodiment, the server may compare the maximum distance within the neighborhood of the reference instance with the distance from the instance to any reference instance in the neighborhood, and select the largest distance as the reachability distance between the instance and the reference instance based on the comparison result. Specifically, the instance x may be defined as i The kth distance is d k (h r (x i ), define the reference instance x j The kth distance is d k (h r (x j ), which represents x j The distance to the kth nearest neighbor of x i and x j The accessibility distance rd k (h r (x i ),h r (x j )) can be calculated using the following formula:

[0120] rd k (h r (x i ),h r (x j ))=max{d k (h r (x j )),d(h r (x i ),h r (x j ))}

[0121] In one embodiment, the server may sum the reachability distances between the instance and each reference instance in the neighborhood of the instance, and calculate the first average reachability distance from the instance to each reference instance in the neighborhood based on the number of reference instances in the neighborhood of the instance. Specifically, for instance x i To the neighborhood N of this instancek (h r (x i )) Each reference instance x j The first average reachable distance can be calculated by the following formula:

[0122]

[0123] Understandably, instance x i To the neighborhood N of this instance k (h r (x i )), which is instance x i The kth distance d k (h r (x i ) within the scope of the .

[0124] In one embodiment, the server may sum the reachability distances between the reference instance and each instance in the neighborhood of the reference instance, and calculate the second average reachability distance from the reference instance to each instance in the neighborhood based on the number of instances in the neighborhood of the reference instance. Specifically, the reference instance x j Neighborhood N to the reference instance k (h r (x j ) in each instance x p The second average reachable distance den k (h r (x j )) can be calculated using the following formula:

[0125]

[0126] Understandably, instance x j To the neighborhood N of this instance k (h r (x j )), which is instance x j The kth distance d k (h r (x j ) within the scope of the .

[0127] In one embodiment, the server may sum the ratio of the second average reachable distance to the first average reachable distance in the neighborhood of the instance, and calculate the local outlier factor of the instance based on the ratio of the summation result to the number of reference instances in the neighborhood of the instance. Specifically, for instance x i Local outlier factor LOF k (h r (x i )) can be calculated using the following formula:

[0128]

[0129] In the above embodiment, for each clustered instance, a first average reachable distance from the instance to each reference instance in the neighborhood can be determined based on the reachability distance between the instance and each reference instance in the neighborhood. For each reference instance, a second average reachable distance from the reference instance to each instance in the reference instance's neighborhood can be determined. Based on the first average reachable distance and the second average reachable distance corresponding to each reference instance, the local outlier factor of each instance can be quickly calculated, improving the efficiency of the calculation of the local outlier factor.

[0130] In one embodiment, Figure 3 As shown, step 206, i.e., the step of selecting a target instance from the instances to be annotated and obtaining the relationship information annotated for the target instance, specifically includes:

[0131] Step 302: Select a target instance for the first round of adversarial training from the instances to be labeled, and obtain the relationship information labeled for the selected target instance.

[0132] Specifically, experts can select a small number of target instances with the largest amount of information from the instances to be labeled as training data for the first round of adversarial training. Then, the server can obtain the target instances selected from the instances to be labeled for the first round of adversarial training, and annotate the relationship information of the selected target instances through manual labeling. The server can obtain the relationship information annotated for the selected target instances.

[0133] In step 304, the first round is determined as the current round. In this round, the encoder and the discriminator are adversarially trained based on the target instance that has been labeled with relationship information. The encoder after this round of adversarial training encodes the current instance to be labeled to obtain the relationship features corresponding to the current instance to be labeled.

[0134] Specifically, the server may determine the first round of adversarial training as the current round and input the target instances labeled with relationship information as training data into the adversarial network consisting of an encoder and a discriminator. The server may perform adversarial training on the encoder and discriminator in this round based on the target instances labeled with relationship information. The server may then use the encoder after this round of adversarial training to encode the current instance to be labeled, and obtain the relationship features corresponding to the current instance to be labeled.

[0135] In step 306 , the discriminator after this round of adversarial training selects a target instance from the current instances to be labeled based on the relationship features corresponding to the current instances to be labeled, and obtains the relationship information labeled for the selected target instance.

[0136] Specifically, the server can input the relationship features corresponding to the current instance to be labeled into the discriminator after this round of adversarial training, so that the discriminator after this round of adversarial training can select the target instance from the current instance to be labeled based on the relationship features corresponding to the current instance to be labeled, and annotate the relationship information of the selected target instance through manual labeling. Then, the server can obtain the relationship information annotated for the selected target instance.

[0137] In one embodiment, the server may input the relational features corresponding to the current instance to be labeled into the discriminator after this round of adversarial training, and through the discriminator after this round of adversarial training, directly select the target instance from the current instance to be labeled based on the relational features corresponding to the current instance to be labeled, and use the target instance as the model output of the discriminator after this round of adversarial training.

[0138] In step 308, the remaining unlabeled instances in the current instances to be labeled are used as the current instances to be labeled in the next round, and the next round is used as the current round. The process returns to the step of performing adversarial training on the encoder and the discriminator based on the target instances labeled with relationship information in the current round to continue the execution until the adversarial training is stopped when the iterative stopping condition is met.

[0139] Specifically, the server can use the remaining unlabeled instances in the current to-be-labeled instances as the current to-be-labeled instances in the next round, and use the next round as the current round. The server can return to the step of performing adversarial training on the encoder and discriminator in the current round based on the target instances labeled with relationship information, and continue to perform adversarial training until the iterative stopping condition is met.

[0140] Step 310: Obtain target instances selected in each round of adversarial training and relationship information annotated for the selected target instances.

[0141] Specifically, during the iterative training of the adversarial network, the server can obtain the target instances selected in each round of adversarial training, and annotate the relationship information of the target instances selected in each round through manual labeling. Then, the server can obtain the relationship information annotated for the target instances selected in each round.

[0142] In one embodiment, the set of instances marked with relationship information is denoted as X L , the set of instances without labeled relation information is denoted as X U , then during the iterative training of the adversarial network, the encoder's loss function It can be expressed as the following formula:

[0143]

[0144] in, Indicates that instance x belongs to the set of instances annotated with relation information, E θ (x)) represents the relational features output by the encoder, D ψ (E θ (x)) represents the confidence output by the discriminator based on the relational features output by the encoder, It represents the expected probability that the instance x annotated with relation information is identified as the target instance. Indicates that instance x belongs to the set of instances without labeled relation information, It represents the expected probability that an instance x without labeled relation information is classified as a non-target instance.

[0145] In one embodiment, the loss function of the discriminator is It can be expressed as the following formula:

[0146]

[0147] in, represents the expected probability that the instance x marked with relational information is identified as a non-target instance, It represents the expected probability that the instance x without labeled relation information is identified as the target instance.

[0148] In the above embodiment, a small number of target instances are manually selected from the to-be-annotated instances for the first round of adversarial training by an expert, and relationship information annotated for the selected target instances is obtained. The first round is determined as the current round. In this round, the encoder and discriminator are adversarially trained based on the target instances already annotated with relationship information. The encoder after this round of adversarial training encodes the current to-be-annotated instance to obtain the relationship features corresponding to the current to-be-annotated instance. The discriminator after this round of adversarial training selects another small number of target instances from the current to-be-annotated instance based on the relationship features corresponding to the current to-be-annotated instance, and obtains relationship information annotated for the selected target instances. The remaining unannotated to-be-annotated instances in the current to-be-annotated instance are used as the current to-be-annotated instances in the next round, and the next round is used as the current round. The adversarial training is then continued by returning to the step of adversarially training the encoder and discriminator based on the target instances already annotated with relationship information in the current round, and the adversarial training is terminated when the iterative stopping condition is met. In this way, the small number of target instances with the largest amount of information and the relationship information annotated for this small number of target instances with the largest amount of information can be accurately obtained during each round of adversarial training.

[0149] In one embodiment, step 306, that is, selecting target instances from the current instances to be labeled based on the relationship features corresponding to the current instances to be labeled by the discriminator after this round of adversarial training, and obtaining the relationship information labeled for the selected target instances, includes: determining the confidence corresponding to each current instance to be labeled based on the relationship features corresponding to the current instances to be labeled by the discriminator after this round of adversarial training; the confidence is used to characterize the richness of the information contained in the current instance to be labeled; based on the order of confidence from high to low, selecting a preset number of instances from each current instance to be labeled as target instances, and obtaining the relationship information labeled for the selected target instances.

[0150] Specifically, the server may input the relationship features corresponding to the current instances to be labeled into the discriminator after this round of adversarial training. Then, the server may calculate the confidence corresponding to each current instance to be labeled based on the relationship features corresponding to the current instances to be labeled through the discriminator after this round of adversarial training. The server may sort the confidence corresponding to each current instance to be labeled according to the confidence corresponding to each current instance to be labeled. The server may select a preset number of instances from each current instance to be labeled as target instances based on the order of confidence from high to low, and annotate the relationship information of the selected target instances through manual annotation. Then, the server may obtain the relationship information annotated for the selected target instances.

[0151] In the above embodiment, the discriminator, after this round of adversarial training, quickly determines the confidence level of each current instance to be labeled based on the relationship features corresponding to the current instance to be labeled. Based on the order of confidence from high to low, a preset number of instances are accurately selected from the current instance to be labeled as target instances, and the relationship information annotated for this small number of target instances is obtained.

[0152] In one embodiment, step 208, which is the step of training a relational classifier based on a target instance with labeled relational information, specifically includes: inputting the target instance with labeled relational information into the relational classifier to be trained; predicting the predicted relational information corresponding to the target instance through the relational classifier to be trained; and adjusting the model parameters of the relational classifier to be trained based on the difference between the predicted relational information and the labeled relational information corresponding to the target instance.

[0153] The predicted relationship information is the relationship information predicted by the relationship classifier to be trained based on the target instance after the input labeled relationship information.

[0154] Specifically, the server can obtain a relationship classifier to be trained and input the target instance annotated with relationship information into the relationship classifier to be trained. Furthermore, the server can use the relationship classifier to be trained to predict the predicted relationship information corresponding to the target instance. The server can determine the difference between the predicted relationship information and the annotated relationship information corresponding to the target instance, and adjust the model parameters of the relationship classifier to be trained based on the difference between the predicted relationship information and the annotated relationship information corresponding to the target instance.

[0155] In the above embodiment, the relationship classifier to be trained can be used to quickly predict the predicted relationship information corresponding to the target instance, and based on the difference between the predicted relationship information and the labeled relationship information corresponding to the target instance, the model parameters of the relationship classifier to be trained can be accurately adjusted, thereby improving the classification and labeling capabilities of the relationship classifier.

[0156] In one embodiment, each instance in the instance set includes at least two entities; every two entities form an entity pair; the entity pair includes a head entity and a tail entity; the relationship extraction method also includes: determining a relationship triple based on the relationship between the entities in each instance in the instance set; the relationship triple includes a head entity, a tail entity, and the relationship between the head entity and the tail entity; based on the relationship triple, generating a knowledge graph for the business scenario corresponding to the instance set.

[0157] The relationship triplet refers to a set of three elements: a head entity, a tail entity, and a relationship between the head entity and the tail entity.

[0158] Specifically, the server can determine relationship triples based on the relationships between entities in each instance in the instance set. The server can then perform attribute normalization and entity alignment on the relationship triples to generate a knowledge graph for the business scenario corresponding to the instance set.

[0159] In the above embodiment, the relationship between the entities in each instance in the instance set can be used to quickly determine the relationship triples, and based on the relationship triples, the knowledge graph for the business scenario corresponding to the instance set can be quickly and accurately generated.

[0160] In one embodiment, Figure 4As shown, the server can preprocess each sample instance in the sample instance set, and input each preprocessed sample instance into the language model to be trained, so as to predict the predicted relationship features corresponding to each sample instance through the language model to be trained, and iteratively adjust the model parameters of the language model to be trained based on the difference between the relationship features corresponding to the sample instance and the predicted relationship features through back propagation until the iteration stop condition is reached to obtain the trained language model. In a real application scenario, each real instance in the real instance set is preprocessed, and each preprocessed real instance is input into the trained language model, so as to extract the relationship features corresponding to each real instance through the trained language model. Furthermore, the server can perform anomaly detection processing on each real instance in the semantic space based on the relationship features corresponding to the real instance through the anomaly detection algorithm to distinguish between instances of known relationships and instances of unknown relationships (it can be understood that Figure 4 The small gray circles are not mapped near any clusters. These small gray circles represent instances of unknown relationships, and the instances of unknown relationships are used as instances to be labeled. Experts can first select a small number of target instances from this large number of instances to be labeled for relationship information annotation. This small number of selected target instances is used as training data for this round of adversarial training. The server can train the encoder and discriminator through adversarial training based on this small number of target instances with labeled relationship information. The discriminator trained in this round can sort the instances to be labeled based on the relationship features corresponding to the instances to be labeled, so as to select a preset number of target instances with the largest amount of information from the remaining instances to be labeled. The experts then annotate the relationship information of the selected target instances. The server can use the selected target instances as training data for the next round of adversarial training to iteratively train the encoder and discriminator to obtain target instances obtained in each round of iterative training, as well as the relationship information annotated for the target instances. The server can train a relationship classifier based on the labeled target instances, and automatically annotate the remaining large-scale instances to be labeled through the trained relationship classifier.

[0161] like Figure 5 As shown, in one embodiment, a relationship extraction method is provided, which specifically includes the following steps:

[0162] Step 502: Obtain a language model to be trained; obtain a sample instance set; the sample instance set includes sample instances of user-defined known relationships.

[0163] Step 504: input the sample instance set into the language model to be trained, and obtain the predicted relationship features output by the language model to be trained for the sample instances in the sample instance set.

[0164] Step 506, determining the loss value based on the first objective function and the second objective function in the language model to be trained; the first objective function is used to measure the difference between the predicted relationship features and the known relationship features corresponding to the same sample instance; the known relationship features are used to characterize the known relationships of the sample instances; the second objective function is used to constrain the distribution of the predicted relationship features corresponding to each sample instance.

[0165] Step 508: Adjust the model parameters of the language model to be trained based on the loss value to continue iterative training until an iteration stop condition is met to obtain a trained language model.

[0166] Step 510: For each entity pair in each instance of the instance set, insert the head entity start position identifier and the head entity end position identifier before and after the head entity in the entity pair, and insert the tail entity start position identifier and the tail entity end position identifier before and after the tail entity in the entity pair, to obtain each preprocessed instance; the instance set includes instances of known relationships and instances of unknown relationships; each instance includes at least two entities; every two entities form an entity pair; the entity pair includes a head entity and a tail entity.

[0167] In step 512, each pre-processed instance is input into the trained language model, and is sequentially passed through each hidden layer in the trained language model for feature extraction.

[0168] Step 514: for each preprocessed instance, obtain the first hidden layer feature vector output by the last hidden layer corresponding to the head entity starting position identifier in the preprocessed instance, and the second hidden layer feature vector corresponding to the tail entity starting position identifier.

[0169] Step 516: perform vector concatenation on the first hidden layer feature vector and the second hidden layer feature vector to obtain the relational features corresponding to the instance.

[0170] Step 518: Cluster the instances in the instance set in the semantic space according to the semantic similarity between the relationship features.

[0171] Step 520 : For each clustered instance, determine the reachability distance between the instance and each reference instance in the neighborhood.

[0172] Step 522: Determine a first average reachability distance from the instance to each reference instance in the neighborhood based on each reachability distance.

[0173] Step 524 : For each reference instance, determine a second average reachable distance from the reference instance to each instance in the neighborhood of the reference instance.

[0174] Step 526 : Determine the local outlier factor of the instance based on the first average reachable distance and the second average reachable distance corresponding to each reference instance.

[0175] Step 528 : Determine the instances whose local outlier factors are greater than or equal to a preset threshold as instances of unknown relationships, and use them as instances to be labeled; the relationships between entities in the instances to be labeled are unknown.

[0176] Step 530: Select a target instance for the first round of adversarial training from the instances to be labeled, and obtain the relationship information for labeling the selected target instance; the relationship information represents the relationship between the entities in the target instance; the amount of information contained in the target instance is greater than the amount of information contained in the non-target instances other than the target instance in the instances to be labeled.

[0177] In step 532, the first round is determined as this round. In this round, the encoder and the discriminator are adversarially trained based on the target instance that has been labeled with relationship information. The encoder after this round of adversarial training encodes the current instance to be labeled to obtain the relationship features corresponding to the current instance to be labeled.

[0178] In step 534, the discriminator after this round of adversarial training determines the confidence level corresponding to each current instance to be labeled based on the relationship features corresponding to the current instance to be labeled. The confidence level is used to represent the richness of the information contained in the current instance to be labeled.

[0179] Step 536 : Based on the order of confidence from high to low, a preset number of instances are selected from the current instances to be labeled as target instances, and the labeled relationship information for the selected target instances is obtained.

[0180] In step 538, the remaining unlabeled instances in the current instances to be labeled are used as the current instances to be labeled in the next round, and the next round is used as the current round. The process returns to the step of performing adversarial training on the encoder and the discriminator in the current round based on the target instances labeled with relationship information to continue the execution until the adversarial training is stopped when the iterative stopping condition is met.

[0181] Step 540: Obtain target instances selected in each round of adversarial training and relationship information annotated for the selected target instances.

[0182] Step 542: Input the target instance with the labeled relationship information into the relationship classifier to be trained.

[0183] Step 544: predict the predicted relationship information corresponding to the target instance through the relationship classifier to be trained.

[0184] Step 546 : Adjust the model parameters of the relationship classifier to be trained based on the difference between the predicted relationship information and the labeled relationship information corresponding to the target instance.

[0185] Step 548 : Based on the trained relationship classifier, the relationship between entities in the non-target instance is labeled.

[0186] Step 550 : Determine a relationship triple based on the relationship between entities in each instance in the instance set; the relationship triple includes a head entity, a tail entity, and the relationship between the head entity and the tail entity.

[0187] Step 552: Generate a knowledge graph for the business scenario corresponding to the instance set based on the relationship triples.

[0188] The present application also provides an application scenario, which applies the above-mentioned relationship extraction method. Specifically, the relationship extraction method can be applied to the relationship extraction scenario under the text security review business. The server can use the text as an instance and obtain the language model to be trained; obtain a sample instance set; the sample instance set includes sample instances of customized known relationships. The sample instance set is input into the language model to be trained, and the predicted relationship features output by the language model to be trained for the sample instances in the sample instance set are obtained. The loss value is determined according to the first objective function and the second objective function in the language model to be trained; the first objective function is used to measure the difference between the predicted relationship features and the known relationship features corresponding to the same sample instance; the known relationship features are used to characterize the known relationships of the sample instances; the second objective function is used to constrain the distribution of the predicted relationship features corresponding to each sample instance. The model parameters of the language model to be trained are adjusted based on the loss value to continue iterative training until the iteration stop condition is met to obtain a trained language model.

[0189] For each entity pair in each instance of the instance set, the server may insert a head entity start position identifier and a head entity end position identifier before and after the head entity in the entity pair, and insert a tail entity start position identifier and a tail entity end position identifier before and after the tail entity in the entity pair, to obtain each preprocessed instance; the instance set includes instances of known relationships and instances of unknown relationships; each instance includes at least two entities; every two entities form an entity pair; the entity pair includes a head entity and a tail entity. Each preprocessed instance is input into a trained language model and sequentially passed through each hidden layer in the trained language model for feature extraction. For each preprocessed instance, the first hidden layer feature vector corresponding to the head entity start position identifier in the preprocessed instance and the second hidden layer feature vector corresponding to the tail entity start position identifier are obtained. The first hidden layer feature vector and the second hidden layer feature vector are vector-concatenated to obtain the relationship feature corresponding to the instance.

[0190] In the semantic space, the server can cluster each instance in the instance set according to the semantic similarity between the relationship features. For each clustered instance, the reachability distance between the instance and each reference instance in the neighborhood is determined. Based on each reachability distance, the first average reachable distance between the instance and each reference instance in the neighborhood is determined. For each reference instance, the second average reachable distance from the reference instance to each instance in the neighborhood of the reference instance is determined. Based on the first average reachable distance and the second average reachable distance corresponding to each reference instance, the local outlier factor of the instance is determined. Instances with a local outlier factor greater than or equal to a preset threshold are determined as instances of unknown relationships and are used as instances to be labeled; the relationship between entities in the instances to be labeled is unknown.

[0191] The server can select target instances for the first round of adversarial training from the instances to be labeled, and obtain the relationship information annotated for the selected target instances; the relationship information represents the relationship between the entities in the target instances. The first round is determined as this round, and the encoder and the discriminator are adversarially trained in this round based on the target instances that have been annotated with relationship information, and the encoder after this round of adversarial training is used to encode the current instance to be labeled to obtain the relationship features corresponding to the current instance to be labeled; the amount of information contained in the target instance is greater than the amount of information contained in the non-target instances other than the target instance in the instance to be labeled. The discriminator after this round of adversarial training determines the confidence corresponding to each current instance to be labeled based on the relationship features corresponding to the current instance to be labeled; the confidence is used to represent the richness of the information contained in the current instance to be labeled. Based on the order of confidence from high to low, a preset number of instances are selected from each current instance to be labeled as target instances, and the relationship information annotated for the selected target instances is obtained. The remaining unlabeled instances in the current set of instances to be labeled are used as the current instances to be labeled in the next round. The next round is used as the current round. The adversarial training of the encoder and discriminator based on the target instances labeled with relationship information is continued until the iterative stopping condition is met. The target instances selected in each round of adversarial training are obtained, along with the relationship information annotated for the selected target instances.

[0192] The server can input the target instance with the annotated relationship information into the relationship classifier to be trained, and predict the predicted relationship information corresponding to the target instance through the relationship classifier to be trained. Based on the difference between the predicted relationship information and the annotated relationship information corresponding to the target instance, the model parameters of the relationship classifier to be trained are adjusted. Based on the trained relationship classifier, the relationship between the entities in the non-target instance is annotated. According to the relationship between the entities in each instance in the instance set, the relationship triple is determined; the relationship triple includes the head entity, the tail entity, and the relationship between the head entity and the tail entity. Based on the relationship triple, a knowledge graph is generated for the text security review business scenario.

[0193] This application also provides an application scenario that utilizes the aforementioned relationship extraction method. Specifically, the relationship extraction method can be applied to a relationship extraction scenario within a product recommendation business. A server can use product information as an instance and obtain the relationship features corresponding to each instance in a set of instances; the set of instances includes instances with known relationships and instances with unknown relationships. Based on the relationship features, instances with unknown relationships are identified from the set of instances as instances to be labeled; the relationships between entities in the instances to be labeled are unknown. A target instance is selected from the instances to be labeled, and relationship information annotated for the target instance is obtained; the target instance contains more information than the non-target instances in the instances to be labeled. A relationship classifier is trained based on the target instance with the annotated relationship information; the relationship information represents the relationships between entities in the target instance. Based on the trained relationship classifier, the relationships between entities in the non-target instances are labeled. Furthermore, the server can determine relationship triples based on the relationships between entities in each instance in the set of instances; the relationship triples include a head entity, a tail entity, and the relationship between the head and tail entities. Based on the relationship triples, a knowledge graph is generated for the product recommendation business scenario.

[0194] In addition, the relationship extraction method of the present application can also be applied to any of the following scenarios, such as the relationship extraction scenario under the game business, the relationship extraction scenario under the comment barrage security review business, and the relationship extraction scenario under the medical platform.

[0195] Figure 6 The results are obtained by comparing the language model of this application with traditional feature extraction models (including convolutional networks, recurrent neural networks, long short-term memory networks, transformer networks and traditional speech models) under the same data set. Figure 6 It can be seen that the accuracy (92.3), recall rate (84.4), and score value (86.8) calculated based on the accuracy and recall rate corresponding to the language model of the present application are all better than the accuracy, recall rate, and score value calculated based on the accuracy and recall rate corresponding to the traditional feature extraction model. For example, the accuracy (76.8), recall rate (71.3), and score value (73.9) calculated based on the accuracy and recall rate corresponding to the traditional speech model.

[0196] Figure 7 The results are obtained by comparing the relation extraction method of this application with the traditional relation extraction method (including solution 1, solution 2 and solution 3) on three existing datasets (i.e., dataset 1 (FewRel (Noi)), dataset 2 (NYT+FB (Noi)) and dataset 3 (FewRel2.0 (Noi))). Figure 7 It can be seen that the accuracy (75.7), recall (86.7), score calculated based on accuracy and recall (80.8), uniformity (89.0), completeness (91.4), harmonic mean of uniformity and completeness (90.2), and adjusted Rand coefficient (71.3) corresponding to the relationship extraction method of the present application are all better than the accuracy, recall, score calculated based on accuracy and recall, uniformity, completeness, harmonic mean of uniformity and completeness, and adjusted Rand coefficient corresponding to the traditional relationship extraction method. For example, under data set 1, the accuracy (25.8), recall (70.7), score calculated based on accuracy and recall (37.8), uniformity (51.5), completeness (77.3), harmonic mean of uniformity and completeness (61.8), and adjusted Rand coefficient (24.4) corresponding to scheme 1.

[0197] It should be understood that, although the various steps in the flow chart of the above-mentioned embodiments are shown in sequence, these steps are not necessarily performed in sequence. Unless clearly stated herein, the execution of these steps does not have strict order restrictions, and these steps can be performed in other sequences. Moreover, at least a portion of the steps in the above-mentioned embodiments may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0198] In one embodiment, Figure 8 As shown, a relationship extraction device 800 is provided. The device can be a software module or a hardware module, or a combination of the two to form a part of a computer device. The device specifically includes:

[0199] The acquisition module 801 is used to acquire the relationship feature corresponding to each instance in the instance set; the instance set includes instances with known relationships and instances with unknown relationships.

[0200] The identification module 802 is used to identify instances with unknown relationships from the instance set according to relationship features, as instances to be labeled; the relationship between entities in the instances to be labeled is unknown.

[0201] The selection module 803 is used to select a target instance from the instances to be labeled and obtain the relationship information labeled for the target instance; the target instance contains more information than the non-target instances in the instances to be labeled except the target instance.

[0202] The training module 804 is used to train a relation classifier based on the target instance annotated with relation information; the relation information represents the relationship between entities in the target instance.

[0203] The labeling module 805 is used to label the relationship between entities in the non-target instance based on the trained relationship classifier.

[0204] In one embodiment, each instance includes at least two entities; the acquisition module 801 is also used to input each instance in the instance set into the trained language model to identify the relationship between the entities in each instance based on the trained language model, and obtain the relationship features corresponding to each instance.

[0205] In one embodiment, the training module 804 is further used to obtain a language model to be trained; obtain a sample instance set; the sample instance set includes sample instances of customized known relationships; and optimize and train the language model to be trained through the sample instance set to obtain a trained language model.

[0206] In one embodiment, the training module 804 is further used to input the sample instance set into the language model to be trained, obtain the predicted relationship features output by the language model to be trained for the sample instances in the sample instance set; determine the loss value according to the first objective function and the second objective function in the language model to be trained; the first objective function is used to measure the difference between the predicted relationship features and the known relationship features corresponding to the same sample instance; the known relationship features are used to characterize the known relationships of the sample instances; the second objective function is used to constrain the distribution of the predicted relationship features corresponding to each sample instance; and adjust the model parameters of the language model to be trained based on the loss value to continue iterative training until the iteration stop condition is met to obtain a trained language model.

[0207] In one embodiment, every two entities form an entity pair; the entity pair includes a head entity and a tail entity; the acquisition module 801 is also used to, for each entity pair in each instance of the instance set, insert a head entity starting position identifier and a head entity ending position identifier before and after the head entity in the entity pair, and insert a tail entity starting position identifier and a tail entity ending position identifier before and after the tail entity in the entity pair, to obtain each preprocessed instance; input each preprocessed instance into the trained language model, and perform feature extraction in sequence through each hidden layer in the trained language model; for each preprocessed instance, obtain the first hidden layer feature vector corresponding to the head entity starting position identifier in the preprocessed instance, and the second hidden layer feature vector corresponding to the tail entity starting position identifier, output by the last hidden layer; perform vector concatenation on the first hidden layer feature vector and the second hidden layer feature vector to obtain the relational feature corresponding to the instance.

[0208] In one embodiment, the identification module 802 is also used to cluster each instance in the instance set in the semantic space according to the semantic similarity between the relationship features; determine the local outlier factor corresponding to each clustered instance; and determine the instances whose local outlier factor is greater than or equal to a preset threshold as instances of unknown relationships.

[0209] In one embodiment, the identification module 802 is also used to determine, for each instance after clustering, the reachability distance between the instance and each reference instance in the neighborhood; determine the first average reachable distance from the instance to each reference instance in the neighborhood based on each reachability distance; determine, for each reference instance, the second average reachable distance from the reference instance to each instance in the neighborhood of the reference instance; and determine the local outlier factor of the instance based on the first average reachable distance and the second average reachable distance corresponding to each reference instance.

[0210] In one embodiment, the selection module 803 is also used to select target instances for the first round of adversarial training from the instances to be labeled, and obtain the relationship information labeled for the selected target instances; the first round is determined as this round, and the encoder and the discriminator are adversarially trained in this round based on the target instances labeled with relationship information, and the current instance to be labeled is encoded by the encoder after this round of adversarial training to obtain the relationship features corresponding to the current instance to be labeled; the discriminator after this round of adversarial training selects the target instance from the current instance to be labeled based on the relationship features corresponding to the current instance to be labeled, and obtains the relationship information labeled for the selected target instance; the remaining unlabeled instances to be labeled in the current instance to be labeled are used as the current instances to be labeled in the next round, and the next round is used as this round, and the step of performing adversarial training on the encoder and the discriminator in this round based on the target instances labeled with relationship information is returned to continue execution until the adversarial training is stopped when the iterative stopping condition is met; the target instances selected in each round of adversarial training are obtained, as well as the relationship information labeled for the selected target instances.

[0211] In one embodiment, the selection module 803 is also used to determine the confidence corresponding to each current instance to be labeled based on the relationship features corresponding to the current instance to be labeled through the discriminator after this round of adversarial training; the confidence is used to characterize the richness of the information contained in the current instance to be labeled; based on the order of confidence from high to low, a preset number of instances are selected from each current instance to be labeled as target instances, and the relationship information labeled for the selected target instances is obtained.

[0212] In one embodiment, the training module 804 is also used to input the target instance with labeled relationship information into the relationship classifier to be trained; predict the predicted relationship information corresponding to the target instance through the relationship classifier to be trained; and adjust the model parameters of the relationship classifier to be trained based on the difference between the predicted relationship information and the labeled relationship information corresponding to the target instance.

[0213] refer to Figure 9 In one embodiment, each instance in the instance set includes at least two entities; each two entities form an entity pair; the entity pair includes a head entity and a tail entity; the relationship extraction device 800 further includes:

[0214] Generation module 806 is used to determine the relationship triple based on the relationship between the entities in each instance in the instance set; the relationship triple includes the head entity, the tail entity, and the relationship between the head entity and the tail entity; based on the relationship triple, a knowledge graph is generated in the business scenario corresponding to the instance set.

[0215] The above-mentioned relationship extraction device obtains relationship features corresponding to each instance in an instance set, where the instance set includes instances with known relationships and instances with unknown relationships. Based on the relationship features corresponding to each instance in the instance set, instances with unknown relationships can be identified from the instance set as instances to be labeled, where the relationships between entities in the instances to be labeled are unknown. A small number of target instances are selected from the instances to be labeled, where the amount of information contained in the target instances is greater than the amount of information contained in the non-target instances in the candidate instances other than the target instances. After manually labeling the relationship information for this small number of target instances, a relationship classifier can be trained based on this small number of target instances with the labeled relationship information. The relationship information represents the relationships between entities in the target instances. Because the selected small number of target instances contains a wealth of information, the trained relationship classifier has the ability to automatically label relationships. Furthermore, based on the trained relationship classifier, the relationships between entities in the remaining large number of non-target instances can be automatically labeled, avoiding the need to manually label the relationships corresponding to a large number of instances, significantly saving labor and time costs.

[0216] For the specific definition of the relationship extraction device, please refer to the definition of the relationship extraction method above, which will not be repeated here. The various modules in the above-mentioned relationship extraction device can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0217] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 10 As shown. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements a relationship extraction method.

[0218] Those skilled in the art will understand that Figure 10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0219] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0220] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0221] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0222] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0223] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0224] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A relationship extraction method, characterized in that: The method comprises: Obtaining a relational feature corresponding to each instance in the instance set obtained by identifying the relationship between entities in each instance in the instance set using a trained language model; the instance set includes instances with known relationships and instances with unknown relationships; the relational feature is a feature used to characterize the relationship between entities in the instance; the trained language model is obtained by optimizing training using custom sample instances of known relationships; According to the relationship features, identifying instances with unknown relationships from the instance set as instances to be labeled; the relationships between entities in the instances to be labeled are unknown; Obtain target instances selected during each round of adversarial training, and relationship information annotated for the selected target instances; the target instances contain more information than the non-target instances other than the target instances in the instances to be annotated; the target instances selected during each round of adversarial training are a preset number of instances selected from the current instances to be annotated based on the order of confidence from high to low, and the confidence is used to characterize the richness of the information contained in the current instances to be annotated; the confidence is determined by the discriminator after this round of adversarial training and based on the relationship features obtained by encoding the current instances to be annotated using the encoder after this round of adversarial training; the encoder is used to encode the current instances to be annotated to obtain the relationship features corresponding to the current instances to be annotated; the discriminator is used to determine the confidence corresponding to each current instance to be annotated based on the relationship features corresponding to the current instances to be annotated; Training a relation classifier based on the target instance after annotating relation information; the relation information represents the relationship between entities in the target instance; Based on the trained relationship classifier, the relationship between the entities in the non-target instance is labeled.

2. The method according to claim 1, characterized in that Each of the instances includes at least two entities; The acquiring of the relationship feature corresponding to each instance in the instance set obtained by identifying the relationship between entities in each instance in the instance set through the trained language model includes: Each instance in the instance set is input into the trained language model respectively, so as to identify the relationship between entities in each instance based on the trained language model, and obtain the relationship feature corresponding to each instance.

3. The method according to claim 2, characterized in that The trained language model is obtained through a language model training step, and the language model training step includes: Get the language model to be trained; Acquire a sample instance set; the sample instance set includes sample instances of user-defined known relationships; The language model to be trained is optimized and trained by the sample instance set to obtain a trained language model.

4. The method according to claim 3, characterized in that The optimizing and training the language model to be trained by using the sample instance set to obtain a trained language model includes: Inputting the sample instance set into a language model to be trained, and obtaining prediction relationship features output by the language model to be trained for the sample instances in the sample instance set; Determining a loss value based on a first objective function and a second objective function in the language model to be trained; the first objective function is used to measure the difference between the predicted relationship feature and the known relationship feature corresponding to the same sample instance; the known relationship feature is used to characterize the known relationship of the sample instance; and the second objective function is used to constrain the distribution of the predicted relationship feature corresponding to each sample instance; The model parameters of the language model to be trained are adjusted based on the loss value to continue iterative training until an iteration stopping condition is met to obtain a trained language model.

5. The method according to claim 2, characterized in that Every two entities form an entity pair; the entity pair includes a head entity and a tail entity; inputting each instance in the instance set into the trained language model to identify the relationship between the entities in each instance based on the trained language model, and obtaining the relationship feature corresponding to each instance, including: For each entity pair in each instance of the instance set, inserting a head entity start position identifier and a head entity end position identifier before and after the head entity in the entity pair, and inserting a tail entity start position identifier and a tail entity end position identifier before and after the tail entity in the entity pair, to obtain each preprocessed instance; Input each preprocessed instance into a trained language model, and sequentially pass through each hidden layer in the trained language model for feature extraction; For each preprocessed instance, obtaining a first hidden layer feature vector output by the last hidden layer corresponding to the head entity starting position identifier in the preprocessed instance and a second hidden layer feature vector corresponding to the tail entity starting position identifier; The first hidden layer feature vector and the second hidden layer feature vector are concatenated to obtain a relational feature corresponding to the instance.

6. The method according to claim 1, characterized in that The step of identifying instances of unknown relationships from a set of instances based on the relationship features includes: In the semantic space, clustering the instances in the instance set according to the semantic similarity between the relationship features; Determine the local outlier factor corresponding to each instance after clustering; The instances whose local outlier factors are greater than or equal to a preset threshold are determined as instances of unknown relationships.

7. The method according to claim 6, characterized in that The local outlier factor corresponding to each clustered instance is determined, including: For each clustered instance, determine the reachability distance between the instance and each reference instance in the neighborhood; Determining, based on each of the reachability distances, a first average reachability distance from the instance to each reference instance in the neighborhood; For each reference instance, determining a second average reachable distance from the reference instance to each instance in a neighborhood of the reference instance; The local outlier factor of the instance is determined according to the first average reachable distance and the second average reachable distance corresponding to each of the reference instances.

8. The method according to claim 1, characterized in that The method further comprises: Selecting a target instance for a first round of adversarial training from the instances to be labeled; The first round is determined as this round. In this round, the encoder and the discriminator are adversarially trained based on the target instance that has been labeled with relationship information. The encoder after this round of adversarial training encodes the current instance to be labeled to obtain the relationship feature corresponding to the current instance to be labeled. The discriminator after this round of adversarial training selects a target instance from the current instances to be labeled based on the relationship features corresponding to the current instances to be labeled, and obtains the relationship information labeled for the selected target instance; The remaining unlabeled instances to be labeled in the current instances to be labeled are used as the current instances to be labeled in the next round, and the next round is used as the current round. The step of performing adversarial training on the encoder and the discriminator in the current round based on the target instances labeled with relationship information is returned to continue execution until the adversarial training is stopped when the iterative stopping condition is met.

9. The method according to claim 1, characterized in that The training of a relation classifier based on the target instance after labeling the relation information includes: Inputting the target instance after labeling the relationship information into the relationship classifier to be trained; Predicting predicted relationship information corresponding to the target instance by using the relationship classifier to be trained; Based on the difference between the predicted relationship information and the labeled relationship information corresponding to the target instance, the model parameters of the relationship classifier to be trained are adjusted.

10. The method according to any one of claims 1 to 9, characterized in that Each instance in the instance set includes at least two entities; each two entities form an entity pair; the entity pair includes a head entity and a tail entity; the method further includes: Determine a relationship triple according to the relationship between entities in each instance in the instance set; the relationship triple includes a head entity, a tail entity, and the relationship between the head entity and the tail entity; Based on the relationship triples, a knowledge graph is generated for the business scenario corresponding to the instance set.

11. A relationship extraction device, characterized in that: The device comprises: an acquisition module, configured to acquire a relational feature corresponding to each instance in the instance set, obtained by identifying the relationship between entities in each instance in the instance set using a trained language model; the instance set includes instances with known relationships and instances with unknown relationships; the relational feature is a feature used to characterize the relationship between entities in the instance; the trained language model is obtained by optimizing training using custom sample instances of known relationships; an identification module, configured to identify instances of unknown relationships from the instance set based on the relationship features, as instances to be labeled; wherein the relationship between entities in the instances to be labeled is unknown; A selection module is used to obtain the target instance selected in each round of adversarial training and the relationship information annotated for the selected target instance; the amount of information contained in the target instance is greater than the amount of information contained in the non-target instance other than the target instance in the instance to be annotated; the target instance selected in each round of adversarial training is a preset number of instances selected from each current instance to be annotated based on the order of confidence from high to low, and the confidence is used to characterize the richness of the amount of information contained in the current instance to be annotated; the confidence is determined by the discriminator after this round of adversarial training and based on the relationship features obtained by encoding the current instance to be annotated using the encoder after this round of adversarial training; the encoder is used to encode the current instance to be annotated to obtain the relationship features corresponding to the current instance to be annotated; the discriminator is used to determine the confidence corresponding to each current instance to be annotated based on the relationship features corresponding to the current instance to be annotated; A training module, configured to train a relation classifier based on the target instance annotated with relation information; the relation information represents the relationship between entities in the target instance; The labeling module is used to label the relationship between the entities in the non-target instance based on the trained relationship classifier.

12. The device according to claim 11, characterized in that Each of the instances includes at least two entities; the acquisition module is also used to input each instance in the instance set into the trained language model respectively, so as to identify the relationship between the entities in each instance based on the trained language model, and obtain the relationship features corresponding to each instance.

13. The device according to claim 12, characterized in that The training module is also used to obtain a language model to be trained; obtain a sample instance set; the sample instance set includes sample instances of customized known relationships; and optimize and train the language model to be trained through the sample instance set to obtain a trained language model.

14. The device according to claim 13, characterized in that The training module is further configured to input the sample instance set into the language model to be trained, and obtain the predicted relationship features output by the language model to be trained for the sample instances in the sample instance set; Determining a loss value based on a first objective function and a second objective function in the language model to be trained; the first objective function is used to measure the difference between the predicted relationship feature and the known relationship feature corresponding to the same sample instance; the known relationship feature is used to characterize the known relationship of the sample instance; and the second objective function is used to constrain the distribution of the predicted relationship feature corresponding to each sample instance; The model parameters of the language model to be trained are adjusted based on the loss value to continue iterative training until an iteration stopping condition is met to obtain a trained language model.

15. The device according to claim 12, characterized in that Every two entities form an entity pair; the entity pair includes a head entity and a tail entity; the acquisition module is further configured to, for each entity pair in each instance of the instance set, insert a head entity start position identifier and a head entity end position identifier before and after the head entity in the entity pair, and insert a tail entity start position identifier and a tail entity end position identifier before and after the tail entity in the entity pair, to obtain each preprocessed instance; Input each preprocessed instance into a trained language model, and sequentially pass through each hidden layer in the trained language model for feature extraction; For each preprocessed instance, obtain the first hidden layer feature vector corresponding to the starting position identifier of the head entity in the preprocessed instance, and the second hidden layer feature vector corresponding to the starting position identifier of the tail entity, output by the last hidden layer; vector concatenate the first hidden layer feature vector and the second hidden layer feature vector to obtain the relationship feature corresponding to the instance.

16. The device according to claim 11, characterized in that The recognition module is also used to cluster each instance in the instance set in the semantic space according to the semantic similarity between the relationship features; determine the local outlier factor corresponding to each clustered instance; and determine the instance whose local outlier factor is greater than or equal to a preset threshold as an instance of an unknown relationship.

17. The device according to claim 16, characterized in that The identification module is further configured to determine, for each clustered instance, a reachability distance between the instance and each reference instance in the neighborhood; and determine, based on each reachability distance, a first average reachability distance between the instance and each reference instance in the neighborhood; For each of the reference instances, determine a second average reachable distance from the reference instance to each instance in the neighborhood of the reference instance; and determine a local outlier factor of the instance based on the first average reachable distance and the second average reachable distance corresponding to each of the reference instances.

18. The device according to claim 11, characterized in that The selection module is also used to select target instances for the first round of adversarial training from the instances to be labeled; the first round is determined as this round, and adversarial training is performed on the encoder and the discriminator in this round based on the target instances labeled with relationship information, and the encoder after this round of adversarial training is used to encode the current instance to be labeled to obtain the relationship features corresponding to the current instance to be labeled; the discriminator after this round of adversarial training selects the target instance from the current instance to be labeled based on the relationship features corresponding to the current instance to be labeled, and obtains the relationship information labeled for the selected target instance; the remaining unlabeled instances to be labeled in the current instance to be labeled are used as the current instances to be labeled in the next round, and the next round is used as this round, and the step of performing adversarial training on the encoder and the discriminator in this round based on the target instances labeled with relationship information is returned to continue execution until the adversarial training is stopped when the iterative stopping condition is met.

19. The device according to claim 11, characterized in that The training module is also used to input the target instance after the relationship information is labeled into the relationship classifier to be trained; predict the predicted relationship information corresponding to the target instance through the relationship classifier to be trained; and adjust the model parameters of the relationship classifier to be trained based on the difference between the predicted relationship information and the labeled relationship information corresponding to the target instance.

20. The device according to any one of claims 11 to 19, characterized in that Each instance in the instance set includes at least two entities; each two entities form an entity pair; the entity pair includes a head entity and a tail entity; the apparatus further includes: A generation module is used to determine a relationship triple based on the relationship between the entities in each instance in the instance set; the relationship triple includes a head entity, a tail entity, and the relationship between the head entity and the tail entity; based on the relationship triple, a knowledge graph is generated in the business scenario corresponding to the instance set.

21. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are implemented.

22. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

23. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images