Entity relationship extraction method and device for sentences, computer equipment and storage medium

Abstract

The invention relates to the field of natural language processing, in particular to an entity relationship extraction method for sentences, which is used for carrying out multi-level prompt template construction on a target sentence to enrich context information and more accurately capture the relationship between entities. Based on the prompt template and the relation embedding representation and the entity embedding representation constructed by the entity of the target sentence, interactive processing is carried out by adopting a cross attention mechanism, relation information and multi-view entity information are clearly fused together, and the interactive embedding representation of the target sentence is obtained for relation extraction; and the accuracy and efficiency of relation extraction are improved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a sentence entity relationship extraction method, device, computer equipment and storage medium. Background Art

[0002] The relation extraction (RE) task is an important area in natural language processing (NLP), which aims to identify and extract the relationship between entities from text. Continuous relation extraction (CRE) is a task based on relation extraction that requires the model to maintain stable and accurate classification of old relations while learning new relations. It is widely used in social media analysis, machine reading comprehension, and knowledge graph construction.

[0003] However, existing relation extraction methods are all dedicated to extracting relations between entities, but do not make full use of the entity information, which makes it difficult for the model to effectively capture the correlation between relations and entities, thus affecting the accuracy of relation extraction. Summary of the invention

[0004] Based on this, the purpose of the present invention is to provide a sentence entity relationship extraction method, device, computer equipment and storage medium, construct multiple levels of prompt templates for the target sentence to enrich the context information, more accurately capture the relationship between entities, and construct relationship embedding representations and entity embedding representations based on the prompt templates and the entity of the target sentence. The cross-attention mechanism is used for interactive processing, and the relationship information is clearly integrated with the multi-view entity information to obtain the interactive embedding representation of the target sentence for relationship extraction, thereby improving the accuracy and efficiency of relationship extraction. The specific scheme is as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for extracting entity relations of a sentence, comprising the following steps:

[0006] Obtaining a target sentence and a pre-trained entity relationship extraction model, wherein the target sentence includes a head entity and a tail entity; the entity relationship extraction model includes a template construction module, an embedding module, an interaction module and an entity extraction module;

[0007] Inputting the target sentence into the template construction module to construct multiple levels of prompt templates to obtain multiple levels of prompt templates for the target sentence;

[0008] Inputting the target sentence and several levels of prompt templates of the target sentence into the embedding module for embedding processing respectively, and obtaining the entity embedding representation and the relationship embedding representation of the target sentence;

[0009] Inputting the entity embedding representation and the relationship embedding representation of the target sentence into the embedding interaction module, and using a cross attention mechanism to interactively process the entity embedding representation and the relationship embedding representation to obtain an interactive embedding representation of the target sentence;

[0010] The interactive relationship embedding representation of the target sentence is input into the entity extraction module, and cosine similarity calculation is performed based on the interactive relationship embedding representation of the target sentence and the prototype embedding representations corresponding to several preset relationship tags to obtain the cosine similarity values ​​between the target sentence and the several relationship tags; based on the cosine similarity values, the entity relationship extraction result of the target sentence is obtained.

[0011] In a second aspect, an embodiment of the present application provides a sentence entity relationship extraction device, comprising:

[0012] A data acquisition module, used to obtain a target sentence and a pre-trained entity relationship extraction model, wherein the target sentence includes a head entity and a tail entity; the entity relationship extraction model includes a template construction module, an embedding module, an interaction module and an entity extraction module;

[0013] A multi-level prompt template construction module, used for inputting the target sentence into the template construction module to construct multiple levels of prompt templates, and obtaining several levels of prompt templates of the target sentence;

[0014] An embedding processing module, used to input the target sentence and several levels of prompt templates of the target sentence into the embedding module for embedding processing, so as to obtain an entity embedding representation and a relationship embedding representation of the target sentence;

[0015] A feature interaction module, used to input the entity embedding representation and the relationship embedding representation of the target sentence into the embedding interaction module, and adopt a cross-attention mechanism to interactively process the entity embedding representation and the relationship embedding representation to obtain the interactive embedding representation of the target sentence;

[0016] The sentence entity relationship extraction module is used to input the interactive relationship embedding representation of the target sentence into the entity extraction module, perform cosine similarity calculation based on the interactive relationship embedding representation of the target sentence and the prototype embedding representation corresponding to several preset relationship tags, and obtain the cosine similarity value between the target sentence and several relationship tags; according to the cosine similarity value, obtain the entity relationship extraction result of the target sentence.

[0017] In a third aspect, an embodiment of the present application provides a computer device comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor; when the computer program is executed by the processor, the steps of the sentence entity relationship extraction method as described in the first aspect are implemented.

[0018] In a fourth aspect, an embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the sentence entity relationship extraction method as described in the first aspect are implemented.

[0019] In an embodiment of the present application, a sentence entity relationship extraction method, apparatus, computer equipment and storage medium are provided, and multiple levels of prompt templates are constructed for a target sentence to enrich contextual information and more accurately capture the relationship between entities. Relation embedding representations and entity embedding representations constructed based on the prompt templates and the entities of the target sentence are interactively processed using a cross-attention mechanism to explicitly integrate relationship information with multi-view entity information, and obtain an interactive embedding representation of the target sentence for relationship extraction, thereby improving the accuracy and efficiency of relationship extraction.

[0020] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A flowchart of a method for extracting entity relations from sentences provided in one embodiment of the present application;

[0022] Figure 2 A schematic diagram of the process of S2 in a method for extracting entity relations from a sentence provided in one embodiment of the present application;

[0023] Figure 3 A schematic diagram of the process of S3 in a method for extracting entity relations from a sentence provided in one embodiment of the present application;

[0024] Figure 4 A schematic diagram of the flow of S4 in a method for extracting entity relations from a sentence provided in one embodiment of the present application;

[0025] Figure 5 A schematic diagram of the flow of S6 in a method for extracting entity relations from a sentence provided in another embodiment of the present application;

[0026] Figure 6 A schematic diagram of the flow of S62 in a method for extracting entity relations from a sentence provided in another embodiment of the present application;

[0027] Figure 7 A flowchart of S621 in a method for extracting entity relations from a sentence provided in another embodiment of the present application;

[0028] Figure 8 A flowchart of S624 in a method for extracting entity relations from a sentence provided in another embodiment of the present application;

[0029] Fig. 9 A schematic diagram of the structure of a sentence entity relationship extraction device provided in one embodiment of the present application;

[0030] Fig.10 A schematic diagram of the structure of a computer device provided for one embodiment of the present application. DETAILED DESCRIPTION

[0031] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0032] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0033] It should be understood that, although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" / "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determination".

[0034] See also Figure 1 , Figure 1 A flowchart of a method for extracting entity relationships from a sentence provided in one embodiment of the present application, the method comprising the following steps:

[0035] S1: Obtain the target sentence and the pre-trained entity relation extraction model.

[0036] The execution subject of the sentence entity relationship extraction method is a relationship extraction device of the sentence entity relationship extraction method (hereinafter referred to as the relationship extraction device). In an optional embodiment, the relationship extraction device can be a computer device, a server, or a server cluster composed of multiple computer devices.

[0037] In this embodiment, the relationship extraction device can obtain a target sentence and a pre-trained entity relationship extraction model from a preset database, wherein the target sentence includes a head entity and a tail entity; the entity relationship extraction model includes a template construction module, an embedding module, an interaction module and an entity extraction module.

[0038] S2: Inputting the target sentence into the template construction module to construct multiple levels of prompt templates to obtain multiple levels of prompt templates of the target sentence.

[0039] In order to obtain richer context information, more accurately capture the relationship between entities, and alleviate the overfitting phenomenon, in this embodiment, the relationship extraction device inputs the target sentence into the template construction module to construct multiple levels of prompt templates to obtain several levels of prompt templates for the target sentence, wherein the several levels of prompt templates include word level prompts, entity level prompts, and sentence level prompts; the template construction module includes a large language model.

[0040] The word-level hint focuses on contextual information at the specific word level. Through this template, the model can focus on lexical details in the local context, effectively capture fine-grained relationship signals, and help the model extract relationship features at the word level; the entity-level hint focuses on enhancing the model's understanding of the meaning of the entity itself. By utilizing the explanatory information of the entity, the model can capture the semantics of the entity more deeply, allowing the model to obtain relationship features based on a deeper understanding of the entity semantics; the sentence-level hint extracts the relationship between entities from the perspective of the entire sentence, using the global context to help the model more deeply understand the meaning of the sentence and the interaction between entities, helping the model capture relationship features at the sentence level.

[0041] See also Figure 2 , Figure 2 The flowchart of S2 in the entity relationship extraction method of a sentence provided in one embodiment of the present application includes steps S21 to S22, which are specifically as follows:

[0042] S21: Input the target sentence into the large language model, perform entity interpretation according to the head entity and the tail entity in the target sentence, and obtain the entity interpretation representation of the target sentence; use the head entity and the tail entity in the target sentence as prompts, perform example construction according to the large language model, and obtain positive examples and negative examples of the target sentence.

[0043] The large language model (LLM) has powerful natural language understanding and generation capabilities. In this embodiment, the relation extraction device inputs the target sentence into the large language model, performs entity interpretation according to the head entity and the tail entity in the target sentence, and obtains the entity interpretation representation of the target sentence.

[0044] The relation extraction device uses the head entity and the tail entity in the target sentence as prompts, performs example construction according to the large language model, and obtains positive examples and negative examples of the target sentence, as described below:

[0045]

[0046] In the formula, x pos As a positive example, is the positive hint vector corresponding to the head entity output by the large language model, is the positive prompt vector corresponding to the tail entity output by the large language model, s pos is the positive prompt representation of the target sentence output by the large language model, x neg As a negative example, is the negative prompt vector corresponding to the head entity output by the large language model, is the negative prompt vector corresponding to the tail entity output by the large language model, s neg The negative prompt representation of the target sentence output by the large language model.

[0047] S22: constructing the word-level prompt of the target sentence, splicing the word-level prompt with the entity interpretation representation to obtain the entity-level prompt; splicing the word-level prompt with the positive example and the negative example to obtain the sentence-level prompt.

[0048] In this embodiment, the relation extraction device constructs a word level prompt of the target sentence, wherein the word level prompt is:

[0049] X T ={[CLS],e h ,[MASK],e t ,[SEP],s,[SEP]}

[0050] Where, X Tis a word level prompt, [CLS] and [SEP] are special identifiers, e h is the entity vector corresponding to the head entity, e t is the entity vector corresponding to the tail entity, and s is the sentence representation of the target sentence.

[0051] The relation extraction device concatenates the word level prompt with the entity interpretation representation to obtain the entity level prompt, wherein the entity level prompt is:

[0052] X E =X T +M

[0053] Where, X e is the entity level hint, and M is the entity explanation representation.

[0054] The relation extraction device concatenates the word level prompt with the positive example and the negative example to obtain the sentence level prompt, wherein the sentence level prompt is:

[0055] X S =X T +x pos +x neg

[0056] Where, X S Provides sentence level prompts.

[0057] S3: Input the target sentence and several levels of prompt templates of the target sentence into the embedding module for embedding processing, and obtain the entity embedding representation and the relationship embedding representation of the target sentence.

[0058] In this embodiment, the relationship extraction device inputs the target sentence and several levels of prompt templates of the target sentence into the embedding module for embedding processing, and obtains the entity embedding representation and the relationship embedding representation of the target sentence.

[0059] See also Figure 3 , Figure 3 The flowchart of S3 in the entity relationship extraction method of a sentence provided in one embodiment of the present application includes steps S31 to S32, which are specifically as follows:

[0060] S31: Embedding the head entity and the tail entity in the target sentence to obtain the embedding vectors of the head entity and the tail entity, performing full connection processing based on the embedding vectors of the head entity and the tail entity to obtain the entity joint embedding vector of the target sentence, combining the embedding vectors of the head entity, the tail entity and the entity joint embedding vector to obtain the entity embedding representation.

[0061] In this embodiment, the relation extraction device performs embedding processing on the head entity and the tail entity in the target sentence to obtain the embedding vectors of the head entity and the tail entity, and performs full connection processing according to the embedding vectors of the head entity and the tail entity to obtain the entity joint embedding vector of the target sentence, as described below:

[0062] E h =MeanPool(f θ (e h ))

[0063] E t =MeanPool(f θ (e t ))

[0064] E ht =FC(concat(E h ,E t ))

[0065] In the formula, E h is the embedding vector of the head entity, E t is the embedding vector of the tail entity, E ht is the entity joint embedding vector, f θ (·) is the encoder function, MeanPool(·) is the average pooling function, concat(·) is the concatenation operation, and FC(·) is the fully connected layer function.

[0066] The relationship extraction device combines the embedding vectors of the head entity, the tail entity, and the entity joint embedding vector to obtain the entity embedding representation, wherein the entity embedding representation is:

[0067] E=[E h ,E t ,E ht ]

[0068] Where E is the entity embedding representation.

[0069] S32: respectively embed the word level prompts, entity level prompts and sentence level prompts to obtain word level relationship embedding representation, entity level relationship embedding representation and sentence level relationship embedding representation, and fuse the word level relationship embedding representation, entity level relationship embedding representation and sentence level relationship embedding representation according to a preset relationship embedding fusion algorithm to obtain the relationship embedding representation of the target sentence.

[0070] In this embodiment, the relation extraction device embeds the word level prompts, entity level prompts and sentence level prompts respectively to obtain word level relation embedding representation, entity level relation embedding representation and sentence level relation embedding representation, so as to fully capture the context information of the input sequence and provide a deeper semantic understanding, as described below:

[0071]

[0072] In the formula, is the word hierarchical relationship embedding representation, Embedding representation for entity hierarchical relations, Embedding representation for sentence level relations.

[0073] The relation extraction device fuses the word level relation embedding representation, entity level relation embedding representation and sentence level relation embedding representation according to a preset relation embedding fusion algorithm to obtain the relation embedding representation of the target sentence, and fuses the three different levels of context information to perform relation extraction and improve the accuracy of relation extraction, wherein the relation embedding fusion algorithm is:

[0074]

[0075] In the formula, R is the relation embedding representation, is the word hierarchical relationship embedding representation, is the sentence level relationship embedding representation, is the embedding representation of entity hierarchical relationship, and Mean(·) is the average processing function.

[0076] S4: Input the entity embedding representation and the relationship embedding representation of the target sentence into the embedding interaction module, and use the cross-attention mechanism to interactively process the entity embedding representation and the relationship embedding representation to obtain the interactive embedding representation of the target sentence.

[0077] In this embodiment, the relationship extraction device inputs the entity embedding representation and the relationship embedding representation of the target sentence into the embedding interaction module, and uses a cross-attention mechanism to interactively process the entity embedding representation and the relationship embedding representation, explicitly integrating the relationship information with the multi-view entity information, and obtaining the interactive embedding representation of the target sentence.

[0078] See also Figure 4 , Figure 4 The flowchart of S4 in the entity relationship extraction method of a sentence provided in one embodiment of the present application includes steps S41 to S43, which are specifically as follows:

[0079] S41: Constructing an adaptive weight matrix of the target sentence according to the entity joint embedding vector in the entity embedding representation and a preset multi-layer perceptron.

[0080] In this embodiment, the relationship extraction device constructs an adaptive weight matrix of the target sentence according to the entity joint embedding vector in the entity embedding representation and a preset multi-layer perceptron, wherein the adaptive weight matrix is:

[0081] A=σ g (MLP(E ht ))

[0082] Where A is the adaptive weight matrix, E ht is the entity joint embedding vector, σ g (·) is the discrete distribution reparameterization function, and MLP(·) is the multilayer perceptron function.

[0083] S42: Obtain a cross-attention feature representation based on the entity embedding representation, the relationship embedding representation, the adaptive weight matrix, the preset attention mask matrix and the cross-attention calculation algorithm.

[0084] The cross attention calculation algorithm is:

[0085]

[0086] Where ACra(R,E) is the cross attention feature representation, η is the number of weight parameters, α j is the jth adaptive weight parameter in the adaptive weight matrix, Q j , K j 、V j are the jth query, key, and value, respectively, Q j =EW j Q ,K j =EW j K ,V j =RW j V , E is the entity embedding representation, W j Q , W j K , W j V are the preset first, second and third attention parameter matrices respectively, T is the transposition symbol, d h is the dimension parameter, B j is the jth mask parameter in the attention mask matrix, which is used to limit the entity information available in the cross-attention mechanism, σ(·) is the normalized exponential function, is the product symbol, which means the product of the element and the matrix.

[0087] In this embodiment, the relationship extraction device obtains a cross-attention feature representation based on the entity embedding representation, the relationship embedding representation, the adaptive weight matrix, the preset attention mask matrix and the cross-attention calculation algorithm, uses the entity information to obtain an adaptive weight matrix, and uses the adaptive weight matrix to fuse the relationship features with the entity information from different perspectives, thereby integrating multi-perspective cross-attention, highlighting the model's ability to selectively pay attention to different entity information, and thus establishing the correlation between relationships and entities.

[0088] S43: The cross-attention feature representation is processed in sequence through a residual connection layer and a normalization layer to obtain an intermediate feature representation, and the intermediate feature representation is processed in sequence through a self-attention mechanism network and a feedforward network to obtain the interactive embedding representation.

[0089] In this embodiment, the relationship extraction device processes the cross-attention feature representation through a residual connection layer and a normalization layer in sequence to obtain an intermediate feature representation, and processes the intermediate feature representation through a self-attention mechanism network and a feedforward network in sequence to obtain the interactive embedding representation, as described below:

[0090] G a =LN(ACra(R,E)+R)

[0091] G s =LN(SA(G a )+G a )

[0092] R′=LN(FFN(G s )+G s )

[0093] In the formula, G a is the intermediate feature representation, LN(·) is the normalization layer function, G s is the feature representation of the self-attention network output, SA(·) is the self-attention network function, FFN(·) is the feature representation of the feedforward network output, and R′ is the interactive embedding representation.

[0094] S5: Input the interactive relationship embedding representation of the target sentence into the entity extraction module, perform cosine similarity calculation based on the interactive relationship embedding representation of the target sentence and the prototype embedding representations corresponding to several preset relationship tags, and obtain the cosine similarity values ​​between the target sentence and several relationship tags; according to the cosine similarity values, obtain the entity relationship extraction result of the target sentence.

[0095] The prototype embedding representation is obtained after pre-training of the entity extraction module, as the most representative representation in the relation embedding space.

[0096] In this embodiment, the relationship extraction device inputs the interactive relationship embedding representation of the target sentence into the entity extraction module, performs cosine similarity calculation based on the interactive relationship embedding representation of the target sentence and the prototype embedding representations corresponding to several preset relationship tags, and obtains the cosine similarity values ​​between the target sentence and the several relationship tags. Based on the cosine similarity values, the relationship extraction device uses the relationship tag corresponding to the highest cosine similarity value as the entity relationship extraction result to obtain the entity relationship extraction result of the target sentence.

[0097] A multi-level prompt template is constructed for the target sentence to enrich the contextual information and more accurately capture the relationship between entities. The relation embedding representation and entity embedding representation constructed based on the prompt template and the entity of the target sentence are interactively processed using a cross-attention mechanism to explicitly integrate the relation information with the multi-view entity information, and obtain the interactive embedding representation of the target sentence for relation extraction, thereby improving the accuracy and efficiency of relation extraction.

[0098] In an optional embodiment, the step S6 is also included: training the entity relationship extraction model. Figure 5 , Figure 5 The flowchart of S6 in the method for extracting entity relations from sentences provided in another embodiment of the present application includes steps S61 to S63, which are specifically as follows:

[0099] S61: Obtain a training data set, a memory data set, and a relationship label set corresponding to the current task.

[0100] In this embodiment, the relationship extraction device obtains a training data set, a memory data set and a relationship label set corresponding to the current task, wherein the training data set includes a number of sample sentences, the memory data set includes a number of sample sentences accumulated from previous tasks and after screening; the sample sentences include a head entity and a tail entity; the relationship label set includes relationship label vectors of the number of sample sentences; the relationship label vector is used to indicate the relationship between the head entity and the tail entity.

[0101] S62: Based on the training data set, memory data set, relationship label set and entity relationship extraction model to be trained corresponding to the current task, a multi-stage target training method is adopted to perform multi-stage training on the entity relationship extraction model to be trained corresponding to the current task and screen several sample sentences in the training data set corresponding to the current task to obtain the trained entity relationship extraction model corresponding to the current task and several screened sample sentences corresponding to the current task.

[0102] In continuous learning under few-shot conditions, since the amount of data available for training is very limited, the model tends to focus on the noise and details in the data and ignore the underlying overall data distribution, resulting in overfitting of the model to the training data.

[0103] In this embodiment, the relationship extraction device adopts a multi-stage target training method based on the training data set, memory data set, relationship label set and entity relationship extraction model to be trained corresponding to the current task, performs multi-stage training on the entity relationship extraction model to be trained corresponding to the current task, and screens several sample sentences in the training data set corresponding to the current task to obtain the trained entity relationship extraction model corresponding to the current task and several screened sample sentences corresponding to the current task.

[0104] See also Figure 6 , Figure 6 A flow chart of S62 in a method for extracting entity relations from a sentence provided in another embodiment of the present application includes steps S621 to S624, which are specifically as follows:

[0105] S621: Obtain a first-stage loss value based on the training data set, memory data set, relationship label set and entity relationship extraction model to be trained corresponding to the current task, and perform first-stage training on the entity relationship extraction model to be trained corresponding to the current task based on the first-stage loss value to obtain a first-stage training model corresponding to the current task; screen and process a number of sample sentences in the training data set corresponding to the current task to obtain a number of screened sample sentences corresponding to the current task, and add the screened sample sentences corresponding to the current task to the memory data set to obtain a memory data set corresponding to the next task.

[0106] In this embodiment, the relationship extraction device obtains a first-stage loss value based on the training data set, memory data set, relationship label set and entity relationship extraction model to be trained corresponding to the current task, and performs first-stage training on the entity relationship extraction model to be trained corresponding to the current task based on the first-stage loss value to obtain the first-stage training model corresponding to the current task, wherein the first-stage loss value includes a contrastive learning loss value, a weight regularization loss value, an overall category classification error loss value and a similar category classification error loss value.

[0107] See also Figure 7 , Figure 7 A flow chart of S621 in a method for extracting entity relations from a sentence provided in another embodiment of the present application includes steps S6211 to S6216, which are specifically as follows:

[0108] S6211: Input the training data set and the memory data set corresponding to the current task into the entity relationship extraction model to be trained corresponding to the current task, and obtain the positive example embedding representation, negative example embedding representation, sentence hierarchical relationship embedding representation, adaptive weight matrix and interactive embedding representation of several sample sentences in the first stage corresponding to the current task.

[0109] In this embodiment, the relationship extraction device inputs the training data set and the memory data set corresponding to the current task into the entity relationship extraction model to be trained corresponding to the current task, and obtains the positive example embedding representation, negative example embedding representation, sentence hierarchical relationship embedding representation, adaptive weight matrix and interactive embedding representation of several sample sentences in the first stage corresponding to the current task.

[0110] Specifically, for the positive example embedding representation and the negative example embedding representation, the positive example embedding representation and the negative example embedding representation are obtained by embedding the positive example and the negative example relationship of the sample sentence, respectively, as described as follows:

[0111] h p =f θ (x pos )

[0112] h n =f θ (x neg )

[0113] In the formula, h p is the positive example embedding representation, h n Embedding representation for negative examples.

[0114] For the sentence hierarchical relationship embedding representation, adaptive weight matrix and interactive embedding representation, the specific embodiments can refer to step S22 and step S41, which will not be repeated here.

[0115] S6212: Obtain a contrastive learning loss value according to the positive example embedding representation, the negative example embedding representation, the sentence hierarchical relationship embedding representation of several sample sentences in the first stage corresponding to the current task and a preset contrastive learning loss algorithm.

[0116] The contrastive learning loss algorithm is:

[0117]

[0118] Where, L con To compare the learning loss value, is the sentence level relationship embedding representation, h o is the embedding representation of the positive example, b∈{p,n}, b is the selection index, p is the embedding representation selection index of the positive example, n is the embedding representation selection index of the negative example, h b is used to select positive example embedding representation or indicate negative example embedding representation, τ is the temperature coefficient, and cos(·) is the cosine similarity function.

[0119] In this embodiment, the relation extraction device obtains a contrastive learning loss value based on the positive example embedding representation, negative example embedding representation, sentence level relation embedding representation and a preset contrastive learning loss algorithm of several sample sentences in the first stage corresponding to the current task, so as to make full use of the sentence level hint X constructed previously. s Sample sentences generated by the large language model LLM in the process New positive examples x pos and negative example x neg , enhancing the model’s ability to distinguish different categories and making the relational embedding representation output by the model more discriminative in the vector space.

[0120] S6213: Obtain a weight regularization loss value according to the adaptive weight matrix, interactive embedding representation and preset weight regularization loss algorithm of several sample sentences in the first stage corresponding to the current task.

[0121] The weight regularization loss algorithm is:

[0122] L wr =[cos(R′ x ,R′ y )-cos(A x ,A y )] 2

[0123] Where, L wris the weight regularization loss value, R′ x , R′ y are the interactive embedding representations of different x-th and y-th sample sentences, respectively. x , A y are the adaptive weight parameters of the xth and yth sample sentences respectively, and cos(·) is the cosine similarity function.

[0124] Sample sentences belonging to the same relationship category should learn similar adaptive weights, while sample sentences belonging to different relationship categories should learn different adaptive weights as much as possible. In this embodiment, the relationship extraction device obtains a weight regularization loss value based on the adaptive weight matrix, interactive embedding representation and preset weight regularization loss algorithm of several sample sentences in the first stage corresponding to the current task, so as to maintain the correlation between the sample relationship and its adaptive weight, thereby enhancing the model's ability to predict relationships.

[0125] S6214: According to the relationship label set, several sample sentences corresponding to the current task are divided and classified to construct a training data subset of several relationship types corresponding to the current task, wherein the training data subset includes several sample sentences with the same relationship label vector; according to the interactive embedding representation of several sample sentences in the training data subset of several relationship types corresponding to the current task and a preset prototype embedding calculation algorithm, a prototype embedding representation set of the first stage corresponding to the current task is obtained.

[0126] In this embodiment, the relationship extraction device divides and classifies several sample sentences corresponding to the current task according to the relationship label set, and constructs a training data subset of several relationship types corresponding to the current task, wherein the training data subset includes several sample sentences with the same relationship label vector.

[0127] The relation extraction device obtains a prototype embedding representation set of the first stage corresponding to the current task according to the interactive embedding representation of several sample sentences in the training data subset of several relation types corresponding to the current task and a preset prototype embedding calculation algorithm, wherein the prototype embedding representation set includes prototype embedding representations of several relation types, and the prototype embedding calculation algorithm is:

[0128]

[0129] In the formula, is the prototype embedding representation of the jth relation type corresponding to the kth task, is the training data subset of the jth relation type corresponding to the kth task, x i is the xth sample sentence of the training data subset, R i′ is the interactive embedding representation of the i-th sample sentence.

[0130] S6215: Obtain the overall category classification error loss value according to the interactive embedding representation of several sample sentences in the first stage corresponding to the current task, the prototype embedding representation of several relationship types in the prototype embedding representation set, and the preset overall category classification error loss algorithm.

[0131] The overall category classification error loss algorithm is:

[0132]

[0133] Where, L ce is the classification error loss value of all categories, is a set of sample sentences corresponding to the kth task, p i is the prototype embedding representation of the relation type corresponding to the relation label vector of the i-th sample sentence, is the prototype embedding representation set corresponding to the kth task, p l is the prototype embedding representation of the lth relation type in the prototype embedding representation set.

[0134] In this embodiment, the relationship extraction device obtains the overall category classification error loss value based on the interactive embedding representation of several sample sentences in the first stage corresponding to the current task, the prototype embedding representation of several relationship types in the prototype embedding representation set, and the preset overall category classification error loss algorithm.

[0135] S6216: Obtain a prototype embedding representation set of similar relationship categories of the first stage corresponding to the current task, and obtain a similarity category classification error loss value based on the interactive embedding representation of several sample sentences corresponding to the current task, the prototype embedding representation of several relationship types in the prototype embedding representation set, the prototype embedding representation of several relationship types in the prototype embedding representation set of the similar relationship categories, and a preset similarity category classification error loss algorithm.

[0136] In this embodiment, the relationship extraction device obtains a prototype embedding representation set of similar relationship categories of the first stage corresponding to the current task. Specifically, the relationship extraction device sets a threshold β to filter the prototype embedding representations of several relationship types in the prototype embedding representation set to obtain the prototype embedding representation set of similar relationship categories of the first stage corresponding to the current task, as described below:

[0137]

[0138] In the formula, is the prototype embedding representation set of similarity relationship categories corresponding to the kth task, is the prototype embedding representation of the relation type corresponding to the relation label vector of the i-th sample sentence after screening, p i is the prototype embedding representation of the relation type corresponding to the relation label vector of the i-th sample sentence, and d(·) is the cosine distance metric function.

[0139] The relationship extraction device obtains a similarity category classification error loss value according to the interactive embedding representations of several sample sentences corresponding to the current task, the prototype embedding representations of several relationship types in the prototype embedding representation set, the prototype embedding representations of several relationship types in the prototype embedding representation set of similar relationship categories, and a preset similarity category classification error loss algorithm, wherein the similarity category classification error loss algorithm is:

[0140]

[0141] Where, L cl is the classification error loss value of similar categories, p i is the prototype embedding representation of the relation type corresponding to the relation label vector of the i-th sample sentence, is the prototype embedding representation set of similar relationship categories corresponding to the kth task, p a The prototype embedding representation of the ath relationship type in the prototype embedding representation set of the similar relationship categories is provided, so that the model has better relationship prediction ability and the ability to distinguish similar relationship categories.

[0142] The relationship extraction device screens several sample sentences in the training data set corresponding to the current task to obtain several screened sample sentences corresponding to the current task, adds the screened sample sentences corresponding to the current task to the memory data set, and obtains the memory data set corresponding to the next task.

[0143] Specifically, the relationship extraction device calculates cosine similarity based on the relationship embedding representation of several sample sentences in the training data set corresponding to the current task and the prototype embedding representation of the same relationship type obtained, obtains the cosine similarity of all sample sentences of each relationship category, and selects the sample sentences with the highest cosine similarity in each relationship category as the screened sample sentences corresponding to the current task.

[0144] S622: Obtain a second-stage loss value based on the training data set corresponding to the current task, the memory data set corresponding to the next task, the relationship label set, and the first-stage training model corresponding to the current task. Perform second-stage training on the first-stage training model corresponding to the current task based on the second-stage loss value to obtain the second-stage training model corresponding to the current task, wherein the second-stage loss value includes a contrastive learning loss value, a weight regularization loss value, an overall category classification error loss value, and a similar category classification error loss value.

[0145] In this embodiment, the relationship extraction device obtains the second-stage loss value according to the training data set corresponding to the current task, the memory data set corresponding to the next task, the relationship tag set, and the first-stage training model corresponding to the current task, and performs the second-stage training on the first-stage training model corresponding to the current task according to the second-stage loss value to obtain the second-stage training model corresponding to the current task, wherein the second-stage loss value includes the contrastive learning loss value, the weight regularization loss value, the overall category classification error loss value, and the similar category classification error loss value. For specific embodiments, please refer to steps S6211 to S6216, which will not be repeated here.

[0146] The relationship extraction device introduces the memory data set updated in the first stage as the training data for the second stage, and calculates the comparative learning loss value, weight regularization loss value, overall category classification error loss value, and similar category classification error loss value. The second stage training is performed on the basis of the first stage training model, so that the obtained second stage training model can obtain a more accurate relationship embedding representation and a more discriminative prototype embedding representation of the current task training sample, thereby improving the accuracy of relationship extraction.

[0147] S623: Obtain a third-stage loss value based on the memory data set, relationship label set and second-stage training model corresponding to the current task, and perform third-stage training on the second-stage training model corresponding to the current task based on the third-stage loss value to obtain the third-stage training model corresponding to the current task.

[0148] In this embodiment, the relationship extraction device obtains a third-stage loss value based on the memory data set corresponding to the current task, the relationship label set, and the second-stage training model corresponding to the current task. According to the third-stage loss value, the second-stage training model corresponding to the current task is trained in the third stage to obtain the third-stage training model corresponding to the current task, wherein the third-stage loss value includes a category classification error loss value. For specific embodiments, please refer to steps S6214 to S6216, which will not be repeated here.

[0149] The relationship extraction device only uses the memory data set before the update in the first stage as the training data for the third stage, calculates the classification error loss value of all categories and the classification error loss value of similar categories, and conducts the third stage training based on the second stage training model to further improve the model's ability to predict relationships and distinguish similar relationship categories, thereby improving the accuracy of relationship extraction.

[0150] S624: Use the entity relationship extraction model to be trained corresponding to the current task as the teacher model, and the third-stage training model corresponding to the current task as the student model. Use the knowledge distillation method to obtain the knowledge distillation loss value as the fourth-stage loss value based on the memory data set corresponding to the next task and the teacher model and the student model. Perform fourth-stage training on the third-stage training model corresponding to the current task based on the fourth-stage loss value to obtain the fourth-stage training model corresponding to the current task as the trained entity relationship extraction model corresponding to the current task.

[0151] In this embodiment, the relationship extraction device uses the entity relationship extraction model to be trained corresponding to the current task as the teacher model, and the third-stage training model corresponding to the current task as the student model, and adopts the knowledge distillation method to obtain the knowledge distillation loss value as the fourth-stage loss value based on the memory data set corresponding to the next task and the teacher model and the student model.

[0152] The relationship extraction device performs fourth-stage training on the third-stage training model corresponding to the current task according to the fourth-stage loss value, and obtains the fourth-stage training model corresponding to the current task as the trained entity relationship extraction model corresponding to the current task.

[0153] See also Figure 8 , Figure 8 A flow chart of S624 in a method for extracting entity relations from a sentence provided in another embodiment of the present application includes steps S6241 to S6242, which are specifically as follows:

[0154] S6241: Input the memory data set corresponding to the next task into the teacher model and the student model respectively, and obtain the interactive embedding representation of several sample sentences output by the teacher model and the student model corresponding to the current task.

[0155] In this embodiment, the relationship extraction device inputs the memory data set corresponding to the next task into the teacher model and the student model respectively, and obtains the interactive embedding representation of several sample sentences output by the teacher model and the student model corresponding to the current task.

[0156] S6242: Obtain a knowledge distillation loss value based on the interactive embedding representation of several sample sentences output by the teacher model and the student model corresponding to the current task and a preset knowledge distillation loss algorithm.

[0157] The knowledge distillation loss algorithm is:

[0158]

[0159] Where, L kd is the knowledge distillation loss value, is the interactive embedding representation of the i-th sample sentence output by the teacher model, r i I The interactive embedding representation of the i-th sample sentence output by the student model.

[0160] In this embodiment, the relationship extraction device obtains a knowledge distillation loss value based on the interactive embedding representation of several sample sentences output by the teacher model and student model corresponding to the current task and a preset knowledge distillation loss algorithm, so as to perform the fourth stage training on the third stage training model corresponding to the current task, thereby avoiding catastrophic forgetting of the relationships that have been learned in the previous tasks.

[0161] S63: Add several sample sentences that have been screened corresponding to the current task to the memory data set to obtain a memory data set corresponding to the next task; obtain a training data set and a relationship label set corresponding to the next task, and repeat multi-stage target training according to the training data set memory data set and the relationship label set corresponding to the next task according to a preset number of iterations to obtain a trained entity relationship extraction model corresponding to the last task as the target entity relationship extraction model.

[0162] In this embodiment, the relationship extraction device adds several sample sentences that have been screened corresponding to the current task to the memory data set to obtain a memory data set corresponding to the next task; obtains a training data set and a relationship label set corresponding to the next task, and based on the training data set memory data set and the relationship label set corresponding to the next task, repeats multi-stage target training according to a preset number of iterations to obtain a trained entity relationship extraction model corresponding to the last task as the target entity relationship extraction model.

[0163] By continuously reinforcing interactive learning, the model not only obtains a more stable and consistent relationship representation, but also effectively maintains the memory of learned knowledge when processing new tasks, thereby alleviating the problem of catastrophic forgetting, improving the long-term performance and stability of the model in continuous learning, and improving the accuracy of relationship extraction.

[0164] Please refer to Fig. 9 , Fig. 9A schematic diagram of the structure of a sentence entity relationship extraction device provided by an embodiment of the present application. The device can implement all or part of the sentence entity relationship extraction device through software, hardware, or a combination of both. The device 9 includes:

[0165] A data acquisition module 91 is used to obtain a target sentence and a pre-trained entity relationship extraction model, wherein the target sentence includes a head entity and a tail entity; the entity relationship extraction model includes a template construction module, an embedding module, an interaction module and an entity extraction module;

[0166] A multi-level prompt template construction module 92, used for inputting the target sentence into the template construction module to construct multiple levels of prompt templates, and obtaining several levels of prompt templates of the target sentence;

[0167] An embedding processing module 93, used to input the target sentence and several levels of prompt templates of the target sentence into the embedding module for embedding processing, so as to obtain an entity embedding representation and a relationship embedding representation of the target sentence;

[0168] A feature interaction module 94 is used to input the entity embedding representation and the relationship embedding representation of the target sentence into the embedding interaction module, and use a cross attention mechanism to interactively process the entity embedding representation and the relationship embedding representation to obtain an interactive embedding representation of the target sentence;

[0169] The sentence entity relationship extraction module 95 is used to input the interactive relationship embedding representation of the target sentence into the entity extraction module, perform cosine similarity calculation based on the interactive relationship embedding representation of the target sentence and the prototype embedding representation corresponding to several preset relationship tags, and obtain the cosine similarity value between the target sentence and several relationship tags; based on the cosine similarity value, obtain the entity relationship extraction result of the target sentence.

[0170] In an embodiment of the present application, a target sentence and a pre-trained entity relationship extraction model are obtained through a data acquisition module, wherein the target sentence includes a head entity and a tail entity; the entity relationship extraction model includes a template construction module, an embedding module, an interaction module and an entity extraction module; through a multi-level prompt template construction module, the target sentence is input into the template construction module to construct multiple levels of prompt templates, and several levels of prompt templates of the target sentence are obtained; through an embedding processing module, the target sentence and several levels of prompt templates of the target sentence are respectively input into the embedding module for embedding processing, and the entity embedding representation and relationship embedding representation of the target sentence are obtained; Through the feature interaction module, the entity embedding representation and the relationship embedding representation of the target sentence are input into the embedding interaction module, and the entity embedding representation and the relationship embedding representation are interactively processed by the cross attention mechanism to obtain the interactive embedding representation of the target sentence; through the sentence entity relationship extraction module, the interactive relationship embedding representation of the target sentence is input into the entity extraction module, and the cosine similarity is calculated according to the interactive relationship embedding representation of the target sentence and the prototype embedding representation corresponding to several preset relationship tags to obtain the cosine similarity value between the target sentence and several relationship tags; according to the cosine similarity value, the entity relationship extraction result of the target sentence is obtained. Multiple levels of prompt templates are constructed for the target sentence to enrich the context information and more accurately capture the relationship between entities. The relationship embedding representation and the entity embedding representation constructed based on the prompt template and the entity of the target sentence are interactively processed by the cross attention mechanism to clearly integrate the relationship information with the multi-view entity information to obtain the interactive embedding representation of the target sentence for relationship extraction, thereby improving the accuracy and efficiency of relationship extraction.

[0171] Please refer to Fig.10 , Fig.10 The computer device 100 is a schematic diagram of a computer device according to an embodiment of the present application. The computer device 100 includes: a processor 101, a memory 102, and a computer program 103 stored in the memory 102 and executable on the processor 101; the computer device may store a plurality of instructions, which are suitable for being loaded and executed by the processor 101. Figures 1 to 8 The method steps of the embodiment shown in the figure, and the specific execution process can be referred to as shown in the figure Figures 1 to 8 The specific description of the illustrated embodiment will not be repeated here.

[0172] Among them, the processor 101 may include one or more processing cores. The processor 101 uses various interfaces and lines to connect various parts in the server, and executes various functions and processes data of the entity relationship extraction device 9 of the sentence by running or executing instructions, programs, code sets or instruction sets stored in the memory 102, and calling the data in the memory 102. Optionally, the processor 101 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (ProgrambleLogic Array, PLA). The processor 101 can integrate one or more combinations of a central processing unit 101 (Central Processing Unit, CPU), an image processor 101 (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the touch display; the modem is used to process wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor 101, and it can be implemented by a single chip.

[0173] Among them, the memory 102 may include a random access memory 102 (Random Access Memory, RAM), and may also include a read-only memory 102 (Read-Only Memory). Optionally, the memory 102 includes a non-transitory computer-readable storage medium. The memory 102 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 102 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch instructions, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 102 may also be optionally at least one storage device located away from the aforementioned processor 101.

[0174] The present application also provides a storage medium that can store multiple instructions, which are suitable for the processor to load and execute the above-mentioned Figures 1 to 8 The method steps of the embodiment shown in the figure, and the specific execution process can be referred to as shown in the figure Figures 1 to 8 The specific description of the illustrated embodiment will not be repeated here.

[0175] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0176] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0177] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraint algorithm of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0178] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0179] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0180] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0181] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc.

[0182] The present invention is not limited to the above-mentioned embodiments. If various changes or modifications to the present invention do not depart from the spirit and scope of the present invention, and if these changes and modifications fall within the scope of the claims and equivalent technologies of the present invention, the present invention is also intended to include these changes and modifications.

Claims

1. A sentence entity relationship extraction method, characterized in that: The following steps are involved: Obtaining a target sentence and a pre-trained entity relationship extraction model, wherein the target sentence includes a head entity and a tail entity; the entity relationship extraction model includes a template construction module, an embedding module, an interaction module and an entity extraction module; Inputting the target sentence into the template construction module to construct multiple levels of prompt templates to obtain multiple levels of prompt templates for the target sentence; Inputting the target sentence and several levels of prompt templates of the target sentence into the embedding module for embedding processing respectively, and obtaining the entity embedding representation and the relationship embedding representation of the target sentence; Inputting the entity embedding representation and the relationship embedding representation of the target sentence into the embedding interaction module, and using a cross attention mechanism to interactively process the entity embedding representation and the relationship embedding representation to obtain an interactive embedding representation of the target sentence; The interactive relationship embedding representation of the target sentence is input into the entity extraction module, and cosine similarity calculation is performed based on the interactive relationship embedding representation of the target sentence and the prototype embedding representations corresponding to several preset relationship tags to obtain the cosine similarity values ​​between the target sentence and the several relationship tags; based on the cosine similarity values, the entity relationship extraction result of the target sentence is obtained.

2. The sentence entity relationship extraction method according to claim 1, characterized in that: The plurality of levels of prompt templates include word level prompts, entity level prompts and sentence level prompts; the template construction module includes a large language model; The step of inputting the target sentence into the template construction module to construct multiple levels of prompt templates to obtain multiple levels of prompt templates for the target sentence includes the following steps: Input the target sentence into the large language model, perform entity interpretation according to the head entity and the tail entity in the target sentence, and obtain the entity interpretation representation of the target sentence; use the head entity and the tail entity in the target sentence as prompts, perform example construction according to the large language model, and obtain positive examples and negative examples of the target sentence; Construct a word level prompt of the target sentence, and concatenate the word level prompt with the entity interpretation representation to obtain the entity level prompt; and concatenate the word level prompt with the positive example and the negative example to obtain the sentence level prompt.

3. The sentence entity relationship extraction method according to claim 2 is characterized in that: The target sentence and the prompt templates of several levels of the target sentence are respectively input into the embedding module for embedding processing to obtain the entity embedding representation and the relationship embedding representation of the target sentence, including the steps of: Embedding the head entity and the tail entity in the target sentence to obtain the embedding vectors of the head entity and the tail entity, performing full connection processing according to the embedding vectors of the head entity and the tail entity to obtain the entity joint embedding vector of the target sentence, and combining the embedding vectors of the head entity, the tail entity and the entity joint embedding vector to obtain the entity embedding representation; The word level prompt, entity level prompt and sentence level prompt are respectively embedded to obtain word level relationship embedding representation, entity level relationship embedding representation and sentence level relationship embedding representation. According to a preset relationship embedding fusion algorithm, the word level relationship embedding representation, entity level relationship embedding representation and sentence level relationship embedding representation are fused to obtain the relationship embedding representation of the target sentence, wherein the relationship embedding fusion algorithm is: In the formula, R is the relation embedding representation, is the word hierarchical relationship embedding representation, is the sentence level relationship embedding representation, is the embedding representation of entity hierarchical relationship, and Mean(·) is the average processing function.

4. The sentence entity relationship extraction method according to claim 3 is characterized in that: The entity embedding representation and the relationship embedding representation of the target sentence are input into the embedding interaction module, and the entity embedding representation and the relationship embedding representation are interactively processed by using a cross attention mechanism to obtain the interactive embedding representation of the target sentence, including the steps of: According to the entity joint embedding vector in the entity embedding representation and the preset multi-layer perceptron, an adaptive weight matrix of the target sentence is constructed, wherein the adaptive weight matrix is: A=σ g (MLP(E ht )) Where A is the adaptive weight matrix, E ht is the entity joint embedding vector, σ g (·) is the discrete distribution reparameterization function, MLP(·) is the multilayer perceptron function; According to the entity embedding representation, the relationship embedding representation, the adaptive weight matrix, the preset attention mask matrix and the cross attention calculation algorithm, a cross attention feature representation is obtained, wherein the cross attention calculation algorithm is: Where ACra(R,E) is the cross attention feature representation, η is the number of weight parameters, α j is the jth adaptive weight parameter in the adaptive weight matrix, Q j , K j 、V j are the jth query, key, and value, respectively, Q j =EWj j Q ,K j =EW j K ,V j =RW j V , E is the entity embedding representation, W j Q , W j K , W j V are the preset first, second and third attention parameter matrices respectively, T is the transposition symbol, d h is the dimension parameter, B j is the jth mask parameter in the attention mask matrix, σ(·) is the normalized exponential function, is the product symbol, which means the product of the element and the matrix; The cross-attention feature representation is processed in turn through a residual connection layer and a normalization layer to obtain an intermediate feature representation, and the intermediate feature representation is processed in turn through a self-attention mechanism network and a feedforward network to obtain the interactive embedding representation.

5. The sentence entity relationship extraction method according to claim 4 is characterized in that: The method further comprises the steps of: training the entity relationship extraction model, wherein the training of the entity relationship extraction model comprises the steps of: Obtaining a training data set, a memory data set, and a relationship label set corresponding to the current task, wherein the training data set includes a number of sample sentences, the memory data set includes a number of sample sentences accumulated from previous tasks and processed after screening; the sample sentences include a head entity and a tail entity; the relationship label set includes a relationship label vector of the number of sample sentences; the relationship label vector is used to indicate the relationship between the head entity and the tail entity; According to the training data set, memory data set, relationship label set and entity relationship extraction model to be trained corresponding to the current task, a multi-stage target training method is adopted to perform multi-stage training on the entity relationship extraction model to be trained corresponding to the current task and screen several sample sentences in the training data set corresponding to the current task, so as to obtain the trained entity relationship extraction model corresponding to the current task and several screened sample sentences corresponding to the current task; Add several sample sentences that have been screened corresponding to the current task to the memory data set to obtain the memory data set corresponding to the next task; obtain the training data set and relationship label set corresponding to the next task, and repeat the multi-stage target training according to the training data set memory data set and relationship label set corresponding to the next task according to a preset number of iterations to obtain the trained entity relationship extraction model corresponding to the last task as the target entity relationship extraction model.

6. The sentence entity relationship extraction method according to claim 5, characterized in that: The method adopts a multi-stage target training method according to the training data set, memory data set, relationship tag set and entity relationship extraction model to be trained corresponding to the current task, performs multi-stage training on the entity relationship extraction model to be trained corresponding to the current task and screens a number of sample sentences in the training data set corresponding to the current task, and obtains the trained entity relationship extraction model corresponding to the current task and a number of screened sample sentences corresponding to the current task, including the steps of: According to the training data set, memory data set, relationship label set and entity relationship extraction model to be trained corresponding to the current task, a first-stage loss value is obtained; according to the first-stage loss value, the entity relationship extraction model to be trained corresponding to the current task is trained in the first stage to obtain the first-stage training model corresponding to the current task; Screening a number of sample sentences in the training data set corresponding to the current task to obtain a number of sample sentences after screening corresponding to the current task, adding the number of sample sentences after screening corresponding to the current task to the memory data set to obtain a memory data set corresponding to the next task, wherein the first stage loss value includes a contrastive learning loss value, a weight regularization loss value, an overall category classification error loss value, and a similar category classification error loss value; According to the training data set corresponding to the current task, the memory data set corresponding to the next task, the relationship label set and the first-stage training model corresponding to the current task, a second-stage loss value is obtained; according to the second-stage loss value, the first-stage training model corresponding to the current task is trained in the second stage to obtain the second-stage training model corresponding to the current task, wherein the second-stage loss value includes a contrastive learning loss value, a weight regularization loss value, a classification error loss value for all categories and a classification error loss value for similar categories; According to the memory data set, the relationship label set and the second-stage training model corresponding to the current task, a third-stage loss value is obtained; according to the third-stage loss value, the second-stage training model corresponding to the current task is trained in the third stage to obtain the third-stage training model corresponding to the current task, wherein the third-stage loss value includes a category classification error loss value; The entity relationship extraction model to be trained corresponding to the current task is used as the teacher model, and the third-stage training model corresponding to the current task is used as the student model. The knowledge distillation method is adopted to obtain the knowledge distillation loss value as the fourth-stage loss value according to the memory data set corresponding to the next task and the teacher model and the student model; according to the fourth-stage loss value, the third-stage training model corresponding to the current task is trained in the fourth stage to obtain the fourth-stage training model corresponding to the current task as the trained entity relationship extraction model corresponding to the current task.

7. The sentence entity relationship extraction method according to claim 6 is characterized in that: The method obtains a first-stage loss value according to a training data set, a memory data set, a relationship label set and an entity relationship extraction model to be trained corresponding to the current task, and performs first-stage training on the entity relationship extraction model to be trained corresponding to the current task according to the first-stage loss value to obtain a first-stage training model corresponding to the current task, including the steps of: Input the training data set and the memory data set corresponding to the current task into the entity relationship extraction model to be trained corresponding to the current task, and obtain the positive example embedding representation, negative example embedding representation, sentence hierarchical relationship embedding representation, adaptive weight matrix and interactive embedding representation of several sample sentences in the first stage corresponding to the current task; wherein the positive example embedding representation and the negative example embedding representation are obtained by embedding the positive example and negative example relationship of the sample sentences respectively; According to the positive example embedding representation, negative example embedding representation, sentence hierarchical relationship embedding representation and a preset contrastive learning loss algorithm of several sample sentences in the first stage corresponding to the current task, a contrastive learning loss value is obtained, wherein the contrastive learning loss algorithm is: Where, L con To compare the learning loss value, is the sentence level relationship embedding representation, h p is the embedding representation of the positive example, b∈{p,n}, b is the selection index, p is the embedding representation selection index of the positive example, n is the embedding representation selection index of the negative example, h b is used to select positive example embedding representation or indicate negative example embedding representation, τ is the temperature coefficient, and cos(·) is the cosine similarity function; According to the adaptive weight matrix of several sample sentences in the first stage corresponding to the current task, the interactive embedding representation and the preset weight regularization loss algorithm, the weight regularization loss value is obtained, wherein the weight regularization loss algorithm is: L wr =[cos(R′ x ,R′ y )-cos(A x ,A y )] 2 Where, L wr is the weight regularization loss value, R′ x , R′ y are the interactive embedding representations of different x-th and y-th sample sentences, respectively. x , A y are the adaptive weight parameters of the xth and yth sample sentences, respectively, and cos(·) is the cosine similarity function; According to the relationship label set, several sample sentences corresponding to the current task are divided and classified to construct a training data subset of several relationship types corresponding to the current task, wherein the training data subset includes several sample sentences with the same relationship label vector; according to the interactive embedding representation of several sample sentences in the training data subset of several relationship types corresponding to the current task and a preset prototype embedding calculation algorithm, a prototype embedding representation set of the first stage corresponding to the current task is obtained, wherein the prototype embedding representation set includes prototype embedding representations of several relationship types, and the prototype embedding calculation algorithm is: In the formula, is the prototype embedding representation of the jth relation type corresponding to the kth task, is the training data subset of the jth relation type corresponding to the kth task, x i is the xth sample sentence of the training data subset, R i ′ is the interactive embedding representation of the i-th sample sentence; According to the interactive embedding representations of several sample sentences in the first stage corresponding to the current task, the prototype embedding representations of several relationship types in the prototype embedding representation set, and the preset overall category classification error loss algorithm, the overall category classification error loss value is obtained, wherein the overall category classification error loss algorithm is: Where, L ce is the classification error loss value of all categories, is a set of sample sentences corresponding to the kth task, p i is the prototype embedding representation of the relation type corresponding to the relation label vector of the i-th sample sentence, is the prototype embedding representation set corresponding to the kth task, p l is the prototype embedding representation of the lth relation type in the prototype embedding representation set; Obtain a prototype embedding representation set of similar relationship categories in the first stage corresponding to the current task, and obtain a similarity category classification error loss value according to the interactive embedding representation of several sample sentences corresponding to the current task, the prototype embedding representation of several relationship types in the prototype embedding representation set, the prototype embedding representation of several relationship types in the prototype embedding representation set of the similar relationship category, and a preset similarity category classification error loss algorithm, wherein the similarity category classification error loss algorithm is: Where, L cl is the classification error loss value of similar categories, p i is the prototype embedding representation of the relation type corresponding to the relation label vector of the i-th sample sentence, is the prototype embedding representation set of similar relationship categories corresponding to the kth task, p a It is the prototype embedding representation of the ath relation type in the prototype embedding representation set of the similar relation category.

8. The sentence entity relationship extraction method according to claim 6, characterized in that: The knowledge distillation method is used to obtain the knowledge distillation loss value according to the memory data set corresponding to the next task and the teacher model and the student model, including the steps of: Inputting the memory data set corresponding to the next task into the teacher model and the student model respectively, and obtaining the interactive embedding representation of several sample sentences output by the teacher model and the student model corresponding to the current task; The knowledge distillation loss value is obtained according to the interactive embedding representation of several sample sentences output by the teacher model and the student model corresponding to the current task and the preset knowledge distillation loss algorithm, wherein the knowledge distillation loss algorithm is: Where, L kd is the knowledge distillation loss value, is the interactive embedding representation of the i-th sample sentence output by the teacher model, r i ′ The interactive embedding representation of the i-th sample sentence output by the student model.

9. A sentence entity relationship extraction device, characterized in that: include: A data acquisition module, used to obtain a target sentence and a pre-trained entity relationship extraction model, wherein the target sentence includes a head entity and a tail entity; the entity relationship extraction model includes a template construction module, an embedding module, an interaction module and an entity extraction module; A multi-level prompt template construction module, used for inputting the target sentence into the template construction module to construct multiple levels of prompt templates, and obtaining several levels of prompt templates of the target sentence; An embedding processing module, used to input the target sentence and several levels of prompt templates of the target sentence into the embedding module for embedding processing, so as to obtain an entity embedding representation and a relationship embedding representation of the target sentence; A feature interaction module, used to input the entity embedding representation and the relationship embedding representation of the target sentence into the embedding interaction module, and adopt a cross-attention mechanism to interactively process the entity embedding representation and the relationship embedding representation to obtain the interactive embedding representation of the target sentence; The sentence entity relationship extraction module is used to input the interactive relationship embedding representation of the target sentence into the entity extraction module, perform cosine similarity calculation based on the interactive relationship embedding representation of the target sentence and the prototype embedding representation corresponding to several preset relationship tags, and obtain the cosine similarity value between the target sentence and several relationship tags; according to the cosine similarity value, obtain the entity relationship extraction result of the target sentence.

10. A computer device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor; when the computer program is executed by the processor, the steps of the method for extracting entity relations from sentences as described in any one of claims 1 to 8 are implemented.

Images