Document-level relation extraction method and device, equipment and medium

Abstract

The invention relates to the technical field of natural language processing. The invention discloses a document-level relation extraction method and device, equipment and a medium, which can enhance the feature extraction capability and improve the accuracy of a relation extraction result. The method comprises the following steps: acquiring an initial document; the initial document is marked, a marked document is obtained, and the marked document comprises at least one group of entity pairs; based on the marked document, extracting a head entity feature, a tail entity feature and a relationship feature corresponding to a target entity pair, the target entity pair being any one group of entity pairs in the at least one group of entity pairs; calculating the head entity feature, the tail entity feature and the relation feature by adopting an IRTE method to obtain a first relation score corresponding to the target entity pair; analyzing and processing the first relation score, the head entity feature, the tail entity feature and the relation feature by adopting a hybrid expert system to obtain a final relation score; and obtaining a relation extraction result corresponding to the target entity pair based on the final relation score.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology. More specifically, the present application relates to a document-level relationship extraction method, device, equipment and medium. Background Art

[0002] Traditional document-level relationship extraction methods are usually implemented in the following steps: obtain the initial document; perform data preprocessing on the text data of the initial document to obtain the encoding features of the initial document; convert the encoding features into a graph structure based on heuristic rules; capture the implicit relationship between the entities of the initial document and the sentences of the initial document; obtain the semantic relationship between the entities of the initial document (i.e., the relationship extraction result) through a classifier. Among them, the graph structure includes a mention heterogeneous graph, an entity heterogeneous graph, and an entity and volume heterogeneous graph; the mention heterogeneous graph is used to capture the implicit relationship between mentions and sentences, the entity heterogeneous graph is used to capture the implicit relationship between entities and sentences, and the entity and mention heterogeneous graph is used to capture the implicit relationship between entities and mentions; these implicit relationships are all used for feature extraction. This method generates a large number of graph structures. Due to the influence of noise in the graph structure, this method is prone to deviation in the process of relationship extraction, which makes the feature extraction of this method insufficient, resulting in inaccurate relationship extraction results. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a document-level relationship extraction method, apparatus, device and medium, which can enhance the ability of feature extraction and improve the accuracy of relationship extraction results. The embodiments of the present application are mainly implemented through the following technical solutions:

[0004] A first aspect of an embodiment of the present application provides a document-level relationship extraction method, comprising:

[0005] Get initial documentation;

[0006] Marking the initial document to obtain a marked document, wherein the marked document includes at least one set of entity pairs;

[0007] Extracting head entity features, tail entity features, and relationship features corresponding to a target entity pair based on the marked document, wherein the target entity pair is any one of the at least one group of entity pairs;

[0008] The head entity feature, the tail entity feature and the relationship feature are calculated using the IRTE method to obtain a first relationship score corresponding to the target entity pair;

[0009] Using a hybrid expert system to analyze and process the first relationship score, the head entity feature, the tail entity feature, and the relationship feature to obtain a final relationship score;

[0010] A relationship extraction result corresponding to the target entity pair is obtained based on the final relationship score.

[0011] According to an embodiment of the present application, the step of extracting head entity features, tail entity features, and relationship features corresponding to the target entity pair based on the marked document includes:

[0012] Using a pre-trained language model to encode the marked document to obtain a context embedding vector set and a first attention weight set of the marked document;

[0013] Extracting the head entity feature and the tail entity feature according to all entity pairs in the marked document;

[0014] Aggregating the first attention weights corresponding to each entity in the marked document in the first attention weight set by a mean aggregation method to obtain a target attention corresponding to each entity;

[0015] Generate head entity attention and tail entity attention based on all target attentions;

[0016] Calculate a second attention weight of the target entity pair based on the head entity attention and the tail entity attention;

[0017] The relationship feature is calculated based on the context embedding vector set and the second attention weight.

[0018] According to one embodiment of the present application, the calculation formula for the step of encoding the marked document using a pre-trained language model to obtain a context embedding vector set and a first attention weight set of the marked document is:

[0019] H,A=PLM(D)=[h1,…,h l ],[a1,..a l ];

[0020] Wherein, H is the context embedding vector set, A is the first attention weight set, PLM is the pre-trained language model, D is the marked document, h1 is the context embedding vector corresponding to the first word in the marked document, and h l is the context embedding vector corresponding to the lth word in the marked document, a1 is the first attention weight corresponding to the first word in the marked document, and a l is the first attention weight corresponding to the lth word in the marked document.

[0021] According to an embodiment of the present application, the step of extracting the head entity feature and the tail entity feature according to all entity pairs in the marked document includes:

[0022] Computing a global representation of all entities in the labeled document;

[0023] The head entity feature is formed by a global representation corresponding to the first entity in all the entity pairs, and the tail entity feature is formed by a global representation corresponding to the second entity in all the entity pairs;

[0024] Among them, each entity obtains the corresponding global representation through logsumexp pooling, and the calculation formula is: Among them, e i is any entity in the marked document, For entity e i The global representation of For entity e i The number of mentions of entity e i The number of times it appears in the marked document, It represents an embedding vector generated by the j-th mention of the i-th entity at its position m, and the exp function is a natural exponential function. is a reference to entity e i Aggregate features, where R d represents the d-dimensional real number space.

[0025] According to one embodiment of the present application, the first attention weight corresponding to each entity in the marked document in the first attention weight set is aggregated by mean aggregation to obtain the calculation formula of the target attention step corresponding to each entity:

[0026]

[0027] in, For entity e i attention, For entity e i The number of mentions, For the jth entity e i The first attention weight corresponding to its location, R h×l It represents an h×l dimensional real number space, h is the number of attention heads, and l is the length of the labeled document.

[0028] According to one embodiment of the present application, the calculation formula for the second attention weight step of calculating the target entity pair based on the head entity attention and the tail entity attention is:

[0029]

[0030] Among them, q (s,o) is the second attention weight, is the head entity attention, For Hadamard, is the tail entity attention, is the transpose of the head entity attention.

[0031] According to one embodiment of the present application, the calculation formula of the step of calculating the relationship feature based on the context embedding vector set and the second attention weight is:

[0032] c (s,o) =H T q (s,o) ;

[0033] Among them, c (s,o) is the relationship feature, H T is the transpose of the context embedding vector set, q (s,o) is the second attention weight.

[0034] According to an embodiment of the present application, the step of calculating the head entity feature, the tail entity feature and the relationship feature using the IRTE method to obtain a first relationship score corresponding to the target entity pair includes:

[0035] Performing weighted feature fusion and reshaping on the head entity feature, the tail entity feature, and the relationship feature to obtain a first image vector;

[0036] Using an upsampling method to enhance the resolution of the first image vector to obtain a second image vector;

[0037] Using an image feature extraction model to extract features from the first image vector and the second image vector to obtain image feature representation;

[0038] The first relationship score is obtained by calculation based on the image feature representation.

[0039] According to one embodiment of the present application, the calculation formula for the step of performing weighted feature fusion and reshaping on the head entity feature, the tail entity feature and the relationship feature to obtain the first image vector is:

[0040]

[0041] Among them, F img is the first image vector, is the head entity feature, is the tail entity feature, c(s,o) is the relationship feature, α is the learnable parameter of the head entity feature, β is the learnable parameter of the tail entity feature, γ is the learnable parameter of the relationship feature, C is the channel of the first image vector, H is the height of the first image vector, and W is the width of the first image vector.

[0042] According to an embodiment of the present application, the step of using a hybrid expert system to analyze and process the first relationship score, the head entity feature, the tail entity feature, and the relationship feature to obtain a final relationship score includes:

[0043] The hybrid expert system is used to fuse the head entity feature and the relationship feature to obtain a subject score;

[0044] The hybrid expert system is used to fuse the tail entity feature and the relationship feature to obtain an object score;

[0045] The subject score and the object score are summed to obtain a second relationship score;

[0046] The sum of the first relationship score and the second relationship score is calculated to obtain the final relationship score.

[0047] According to an embodiment of the present application, the step of obtaining a relationship extraction result corresponding to the target entity pair based on the final relationship score includes:

[0048] Obtaining multiple probability values ​​and preset relationships of multiple entity relationships;

[0049] Selecting the maximum predicted probability value in the final relationship score as the target probability value of the target entity;

[0050] An entity relationship corresponding to the target probability value is found in the preset relationship as the relationship extraction result.

[0051] A second aspect of the embodiment of the present application provides a document-level relationship extraction device, comprising:

[0052] An initial document acquisition module, used to acquire an initial document;

[0053] A marked document obtaining module, used for marking the initial document to obtain a marked document, wherein the marked document includes at least one set of entity pairs;

[0054] A feature extraction module, configured to extract head entity features, tail entity features, and relationship features corresponding to a target entity pair based on the marked document, wherein the target entity pair is any one of the at least one group of entity pairs;

[0055] A first relationship score obtaining module, configured to calculate the head entity feature, the tail entity feature and the relationship feature using an IRTE method to obtain a first relationship score corresponding to the target entity pair;

[0056] A final relationship score obtaining module, used for analyzing and processing the first relationship score, the head entity feature, the tail entity feature and the relationship feature by using a hybrid expert system to obtain a final relationship score;

[0057] The relationship extraction result obtaining module is used to obtain the relationship extraction result corresponding to the target entity pair based on the final relationship score.

[0058] The third aspect of an embodiment of the present application provides a terminal device, including: a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory, and executing the steps of the document-level relationship extraction method provided in the first aspect of the embodiment of the present application.

[0059] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store a computer program, wherein the computer program enables a computer to execute the steps of the document-level relationship extraction method provided in the first aspect of the embodiment of the present application.

[0060] The beneficial effects of the embodiments of the present application include at least:

[0061] The embodiment of the present application combines the IRTE method and the hybrid expert system to extract the features of the document, which can enrich the feature space and make the relationship extraction result more accurate. Specifically, the embodiment of the present application obtains a labeled document by labeling the initial document, and the labeled document contains at least one group of entity pairs; extracts the head entity feature, the tail entity feature and the relationship feature corresponding to the target entity pair based on the labeled document, wherein the target entity pair is any one of the at least one group of entity pairs; uses the IRTE method to calculate the head entity feature, the tail entity feature and the relationship feature to obtain the first relationship score corresponding to the target entity pair; uses the hybrid expert system to analyze and process the first relationship score, the head entity feature, the tail entity feature and the relationship feature to obtain the final relationship score; finally, based on the final relationship score, obtains the relationship extraction result corresponding to the target entity pair. Compared with the prior art, the embodiment of the present application can more fully capture the multiple relationships and complex context information in the document, and the present application does not generate a graph structure, which can overcome the noise influence problem existing in the traditional method. Therefore, the embodiment of the present application can enhance the ability of feature extraction and improve the accuracy of the relationship extraction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0063] Figure 1 A flowchart of the document-level relationship extraction method of the present application in some embodiments;

[0064] Figure 2 A flowchart of the document-level relationship extraction method of the present application in some embodiments;

[0065] Figure 3 This is a principle block diagram of the document-level relationship extraction device of the present application in some embodiments;

[0066] Figure 4 This is a functional block diagram of the terminal device of the present application in some embodiments. DETAILED DESCRIPTION

[0067] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.

[0068] It should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0069] The terms "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0070] The terms "comprises," "includes," or any other variations thereof are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or elements is not necessarily limited to those steps or elements explicitly listed but may include other steps or elements not explicitly listed or inherent to such process, method, product, or apparatus.

[0071] Unless otherwise defined, all technical and scientific terms used in the specification of this application have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used in the specification of this application includes any and all combinations of one or more related listed items.

[0072] The specific implementation of the present application is further described below in conjunction with the accompanying drawings.

[0073] <Document Level Relation Extraction Method>

[0074] like Figure 1 and Figure 2 As shown, Figure 1 A flowchart of a document-level relationship extraction method provided in the first aspect of the embodiment of the present application, Figure 2 FIG. 1 is a flowchart of the document-level relationship extraction method. Figure 1 In the document level relationship extraction method, the document level relationship extraction method includes:

[0075] S1. Get the initial document. This step can refer to Figure 2 , and then click on the Documentation step.

[0076] The initial document may be a word document, and text content is recorded in the initial document.

[0077] S2. Mark the initial document to obtain a marked document, wherein the marked document contains at least one set of entity pairs. Figure 2 The "Document Preprocessing" step in .

[0078] Specifically, the embodiment of the present application uses the marking symbol "*" to emphasize the starting position and the ending position of the mentioned entity (ie, the appearing entity).

[0079] Exemplarily, the content of the initial document is "In 2023, Zhang San and Li Si co-founded Company A. The goal of Zhang San and Li Si is to promote the development of artificial intelligence. Company A has established cooperation with University B to support scientific research projects." After being processed in step S2, the content of the marked document is "In 2023, *Zhang San* and *Li Si* co-founded *Company A*. The goal of *Zhang San* and *Li Si* is to promote the development of artificial intelligence. *Company A* has established cooperation with *University B* to support scientific research projects."

[0080] In the above example, there are four groups of entity pairs in the labeled document, namely (Zhang San, Li Si), (Zhang San, Company A), (Li Si, Company A) and (Company A, University B). Among them, (Zhang San, Li Si), (Zhang San, Company A) and (Li Si, Company A) are all related entity pairs, and (Company A, University B) is an unrelated entity pair. Among the multiple groups of related entity pairs, the entity pair relationship that needs to be predicted for (Zhang San, Li Si) is a partnership relationship, the entity pair relationship that needs to be predicted for (Zhang San, Company A) is a founder relationship, and the entity pair relationship that needs to be predicted for (Li Si, Company A) is a founder relationship. Among the unrelated entity pairs, the entity pair relationship that needs to be predicted for (Company A, University B) is an unrelated relationship.

[0081] In the embodiment of the present application, D is used to represent the marked document, that is, the preprocessed document, and the length of the marked document is l. Therefore, D can be represented as Among them, x t Represented as the word at position t. The initial document contains n entities

[0082] S3. Extracting head entity features, tail entity features, and relationship features corresponding to a target entity pair based on the marked document, wherein the target entity pair is any one of the at least one group of entity pairs.

[0083] Furthermore, the step of extracting head entity features, tail entity features, and relationship features corresponding to the target entity pair based on the marked document includes:

[0084] S31, using a pre-trained language model to encode the labeled document to obtain a context embedding vector set and a first attention weight set of the labeled document. The step S31 can be understood as Figure 2 The "Model Encoding" step in .

[0085] The pre-trained language model may be a BERT (Bidirectional Encoder Representations from Transformers) pre-trained language model, a RoBerta (Robustly Optimized BERT Pretraining Approach) pre-trained language model, or a DeBerta (Decoding-enhanced BERT with Disentangled Attention) pre-trained language model.

[0086] The pre-trained language model encodes each word in the labeled document into a d-dimensional vector. Therefore, assuming that the document encoding H after the pre-trained language model is [82,768], h1,h2…h 82 Respectively represent the encoding vector at that position; the dimension of self-attention A is [12,82,82], where 12 represents the number of attention heads, and (82,82) represents the mutual influence between 82 words, i.e., attention. The d-dimensional vector can be a 768-dimensional vector or a 1024-dimensional vector. The specific dimension can be determined by those skilled in the art according to actual needs.

[0087] Each element in the context embedding vector set is a context embedding vector.

[0088] Each element in the first attention weight set is a first attention weight.

[0089] Furthermore, the calculation formulas of the context embedding vector set and the first attention weight set are as follows:

[0090] H,A=PLM(D)=[h1,…,h l ],[a1,..a l ];

[0091] Wherein, H is the context embedding vector set, A is the first attention weight set, PLM is the pre-trained language model, D is the marked document, h1 is the context embedding vector corresponding to the first word in the marked document, and h l is the context embedding vector corresponding to the lth word in the marked document, a1 is the first attention weight corresponding to the first word in the marked document, and a l is the first attention weight corresponding to the lth word in the marked document.

[0092] It can be understood that in the above formula, H=[h1,…,h l ],A=[a1,..a l ].

[0093] S32: Extract the head entity feature and the tail entity feature according to all entity pairs in the marked document.

[0094] Further, the step S32 includes:

[0095] S321. Calculate the global representation of all entities in the marked document.

[0096] Furthermore, in the embodiment of the present application, each entity in the marked document can obtain a corresponding global representation through logsumexp pooling, and the specific formula is as follows:

[0097]

[0098] Among them, e i is any entity in the marked document, For entity e i The global representation of For entity e i The number of mentions of entity e i The number of times it appears in the marked document, It represents an embedding vector generated by the j-th mention of the i-th entity at its position m, and the exp function is a natural exponential function. is a reference to entity e i Aggregate features, where R d represents the d-dimensional real number space.

[0099] For example, when the entity is Zhang San and the entity is mentioned twice, the positions where the entity appears twice are recorded as m1 and m2 respectively. It can be understood that the calculation formula of the global representation corresponding to the entity is:

[0100]

[0101] Among them, Zhang San is the entity, h 张三 is the global representation corresponding to entity Zhang San, An embedding vector generated for the entity Zhang San at the first appearance position m1, An embedding vector generated for the entity Zhang San at position m2 where it appears for the second time.

[0102] S322: The head entity feature is formed by a global representation corresponding to the first entity in all the entity pairs, and the tail entity feature is formed by a global representation corresponding to the second entity in all the entity pairs.

[0103] Referring to the example of the marked document "In 2023, *Zhang San* and *Li Si* co-founded *Company A*. The goal of *Zhang San* and *Li Si* is to promote the development of artificial intelligence. *Company A* has established a cooperation with *University B* to support scientific research projects", it can be determined that the head entity feature is The tail entity is characterized by

[0104] It should be understood that the steps S32 and S33 can be understood as Figure 2 The "entity embedding" step in Entity Embedding is an aggregation of encoding features of multiple mentioned entities.

[0105] S33, aggregating the first attention weights corresponding to each entity in the marked document in the first attention weight set by a mean aggregation method to obtain a target attention corresponding to each entity. The target attention is also the first attention weight aggregation of the entity.

[0106] In the embodiment of the present application, the first attention weight set related to the entity e is averaged by aggregation. i The corresponding first attention weights are aggregated to obtain entity e i The corresponding target attention calculation formula is as follows:

[0107]

[0108] in, For entity e i attention, For entity e i The number of mentions, For the jth entity e i The first attention weight corresponding to its location, R h×l It represents an h×l dimensional real number space, h is the number of attention heads, and l is the length of the labeled document.

[0109] For example, referring to the example of the marked document "In 2023, *Zhang San* and *Li Si* co-founded *Company A*. The goal of *Zhang San* and *Li Si* is to promote the development of artificial intelligence. *Company A* has established cooperation with *University B* to support scientific research projects", the calculation formula of the target attention corresponding to the entity "Zhang San" is as follows:

[0110]

[0111] Among them A 张三 is the target attention of entity “Zhang San”, N 张三 is the number of mentions of the entity "Zhang San", For the jth entity e i The first attention weight corresponding to its location, R h×l represents an h×l-dimensional real number space, where h is the number of attention heads and l is the length of the labeled document. Assuming that the entity "Zhang San" appears twice in the labeled document, it can be represented as is the first attention weight of the first entity "Zhang San" at its location, is the first attention weight corresponding to the second entity “Zhang San” at its location.

[0112] The calculation formula for the target attention corresponding to the entity "Li Si" is as follows:

[0113]

[0114] Among them A 李四 is the target attention of entity “Li Si”, N 李四 is the number of mentions of the entity "Li Si", For the jth entity e i The first attention weight corresponding to its location, R h×l It represents h×l dimensional real number space, j is the number of attention heads, and l is the length of the labeled document.

[0115] The calculation formula for the target attention corresponding to the entity "Company A" is as follows:

[0116]

[0117] Among them A A The target attention of the company is entity "Company A", N A The number of mentions where the company is the entity "Company A", For the jth entity e i The first attention weight corresponding to its location, R h×l It represents an h×l dimensional real number space, h is the number of attention heads, and l is the length of the labeled document.

[0118] The calculation formula for the target attention corresponding to the entity "University B" is as follows:

[0119]

[0120] Among them A B The university is the target attention of the entity "B University", N B The number of mentions where the university is the entity "B University", For the jth entity e iThe first attention weight corresponding to its location, R h×l It represents an h×l dimensional real number space, h is the number of attention heads, and l is the length of the labeled document.

[0121] S34. Generate head entity attention and tail entity attention based on all target attentions.

[0122] In the embodiment of the present application, the head entity attention is represented as The tail entity attention is expressed as

[0123] S35. Calculate a second attention weight of the target entity pair based on the head entity attention and the tail entity attention.

[0124] The embodiment of the present application uses the method of summing and averaging to calculate the aggregate entity pair (ie, the target entity pair) (e s ,e o )’s second attention weight.

[0125] Specifically, the calculation formula of step S35 is as follows:

[0126]

[0127] Among them, q (s,o) is the second attention weight, is the head entity attention, For Hadamard, is the tail entity attention, is the transpose of the head entity attention.

[0128] q (s,o) is a distribution that reveals each word-to-entity pair (e s ,e o )’s weight (i.e. importance).

[0129] S36. Calculate the relationship feature based on the context embedding vector set and the second attention weight.

[0130] The calculation formula of S36 is as follows:

[0131] c (s,o) =H T q (s,o) ;

[0132] Among them, c (s,o) is the relationship feature, H T is the transpose of the context embedding vector set, q (s,o) is the second attention weight. (s,o)It is also the weighted average of all word encodings.

[0133] The steps can be understood as Figure 2 The "local context pooling" step in . Local context pooling is the aggregation of attention weights for multiple mentions of entities.

[0134] S4, using the IRTE method to calculate the head entity feature, the tail entity feature and the relationship feature to obtain a first relationship score corresponding to the target entity pair. The S4 step can be understood as Figure 2 The "IRTE cross-modal feature fusion" step in .

[0135] The IRTE method aims to enrich the extracted text features through the application of cross-modal feature conversion and image feature extraction model (ie, pre-trained image model).

[0136] Furthermore, the S4 step includes:

[0137] S41, performing weighted feature fusion and reshaping on the head entity feature, the tail entity feature and the relationship feature to obtain a first image vector.

[0138] It should be understood that the first image vector is similar to image information. Assume that before step S41 is executed, the dimensions of the head entity feature, the tail entity feature and the relationship feature are all [82, 768], and after step S41 is executed, the dimensions of the head entity feature, the tail entity feature and the relationship feature are all [82, 32, 24], thereby making the dimension of the first image vector [82, 3, 32, 24].

[0139] Furthermore, the calculation formula of step S41 is:

[0140]

[0141] Among them, F img is the first image vector, is the head entity feature, is the tail entity feature, c (s,o) is the relationship feature, α is the learnable parameter of the head entity feature, β is the learnable parameter of the tail entity feature, γ is the learnable parameter of the relationship feature, C is the channel of the first image vector, H is the height of the first image vector, and W is the width of the first image vector.

[0142] S42: Use an upsampling method to enhance the resolution of the first image vector to obtain a second image vector.

[0143] When the dimension of the first image vector is [82, 3, 32, 24], the dimension of the second image vector is [82, 3, 64, 48].

[0144] Furthermore, the calculation formula of step S42 is:

[0145] F img-up =UpSampling(F img );

[0146] Among them, F img-up is the second image vector, F img is the first image vector, and UpSampling is the upsampling method.

[0147] S43. Use an image feature extraction model to perform feature extraction on the first image vector and the second image vector to obtain image feature representation.

[0148] The image feature extraction model can be a dinov2 model, a vit model or a convnextv model. The dinov2 model is a transformer-based architecture, and a transformer is a model that uses an attention mechanism to process sequence data. The vit model (Vision Transformer) is a model based on a self-attention mechanism for image tasks. The vit model applies the Transformer model in natural language processing to image processing and processes image data through a self-attention mechanism. The convnextv model aims to incorporate some successful design concepts in the Vision Transformer (ViT) into a pure convolutional neural network to improve its performance.

[0149] In other implementations, the image feature extraction model may also be other models, which may be specifically determined by those skilled in the art according to actual needs.

[0150] The dimension of the image feature representation is [82,512].

[0151] Furthermore, the calculation formula of step S43 is:

[0152] R img =Model(F img )+Model(F img-up );

[0153] Among them, R img is the image feature representation, Model is the image feature extraction model, F img is the first image vector, F img-upis the second image vector.

[0154] S44. Obtain the first relationship score by calculation based on the image feature representation.

[0155] The embodiment of the present application uses a linear layer to obtain the first relationship score of the IRTE method.

[0156] Specifically, the calculation formula of S44 is:

[0157] logit IRTE =Linear(R img );

[0158] Among them, logit IRTE is the score for the first relationship, Linear is a linear function, R img is the feature representation of the image.

[0159] S5: Use a hybrid expert system to analyze and process the first relationship score, the head entity feature, the tail entity feature and the relationship feature to obtain a final relationship score. The step S5 can be understood as: Figure 2 The "Hybrid Expert System Enhancement Features" step in .

[0160] The Mixture of Experts (MoE) is an architecture that improves the prediction performance and efficiency of the model by combining multiple sub-models (called "experts"). The hybrid expert system includes an expert network and a gating mechanism. The expert network is responsible for in-depth analysis and processing of the input features, and the gating mechanism (Gate) is responsible for weighting the output of the expert network and selecting the k experts with the highest weights for output. Suppose the input feature x is represented by multiple expert networks f i (x) processing, each expert i produces an output.

[0161] Each expert network in the hybrid expert system consists of a linear transformation layer, a GELU activation function and a layer normalization layer. The gating mechanism consists of a linear layer and a Tanh activation function.

[0162] The gating mechanism is responsible for assigning weights to each expert. The specific formula for calculating the expert weights is:

[0163]

[0164] Among them, w i is the weight of each expert, g(x) is the gating mechanism (i.e., the gating score of expert i), k is the number of selected experts, is the exponential function value of the gating mechanism g(x) of expert i.

[0165] The calculation formula for the expert allocation weights finally outputs the weighted sum of the outputs of the selected experts as

[0166] Furthermore, the step S5 includes:

[0167] S51, using the hybrid expert system to fuse the head entity feature and the relationship feature to obtain a subject score.

[0168] In an embodiment of the present application, four expert networks are set in the MoE method (i.e., the hybrid expert system) to fuse the head entity features (i.e., the subject representation) and the relationship features to form a context-enhanced representation of the subject (i.e., the subject score).

[0169] Furthermore, the calculation formula of step S51 is:

[0170]

[0171] Among them, logit s Scoring the subject, MoE is a hybrid expert system method, is the head entity feature, c (s,o) is the characteristic of the relationship.

[0172] S52: using the hybrid expert system to fuse the tail entity feature and the relationship feature to obtain an object score.

[0173] In an embodiment of the present application, four expert networks are set in the MoE method to fuse the tail entity feature (ie, the object representation) and the relationship feature to form a context-enhanced representation of the object (ie, the object score).

[0174] Furthermore, the calculation formula of step S52 is:

[0175]

[0176] Among them, logit o Scoring the object, MoE is a hybrid expert system method, is the tail entity feature, c (s,o) is the characteristic of the relationship.

[0177] S53: summing the subject score and the object score to obtain a second relationship score.

[0178] Furthermore, the calculation formula of step S53 is:

[0179] logit MoE =logits +logit o ;

[0180] Among them, logit MoE is the second relationship score (i.e., the relationship classification score of the hybrid expert system method), logit s Score the subject, logit o The object is rated.

[0181] S54: Calculate the sum of the first relationship score and the second relationship score to obtain the final relationship score.

[0182] Furthermore, the calculation formula of step S54 is:

[0183] logit=logit IRTE +logit MoE ;

[0184] Among them, logit is the final relationship score, logit IRTE Score the first relationship, logit MoE The second relationship is scored.

[0185] S6: Obtain a relationship extraction result corresponding to the target entity pair based on the final relationship score. S6 can be understood as Figure 2 The "Relationship Prediction" step in .

[0186] Furthermore, the step S6 comprises:

[0187] S61. Obtain preset relationships between multiple probability values ​​and multiple entity relationships.

[0188] Exemplarily, the multiple probability values ​​are 0.7, 0.1 and 0.2, and the multiple entity relationships are partnership, founder relationship and no relationship, then the preset relationship is 0.7 for partnership, 0.1 for founder relationship and 0.2 for no relationship.

[0189] S62: Select the maximum predicted probability value in the final relationship score as the target probability value of the target entity.

[0190] Exemplarily, if the final relationship score is logit=[0.7, 0.1, 0.2], the maximum predicted probability value 0.7 is selected as the target probability value.

[0191] S63: Find an entity relationship corresponding to the target probability value in the preset relationship as the relationship extraction result.

[0192] Based on the above example, since the probability value of 0.7 corresponds to a partnership, the partnership is taken as the relationship extraction result.

[0193] Through the above technical solution, the embodiment of the present application combines the IRTE method and the hybrid expert system to extract the features of the document, which can enrich the feature space and make the relationship extraction results more accurate. Compared with the prior art, the embodiment of the present application can more fully capture the multiple relationships and complex context information in the document, and the present application does not generate a graph structure, which can overcome the noise influence problem existing in the traditional method. Therefore, the embodiment of the present application can enhance the ability of feature extraction and improve the accuracy of relationship extraction results.

[0194] In some implementations, after step S5, the document-level relationship extraction method further includes:

[0195] S71. Divide the at least one group of entity pairs into two types: related entity pairs and unrelated entity pairs.

[0196] Exemplarily, referring to the example of the marked document "In 2023, *Zhang San* and *Li Si* co-founded *Company A*. The goal of *Zhang San* and *Li Si* is to promote the development of artificial intelligence. *Company A* has established cooperation with *University B* to support scientific research projects", (Zhang San, Li Si), (Zhang San, Company A), and (Li Si, Company A) are classified as the related entity pairs, and (Company A, University B) is classified as the unrelated entity pair.

[0197] S72: Perform a comparative learning operation on the related entity pairs and the unrelated entity pairs to obtain a comparative loss. Figure 2 The “contrastive learning” step in .

[0198] Furthermore, the step S72 includes:

[0199] S721: Record the related entity pairs as positive samples, and record the unrelated entity pairs as negative samples.

[0200] Based on the example of step S71, (Zhang San, Li Si), (Zhang San, Company A), (Li Si, Company A) are recorded as positive samples, and (Company A, University B) is recorded as a negative sample.

[0201] S722: Calculate the positive sample similarity of the positive sample by using cosine similarity.

[0202] Furthermore, the calculation formula of step S722 is:

[0203]

[0204] Among them, positive is the positive sample similarity, "·" represents the dot product of two vectors, is the head entity feature corresponding to the positive sample, c (s,o) is the relationship feature, is the tail entity feature corresponding to the positive sample, For vector The 2-norm of For vector The 2-norm of .

[0205] Based on the example of step S721,

[0206] The 2-norm, also known as the Euclidean norm or the L2-norm, is one of the most commonly used norms in vector spaces.

[0207] S723: Calculate the negative sample similarity of the negative samples by using cosine similarity.

[0208] Furthermore, the calculation formula of step S723 is:

[0209]

[0210] Among them, negative is the negative sample similarity, “·” represents the dot product of two vectors, is the head entity feature, c (s,o) is the relationship feature, is the relationship feature, For vector The 2-norm of For vector The 2-norm of .

[0211] Based on the example of step S721,

[0212] S724: Calculate the contrast loss by combining the positive sample similarity and the negative sample similarity.

[0213] Specifically, the calculation formula of S724 is:

[0214]

[0215] Among them, Loss contrastive is the contrast loss, mean represents the sum and average, exp function is an exponential function with the natural constant e as the base, positive is the positive sample similarity, and negative is the negative sample similarity.

[0216] The embodiment of the present application adjusts the parameters of the image feature extraction model based on the contrast loss so that the image feature extraction model can output a more accurate image feature representation.

[0217] Contrastive learning can enhance the model's ability to distinguish different relationship categories by calculating the cosine similarity between positive and negative samples.

[0218] In some implementations, after step S72, the document-level relationship extraction method further includes:

[0219] S81. Set the threshold of the TH class.

[0220] Exemplarily, the threshold is 0.5. In other embodiments, the specific value of the threshold can be determined by those skilled in the art according to actual needs.

[0221] The TH class is a specific class, and the threshold is an adaptive threshold for each sample.

[0222] S82. When the maximum predicted probability of the relationship corresponding to the target related entity pair is greater than the threshold, the target related entity pair is classified into a positive class; when the maximum predicted probability of the relationship corresponding to the target related entity pair is less than or equal to the threshold, the target related entity pair is classified into a negative class, and the target related entity pair is any related entity pair among all related entity pairs; all unrelated entity pairs are classified into a negative class.

[0223] The positive class can be understood as a class with a relationship, and the negative class can be understood as a class with no relationship.

[0224] Exemplarily, all related entity pairs include (Zhang San, Li Si), (Zhang San, Company A), (Li Si, Company A), where the maximum predicted probability of the relationship corresponding to (Zhang San, Li Si) is 0.7, the maximum predicted probability of the relationship corresponding to (Zhang San, Company A) is 0.4, and the maximum predicted probability of the relationship corresponding to (Li Si, Company A) is 0.8. All unrelated entity pairs include (Company A, University B), and the corresponding maximum predicted probability of the relationship is 0.9. Based on the threshold of 0.5, it can be determined that the positive class includes (Zhang San, Li Si) and (Li Si, Company A), and the negative class includes (Zhang San, Company A) and (Company A, University B).

[0225] S83. Calculate a corresponding positive class probability value based on the positive class.

[0226] Furthermore, the calculation formula of step S83 is:

[0227]

[0228] Among them, P(ri ) is the positive class probability value, The category is positive r i The probability score of , T is the temperature parameter, Score the probability of the class being TH.

[0229] S84. Calculate a corresponding negative class probability value based on the negative class.

[0230] Furthermore, the calculation formula of step S84 is:

[0231]

[0232] Among them, P(r TH ) is the negative class probability value, Score the probability of the class being TH, For categories TH and negative category N T The probability score, N T is the negative class subset, and TH is the threshold.

[0233] S85. Calculate a classification loss function based on the positive class probability value and the negative class probability value.

[0234] Furthermore, the calculation formula of step S85 is:

[0235]

[0236] Among them, Loss AGL is the classification loss function, P T is the positive subset, P(r i ) is the positive class probability value, P(r TH ) is the negative class probability value, γ is a dynamic value, more specifically, γ = initial γ *(1+decay rate *epoch), initial γ is the initial value, decay rate is the decay rate, and epoch is the number of model training times.

[0237] S86: Calculate the total loss based on the classification loss function and the contrast loss. The step S86 can be understood as: Figure 2 The "AGL loss function calculation" step in .

[0238] Furthermore, the calculation formula of S86 is:

[0239] Loss RE =Loss AGL +αLoss contrastive ;

[0240] Among them, Loss RE is the total loss, Loss AGL is the classification loss function, Loss contrastive is the contrast loss, and α is a learnable parameter.

[0241] The total loss is used to adjust the parameters of the document-level relationship extraction model, including parameters of network structures such as the pre-trained language model, the IRTE method, and the hybrid expert system.

[0242] The contrast loss and total loss in the embodiment of the present application are both adaptive and dynamically adjusted loss functions, both of which can continuously optimize the model in each iteration, improving the convergence speed and performance of the model. In addition, the embodiment of the present application uses contrastive learning and AGL loss function to better handle long-tail category problems and improve the model's ability to identify rare relationships.

[0243] <Document Level Relation Extraction Device>

[0244] like Figure 3 FIG. 1 is a block diagram showing the principle of a document-level relationship extraction device provided by the second aspect of the embodiment of the present application. Figure 3 In the document level relationship extraction device 100, the document level relationship extraction device 100 comprises:

[0245] An initial document acquisition module 101 is used to acquire an initial document;

[0246] A marked document obtaining module 102 is used to mark the initial document to obtain a marked document, wherein the marked document includes at least one set of entity pairs;

[0247] A feature extraction module 103, configured to extract head entity features, tail entity features, and relationship features corresponding to a target entity pair based on the marked document, wherein the target entity pair is any one of the at least one group of entity pairs;

[0248] A first relationship score obtaining module 104, configured to calculate the head entity feature, the tail entity feature and the relationship feature using an IRTE method to obtain a first relationship score corresponding to the target entity pair;

[0249] A final relationship score obtaining module 105 is used to analyze and process the first relationship score, the head entity feature, the tail entity feature and the relationship feature using a hybrid expert system to obtain a final relationship score;

[0250] The relationship extraction result obtaining module 106 is used to obtain the relationship extraction result corresponding to the target entity pair based on the final relationship score.

[0251] <Terminal Equipment>

[0252] A third aspect of the embodiments of the present application provides a terminal device. The principle block diagram of the terminal device can be as follows: Figure 4 As shown. The terminal device includes a processor, a memory, a network interface, a display screen and a temperature sensor connected via a system bus. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a document-level relationship extraction method is implemented. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the temperature sensor is pre-set inside the terminal device to detect the operating temperature of the internal device.

[0253] Those skilled in the art will understand that Figure 4 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal device to which the scheme of the present invention is applied. The specific terminal device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0254] In some embodiments, an embodiment of the present application provides a terminal device, including: a processor and a memory, the memory is used to store a computer program, the processor is used to call and run the computer program stored in the memory, and execute the steps of the document-level relationship extraction method provided in the first aspect of the above-mentioned embodiment of the present application.

[0255] <Computer Readable Storage Medium>

[0256] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store a computer program, wherein the computer program enables a computer to execute the steps of the document-level relationship extraction method provided in the first aspect of the embodiment of the present application.

[0257] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0258] The technical features of the above embodiments can be combined without changing the basic principles of the present application. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0259] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the scope of patent protection of the present application shall be subject to the attached claims.

Claims

1. A document-level relationship extraction method, characterized in that: include: Get initial documentation; Marking the initial document to obtain a marked document, wherein the marked document includes at least one set of entity pairs; Extracting head entity features, tail entity features, and relationship features corresponding to a target entity pair based on the marked document, wherein the target entity pair is any one of the at least one group of entity pairs; The head entity feature, the tail entity feature and the relationship feature are calculated using the IRTE method to obtain a first relationship score corresponding to the target entity pair; Using a hybrid expert system to analyze and process the first relationship score, the head entity feature, the tail entity feature, and the relationship feature to obtain a final relationship score; A relationship extraction result corresponding to the target entity pair is obtained based on the final relationship score.

2. The document-level relationship extraction method according to claim 1, characterized in that: The step of extracting head entity features, tail entity features and relationship features corresponding to the target entity pair based on the marked document comprises: Using a pre-trained language model to encode the marked document to obtain a context embedding vector set and a first attention weight set of the marked document; Extracting the head entity feature and the tail entity feature according to all entity pairs in the marked document; Aggregating the first attention weights corresponding to each entity in the marked document in the first attention weight set by a mean aggregation method to obtain a target attention corresponding to each entity; Generate head entity attention and tail entity attention based on all target attentions; Calculate a second attention weight of the target entity pair based on the head entity attention and the tail entity attention; Calculate the relationship feature based on the context embedding vector set and the second attention weight; The calculation formula for the step of encoding the labeled document using the pre-trained language model to obtain the context embedding vector set and the first attention weight set of the labeled document is H, A = PLM (D) = [h1, ..., h l ],[a1,..a l ]; where H is the context embedding vector set, A is the first attention weight set, PLM is the pre-trained language model, D is the labeled document, h1 is the context embedding vector corresponding to the first word in the labeled document, and h l is the context embedding vector corresponding to the lth word in the marked document, a1 is the first attention weight of the first word in the marked document, and a l is the first attention weight corresponding to the lth word in the marked document; The step of extracting the head entity feature and the tail entity feature according to all entity pairs in the marked document includes calculating the global representation of all entities in the marked document; the head entity feature is formed by the global representation corresponding to the first entity in all the entity pairs, and the tail entity feature is formed by the global representation corresponding to the second entity in all the entity pairs; wherein each entity obtains the corresponding global representation through logsumexp pooling, and the calculation formula is: Among them, e i is any entity in the marked document, For entity e i The global representation of For entity e i The number of mentions, represents an embedding vector generated by the jth mention of the i-th entity at its position m. The exp function is a natural exponential function. is a reference to entity e i Aggregate features, where R d represents the d-dimensional real number space; The first attention weight corresponding to each entity in the marked document in the first attention weight set is aggregated by the mean aggregation method to obtain the calculation formula of the target attention step corresponding to each entity: in, For entity e i attention, For entity e i The number of mentions, For the jth entity e i The first attention weight corresponding to its location, R h×l It represents h×l dimensional real number space, h is the number of attention heads, and l is the length of the labeled document; The calculation formula of the second attention weight step of calculating the target entity pair based on the head entity attention and the tail entity attention is: Among them, q (s,o) is the second attention weight, is the head entity attention, For Hadamard, is the tail entity attention, is the transpose of the head entity attention.

3. The document-level relationship extraction method according to claim 2, characterized in that: The calculation formula of the step of calculating the relationship feature based on the context embedding vector set and the second attention weight is: c (s,o) =H T q (s,o) ; Among them, c (s,o) is the relationship feature, H T is the transpose of the context embedding vector set, q (s,o) is the second attention weight.

4. The document-level relationship extraction method according to claim 1, characterized in that: The step of using the IRTE method to calculate the head entity feature, the tail entity feature and the relationship feature to obtain a first relationship score corresponding to the target entity pair includes: Performing weighted feature fusion and reshaping on the head entity feature, the tail entity feature, and the relationship feature to obtain a first image vector; Using an upsampling method to enhance the resolution of the first image vector to obtain a second image vector; Using an image feature extraction model to extract features from the first image vector and the second image vector to obtain image feature representation; The first relationship score is obtained by calculation based on the image feature representation.

5. The document-level relationship extraction method according to claim 4, characterized in that: The calculation formula for the step of performing weighted feature fusion and reshaping on the head entity feature, the tail entity feature and the relationship feature to obtain the first image vector is: Among them, F img is the first image vector, is the head entity feature, is the tail entity feature, c (s,o) is the relationship feature, α is the learnable parameter of the head entity feature, β is the learnable parameter of the tail entity feature, γ is the learnable parameter of the relationship feature, C is the channel of the first image vector, H is the height of the first image vector, and W is the width of the first image vector.

6. The document-level relationship extraction method according to claim 1, characterized in that: The step of using a hybrid expert system to analyze and process the first relationship score, the head entity feature, the tail entity feature, and the relationship feature to obtain a final relationship score comprises: The hybrid expert system is used to fuse the head entity feature and the relationship feature to obtain a subject score; The hybrid expert system is used to fuse the tail entity feature and the relationship feature to obtain an object score; The subject score and the object score are summed to obtain a second relationship score; The sum of the first relationship score and the second relationship score is calculated to obtain the final relationship score.

7. The document-level relationship extraction method according to claim 1, characterized in that: The step of obtaining a relationship extraction result corresponding to the target entity pair based on the final relationship score comprises: Obtaining multiple probability values ​​and preset relationships of multiple entity relationships; Selecting the maximum predicted probability value in the final relationship score as the target probability value of the target entity; An entity relationship corresponding to the target probability value is found in the preset relationship as the relationship extraction result.

8. A document-level relationship extraction device, characterized in that: include: An initial document acquisition module, used to acquire an initial document; A marked document obtaining module, used for marking the initial document to obtain a marked document, wherein the marked document includes at least one set of entity pairs; A feature extraction module, configured to extract head entity features, tail entity features, and relationship features corresponding to a target entity pair based on the marked document, wherein the target entity pair is any one of the at least one group of entity pairs; A first relationship score obtaining module, configured to calculate the head entity feature, the tail entity feature and the relationship feature using an IRTE method to obtain a first relationship score corresponding to the target entity pair; A final relationship score obtaining module, used for analyzing and processing the first relationship score, the head entity feature, the tail entity feature and the relationship feature by using a hybrid expert system to obtain a final relationship score; The relationship extraction result obtaining module is used to obtain the relationship extraction result corresponding to the target entity pair based on the final relationship score.

9. A terminal device, characterized in that: include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the steps of the document-level relationship extraction method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program enables a computer to execute the steps of the document-level relationship extraction method described in any one of claims 1 to 7.

Images