Entity relation extraction graph construction method based on bidirectional interactive decoding under prior constraint

Abstract

This invention discloses a method for constructing an entity relation extraction graph under prior constraints and bidirectional interactive decoding, belonging to the field of knowledge engineering technology. The method includes: constructing an entity relation triplet extraction model; performing end-to-end parameter optimization on the entity relation triplet extraction model using R-Drop consistency regularization and FGM adversarial training strategies to obtain a trained entity relation triplet extraction model; inputting the unstructured text to be processed into the trained entity relation triplet extraction model, outputting a set of entity relation triplets; normalizing the entity relation triplet set, mapping it to construct a knowledge graph, and storing the knowledge graph in a graph database. This invention solves the problem that existing knowledge graph construction processes lack bidirectional semantic interaction between entities and relations, and the decoding process lacks constraints on entity relations, leading to insufficient accuracy in extraction results and thus affecting the quality of the knowledge graph.

Description

Technical Field

[0001] This invention belongs to the field of knowledge engineering technology, specifically relating to a method for constructing an entity relationship extraction graph under prior constraints and bidirectional interactive decoding. Background Technology

[0002] With the deepening of digital transformation, the scale of unstructured text data across industries continues to grow, encompassing various forms such as industry procedures, business records, technical documents, and public reports. Rapidly extracting structured knowledge from massive amounts of text to achieve efficient information organization and utilization has become a common need across various fields. Knowledge graphs, with their structured knowledge organization capabilities and intuitive semantic relationships, have become the core foundation supporting downstream applications such as intelligent retrieval, knowledge question answering, risk management, and decision support. However, a large amount of existing knowledge exists in the form of unstructured text, which cannot be directly used for knowledge graph construction. Information extraction techniques are needed to automatically extract structured knowledge from the text. While entity extraction can identify semantic entities in text, simply obtaining entity information is insufficient to meet the core requirement of knowledge graph construction for acquiring relationships between entities.

[0003] Therefore, entity relation extraction methods are increasingly being applied to knowledge graph construction. Existing methods are mainly divided into two categories: pipelined extraction methods and joint entity relation extraction methods. Pipeline extraction methods separate entity recognition and relation extraction into two independent subtasks. While this method is logically simple, it severs the semantic dependency between entity recognition and relation extraction. Errors from preceding tasks are directly propagated to subsequent tasks, leading to a decline in overall extraction performance and making it difficult to meet the needs of knowledge graph construction in complex scenarios. Joint entity relation extraction methods share text features within a unified framework, simultaneously completing entity recognition and relation extraction tasks, enhancing the semantic association between subtasks and mitigating propagation errors to some extent. However, traditional joint extraction methods are mostly based on annotation matrices or graph structure modeling, lacking flexibility when handling complex scenarios such as overlapping entities and nested relations, and struggling to adapt to diverse text structures.

[0004] Currently, generative joint extraction methods model triple extraction as a sequence generation task, supporting complex scenarios such as overlapping triples. However, existing generative joint extraction methods mostly employ unidirectional autoregressive decoding strategies, lacking bidirectional semantic interaction between entity generation and relation prediction, leading to semantic biases in the generated triples. Furthermore, the decoding process lacks effective constraints on the compatibility of entity relation types, resulting in semantically incompatible triples, thus affecting the accuracy and reliability of subsequent knowledge graph construction. Therefore, improving the quality of joint entity relation extraction and achieving the construction of high-quality knowledge graphs from unstructured text is a problem that needs to be solved. Summary of the Invention

[0005] The purpose of this invention is to provide a method for constructing an entity relation extraction graph under prior constraints and bidirectional interactive decoding. This method solves the problem that existing knowledge graph construction processes lack bidirectional semantic interaction between entities and relations, and the decoding process lacks constraints on entity relations, resulting in insufficient accuracy of extraction results and thus affecting the quality of the knowledge graph.

[0006] The technical solution adopted in this invention is a method for constructing an entity relation extraction graph under prior constraints and bidirectional interactive decoding, comprising: Step 1: Construct the entity relation triple extraction model; the entity relation triple extraction model includes an encoder module, a bidirectional interactive decoder module, and a structured prior constraint module; Step 2: Perform end-to-end parameter optimization on the entity relation triplet extraction model using R-Drop consistency regularization and FGM adversarial training strategy to obtain the trained entity relation triplet extraction model. Step 3: Input the unstructured text to be processed into the trained entity relation triple extraction model, and output a set of entity relation triples; Step 4: Normalize the set of entity relation triples, map and construct a knowledge graph, and store the knowledge graph in a graph database.

[0007] The invention is further characterized by: The encoder module includes a word segmentation unit, an embedding layer, a pre-trained language model encoder, and a BiLSTM bidirectional temporal network. The word segmentation unit is used to segment unstructured text into words. The embedding layer is used to generate word content embeddings and relative position embeddings and superimpose them to obtain the initial sequence encoding. The pre-trained language model encoder adopts the DeBERTa-v3-large model with a built-in multi-layer bidirectional Transformer encoder to extract global semantic features of the text. The BiLSTM bidirectional temporal network is used to supplement local context and temporal dependency features and output text sequence features. The bidirectional interactive decoder consists of multiple stacked Transformer decoding layers, with an output layer at the end of the stack. Each Transformer decoding layer includes a self-attention layer, an entity-relation cross-attention layer, and a feedforward network layer. Each Transformer decoding layer is configured with a relation embedding pool and an entity memory for bidirectional entity-relation cross-queries. The relation embedding pool stores learnable relation embedding vectors of all predefined relation types, while the entity memory dynamically stores the span representation vectors and entity type information of entities generated during the decoding process. The output layer includes a linear mapping layer and a Softmax normalization unit. The linear mapping layer maps the hidden state output from the last decoding layer to the global output vocabulary space, obtaining the original prediction scores for each candidate noun. The Softmax normalization unit normalizes the prediction scores to a probability distribution on the vocabulary, selecting the noun with the highest probability as the generation result for the current time step. The structured prior constraint module includes a step-gated mask unit and a type constraint matrix unit; the step-gated mask unit is used to mask invalid prediction positions at different stages of decoding, and the type constraint matrix unit is used to constrain the legality of relation prediction based on the correspondence between entity type and relation type; The structured prior constraint module acts between the linear mapping layer and the Softmax normalization unit. It applies step-gated mask constraints and type constraint matrix constraints to the original prediction scores output by the linear mapping layer before inputting them into the Softmax normalization unit.

[0008] In step 2, R-Drop consistency regularization is used to optimize the model parameters, specifically including: Applying two independent forward propagations to the same input text sequence, and leveraging the randomness of Dropout, yields two sets of sub-model output distributions, denoted as follows: and ,in, This represents the input text sequence. This represents the output sequence of the target triplet; Indicates the given input Time model prediction output The probability distribution; Based on the supervised loss of the conventional sequence generation task, a consistency regularization loss is defined by minimizing the bidirectional KL divergence between the output distributions of the two sub-models. , is represented as:

[0009] In the formula, This represents the calculated consistency regularization loss. and This indicates the calculation of the KL divergence between the output distributions of the two sub-models; The total training loss of the entity relation triple extraction model is the weighted sum of the task supervision loss and the consistency regularization loss, expressed as:

[0010] In the formula, This represents the total training loss of the model. This represents the supervised loss for a regular sequence generation task. The weighting coefficients for consistency regularization loss.

[0011] In step 2, FGM adversarial training is used in the embedding layer of the encoder module to optimize the model execution parameters, specifically including: In each training iteration, the input word embeddings are first calculated. Regarding the present gradient And based on gradient Construct the adversarial perturbation vector as follows:

[0012] In the formula, For adversarial perturbation vectors, This is the disturbance amplitude control coefficient. Represents the L2 norm; Subsequently, the word embeddings after superimposing adversarial perturbation vectors are... The input model undergoes forward propagation to calculate the adversarial loss, which is the word embeddings of the model after adding adversarial perturbation vectors. When used as input, the task loss between the predicted output and the labeled ground truth is calculated; and the model parameters are updated based on the gradient of the adversarial loss.

[0013] In step 3, the unstructured text to be processed is input into the trained entity relation triple extraction model. The encoder module first performs the following processing: Step 3.1: Perform word segmentation on the acquired unstructured text to obtain the corresponding input word sequence. ,in, Indicates the first i Each word element, The total number of lexical units is represented; lexical content embeddings and relative position embeddings are generated from the lexical sequence, and the two types of embeddings are superimposed to form the initial sequence encoding. The initial sequence encoding is fed into the pre-trained DeBERTa-v3-large model, and the text global context semantic encoding is completed through the multi-layer bidirectional Transformer encoder inside the model, and the text global semantic features are output. Step 3.2: Input the feature sequence composed of global semantic features of the text into the BiLSTM bidirectional temporal network, extract features from the feature sequence from both the forward and backward directions, process the bidirectional temporal dependencies between words, capture local contextual information, and output text sequence features that fuse global and local contextual information. ,in Indicates the first i The vector representation of each word in the context semantic space.

[0014] In step 3, the bidirectional interactive decoder receives the text sequence features output by the encoder module and outputs entities using an autoregressive sequence generation method. Relational triples; a single triple is generated sequentially in the order of head entity, tail entity, and relation type. Different triples are distinguished by a special delimiter, which is a preset identifier used to delineate the boundaries of adjacent triples in the autoregressive generation sequence. The relation embedding pool stores all learnable embedding vectors of predefined relation types, represented as , Indicates the first A relation type, This represents the total number of predefined relation types; The entity memory is used to dynamically store the span representation vector and entity type information of entities generated during the decoding process, represented as: , Indicates the first A decoded entity, This indicates the number of entities that have been decoded so far. At each autoregressive generation time step, the bidirectional interactive decoder executes the following steps: Step 3.3: Model the features of the currently generated sequence representation through a self-attention layer, calculate the self-attention output, and then obtain the current decoding state through residual connections and layer normalization; Step 3.4: Input the current decoding state into the entity-relationship cross-attention layer, and perform processing according to the current generation stage: When the decoder is in the entity generation stage, based on the current decoding state For the query vector of cross-attention, the relation embedding pool Attention query is performed; the relation embedding vectors corresponding to each relation type stored in the relation embedding pool are mapped to form the query vector, key, and value in the entity generation stage:

[0015] In the formula, This is the learnable parameter matrix corresponding to the entity generation stage. This is the query vector for the entity generation phase. and These are the key matrix and value matrix obtained from relation embedding pool mapping, respectively; The cross-attention weights for semantic enhancement of relations are calculated and represented as follows:

[0016] In the formula, This is the cross-attention weight matrix for relational semantic enhancement. This represents the transpose of the key matrix obtained from the relation embedding pool mapping. Represents the hidden dimension of a vector; Use the Softmax activation function; The decoding state during the entity generation stage is obtained using a residual update method:

[0017] In the formula, This indicates the decoding state of the entity generation stage after relational semantic enhancement; When the decoder is in the relation prediction phase, based on the current decoding state For the query vector of cross-attention, the entity memory is retrieved in reverse. The span representation vectors of each generated entity stored in the entity memory are mapped to form the query vector, key, and value in the relation prediction stage:

[0018] In the formula, This is the learnable parameter matrix corresponding to the relationship prediction stage. This is the query vector for the relation prediction phase. and These are the key matrix and value matrix obtained from mapping from the entity memory, respectively; The cross-attention weights for entity semantic enhancement are calculated and expressed as follows:

[0019] In the formula, Cross-attention weight matrix for entity semantic enhancement; This represents the transpose of the key matrix obtained by mapping from the entity memory; The decoding state in the relation prediction stage is obtained using a residual update method:

[0020] In the formula, This indicates the decoding state during the relation prediction stage after entity semantic enhancement; Step 3.5: Input the enhanced decoding state obtained through the entity-relationship cross-attention layer into the feedforward network for feature transformation to obtain the output representation of the current time step; Step 3.6: The output representation is converted into the prediction score in the global output vocabulary space through the linear mapping layer. The candidate words are screened and their probabilities are adjusted by the structured prior constraint module. After Softmax normalization, the generation probability of the next word is obtained. The word with the highest probability is selected as the generation result of the current time step. Repeat steps 3.3 to 3.6 until the head entity, tail entity, and relation type are generated, forming a complete entity-relation triplet.

[0021] The structured prior constraint module includes a step-gated mask unit and a type constraint matrix unit; in step 3.6, after the linear mapping layer outputs the prediction score, the structured prior constraint module performs constraint processing on the prediction score, specifically including: The step-gated masking unit executes a step-gated decoding strategy: It constructs a global output vocabulary, which is formed by concatenating the entity candidate span set extracted from the input text with a predefined set of relation types; the entity candidate span set is a dynamically constructed span representation set after encoding all possible continuous segments in the input text; before each time step, it constructs a mask vector of the same length as the global output vocabulary based on the current generation stage. When in the entity generation stage, the index position of the corresponding relation type in the mask vector is assigned a value of negative infinity, and the index position of the corresponding entity candidate span is assigned a value of zero; When in the relation prediction stage, the index position of the corresponding entity candidate span in the mask vector is assigned a value of negative infinity, and the index position of the corresponding relation type is assigned a value of zero; The mask vector is added to the original prediction score output by the linear mapping layer using tensor addition, and then normalized by Softmax to make the generation probability of candidate positions assigned a value of negative infinity zero, thereby physically masking candidate words that are not in the current generation stage.

[0022] The type constraint matrix elements only apply to the relation prediction stage: First, construct the type constraint matrix. ,in, and These represent the number of entity types and the number of relation types, respectively. The type constraint matrix is ​​constructed based on predefined entity type and relation type constraint rules, and encodes the header entity type by statistically analyzing the combination relationships of entity types in the training data. With tail entity type The prior scores of the rationality of each relation type under different combinations; In the relationship prediction phase, based on the generated head entity types... Ending entity type From the predefined type constraint matrix The corresponding compatibility bias vector is obtained from the index. and compatibility bias vector The adjusted relationship prediction score is obtained by superimposing it onto the original relationship prediction score, as shown below:

[0023] In the formula, This is the adjusted score after adding compatibility bias. This represents the original prediction score corresponding to the relation type candidate output by the bidirectional interactive decoder through the linear mapping layer.

[0024] Step 4 specifically includes the following steps: Step 4.1: Perform normalization processing on the output entity-relation triplet set, specifically including: standardizing the format of the entity name text of the triplet, eliminating differences in abbreviations, symbols, and writing formats; merging duplicate triplets with identical head entities, relations, and tail entities; filtering out incomplete triplets with missing entity or relation information to obtain a standardized entity-relation triplet set; Step 4.2: Map the standardized entity relation triple set to a graph structure: Map the head entity and tail entity in the triple to nodes in the knowledge graph, with each node storing the entity name, entity type, and source text identifier attributes; map the relations in the triple to directed edges from the head entity node to the tail entity node, and all nodes and edges together form a directed heterogeneous attribute graph with entities as nodes and semantic relations as edges; Step 4.3: Use a graph database based on the attribute graph model as the storage engine, organize entity nodes, node attributes, relationship edges and edge attributes according to preset mapping rules to realize the structured storage of the knowledge graph, support the CRUD operations on the knowledge graph through query language, and provide a data foundation for downstream knowledge retrieval, association analysis and intelligent applications.

[0025] The beneficial effects of this invention are: This invention presents a method for constructing an entity relation extraction graph under bidirectional interactive decoding with prior constraints. By optimizing the joint entity relation extraction process, it improves the accuracy of extracting structured knowledge from unstructured text, providing more reliable entity relation triplet data for subsequent knowledge graph construction. First, in the bidirectional interactive enhanced decoder, an entity-relation cross-attention module based on a relation embedding pool and an entity memory is constructed. This module uses a bidirectional cross-query mechanism to perceive potential relation semantics during entity generation and to fuse and aggregate generated entity features during relation prediction, alleviating the semantic interaction deficiency problem caused by traditional unidirectional autoregressive decoding. Second, structured prior constraints are integrated into the decoding process in the form of step-gated masks and type constraint matrices. These constraints work synergistically with the semantic features enhanced by bidirectional interaction, optimizing triplet generation under the guidance of entity type and relation type constraint rules, thereby improving the semantic consistency and structural rationality of the generated entity relation triples. Finally, by combining R-Drop consistency regularization and FGM adversarial training strategy to optimize the model training process, the model can improve the accuracy of extraction results in scenarios with relatively limited labeled data, providing high-quality structured knowledge for subsequent knowledge graph construction. Attached Figure Description

[0026] Figure 1 This is a flowchart of the entity relationship extraction graph construction method under prior constraints and bidirectional interactive decoding of the present invention; Figure 2 This is a schematic diagram of the decoder's single-step calculation process in this invention; Figure 3 This is a flowchart of the bidirectional interactive decoding mechanism for entities and relationships in this invention; Figure 4 This is a schematic diagram illustrating the working principle of the type constraint matrix in this invention; Figure 5 This is a partial knowledge graph structure diagram constructed based on entity relation triples in this invention. Detailed Implementation

[0027] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0028] Example 1: This embodiment provides a method for constructing an entity relation extraction graph under prior constraints and bidirectional interactive decoding, such as... Figure 1 As shown, please follow these steps: Step 1: Construct the entity relation triple extraction model; the entity relation triple extraction model includes an encoder module, a bidirectional interactive decoder module, and a structured prior constraint module; Step 2: Perform end-to-end parameter optimization on the entity relation triplet extraction model using R-Drop consistency regularization and FGM adversarial training strategy to obtain the trained entity relation triplet extraction model. Step 3: Input the unstructured text to be processed into the trained entity relation triple extraction model, and output a set of entity relation triples; Step 4: Normalize the set of entity relation triples, map and construct a knowledge graph, and store the knowledge graph in a graph database.

[0029] Example 2: Based on Example 1, the entity relationship triple extraction model constructed in step 1 consists of the following modules: The encoder module includes a word segmentation unit, an embedding layer, a pre-trained language model encoder, and a BiLSTM bidirectional temporal network. The word segmentation unit is used to segment unstructured text into words. The embedding layer generates word content embeddings and relative position embeddings, which are then superimposed to obtain the initial sequence encoding. The pre-trained language model encoder uses the DeBERTa-v3-large model with a built-in multi-layer bidirectional Transformer encoder to extract global semantic features of the text. The DeBERTa-v3-large model is a large-scale pre-trained English language model optimized based on the Transformer architecture. Its core employs a decoupled attention mechanism, decoupling the content representation and relative position representation of words, and calculating attention weights from multiple dimensions, which can more accurately capture the semantic associations and positional dependencies between words in the text. Simultaneously, this model uses replacement word detection as its core pre-training objective, which can make fuller use of all words for training compared to traditional masked language modeling. The BiLSTM (Bidirectional Long Short-Term Memory) bidirectional temporal network is used to supplement local context and temporal dependency features, outputting text sequence features. BiLSTM bidirectional temporal network extracts features from sequences simultaneously in both forward and backward directions, effectively handling temporal dependencies and local contextual information in text, adapting to variable-length sequence data, and mitigating the information loss problem caused by long-distance dependencies.

[0030] In this embodiment, the token segmentation unit is implemented using the Tokenizer tokenizer that is compatible with the DeBERTa-v3-large model. The Tokenizer is an independent preprocessing component released with the DeBERTa-v3-large model, responsible for segmenting the original unstructured text into discrete token sequences that the model can process.

[0031] The word segmentation unit, embedding layer, pre-trained language model encoder, and BiLSTM bidirectional temporal network are connected in series within the encoder module: the output of the word segmentation unit is connected to the input of the embedding layer, the output of the embedding layer is connected to the input of the pre-trained language model encoder, the output of the pre-trained language model encoder is connected to the input of the BiLSTM bidirectional temporal network, and the output of the BiLSTM bidirectional temporal network serves as the overall output of the encoder module, which is then connected to the input of the subsequent bidirectional interactive decoder module.

[0032] The bidirectional interactive decoder consists of multiple stacked Transformer decoding layers, with an output layer at the end of the stack. Each Transformer decoding layer includes a self-attention layer, an entity-relation cross-attention layer (ER Cross-Attention), and a feedforward network layer (FFN). Each Transformer decoding layer is configured with a relation embedding pool and an entity memory for bidirectional entity-relation cross-queries. The relation embedding pool stores learnable relation embedding vectors for all predefined relation types. The entity memory dynamically stores the span representation vectors and entity type information of entities generated during the decoding process. The span representation vector is a semantic vector calculated based on the hidden states corresponding to the start and end positions of the entity in the text. The output layer includes a linear mapping layer and a Softmax normalization unit. The linear mapping layer maps the hidden states output by the last decoding layer to the global output vocabulary space, obtaining the original prediction scores for each candidate word. The Softmax normalization unit normalizes the prediction scores to a probability distribution on the vocabulary, selecting the word with the highest probability as the generation result for the current time step.

[0033] The structured prior constraint module includes a step-gated mask unit and a type constraint matrix unit; the step-gated mask unit is used to mask invalid prediction positions at different stages of decoding, and the type constraint matrix unit is used to constrain the legality of relation prediction based on the correspondence between entity type and relation type; The structured prior constraint module acts between the linear mapping layer and the Softmax normalization unit. It applies step-gated mask constraints and type constraint matrix constraints to the original prediction scores output by the linear mapping layer before inputting them into the Softmax normalization unit.

[0034] Example 3: Unstructured text often features colloquial language, irregular structure, and descriptive noise. When training an entity relation triplet extraction model on a finite number of labeled datasets, the model is prone to overfitting to the limited training samples and becoming overly sensitive to minor perturbations in the input text, thus compromising its stability in complex real-world scenarios. To mitigate the overfitting problem under limited data conditions, this embodiment introduces a regularization training strategy from two aspects: output distribution constraints and input perturbations. Specifically, R-Drop consistency regularization and FGM adversarial training strategies are used to perform end-to-end parameter optimization on the entity relation triplet extraction model.

[0035] R-Drop consistency regularization is used to optimize the model execution parameters, specifically including: During model training, two independent forward propagations are applied to the same input text sequence. Under the influence of Dropout randomness, two sets of sub-model output distributions are obtained, denoted as follows: and ,in, This represents the input text sequence. This represents the output sequence of the target triplet; Indicates the given input Time model prediction output The probability distribution; Building upon the supervised loss of conventional sequence generation tasks, a consistency regularization loss is defined by minimizing the bidirectional KL (Kullback-Leibler) divergence between the output distributions of the two sub-models. , is represented as:

[0036] In the formula, This represents the calculated consistency regularization loss. and This indicates the calculation of the KL divergence between the output distributions of the two sub-models; Therefore, the total training loss of the entity relation triple extraction model is the weighted sum of the task supervision loss and the consistency regularization loss, expressed as:

[0037] In the formula, This represents the total training loss of the model. This represents the supervised loss for a regular sequence generation task. The weighting coefficients for consistency regularization loss. The value ranges from 0.1 to 1.0. In this embodiment, an empirical value is used. =0.2, to balance the weights of task loss and consistency constraints.

[0038] This constraint requires the model to maintain the consistency of the output distribution under random deactivation conditions while fitting the labeled data, thereby reducing the model's dependence on individual features and alleviating the overfitting problem.

[0039] The FGM adversarial training strategy involves using FGM adversarial training at the embedding layer of the encoder module to optimize model parameters. Its basic principle is to inject a small perturbation along the gradient direction of the loss function into the input word embedding vector, generating adversarial examples and improving the model's robustness against interference. Specifically, this includes: In each training iteration, the input word embeddings are first calculated. Regarding the present gradient And based on gradient Construct the adversarial perturbation vector as follows:

[0040] In the formula, For adversarial perturbation vectors, This is the disturbance amplitude control coefficient. Represents the L2 norm; Subsequently, the word embeddings after superimposing adversarial perturbation vectors are... The input model undergoes forward propagation to calculate the adversarial loss, which is the word embeddings of the model after adding adversarial perturbation vectors. When used as input, the task loss between the predicted output and the labeled ground truth is calculated; and the model parameters are updated based on the gradient of the adversarial loss. Adversarial training improves the model's stability under local perturbations in the input space by actively seeking out and resisting the small perturbations that make the model most prone to prediction errors.

[0041] Example 4: Based on Example 3, in step 3, the unstructured text to be processed is input into the trained entity relation triple extraction model, and the encoder module first performs the following processing: Step 3.1: Perform word segmentation on the acquired unstructured text to obtain the corresponding input word sequence. ,in, Indicates the first i Each word element, The total number of lexical units is represented; lexical content embeddings and relative position embeddings are generated from the lexical sequence, and the two types of embeddings are superimposed to form the initial sequence encoding. The initial sequence encoding is fed into the pre-trained DeBERTa-v3-large model, and the text global context semantic encoding is completed through the multi-layer bidirectional Transformer encoder inside the model, and the text global semantic features are output. Step 3.2: To further enhance the local context modeling capability of the sequence, the feature sequence composed of global semantic features of the text is input into a BiLSTM bidirectional temporal network for supplementary modeling. The BiLSTM bidirectional temporal network extracts features from the feature sequence in both forward and backward directions, processes the bidirectional temporal dependencies between words, captures local contextual information, and outputs text sequence features that fuse global and local contextual information. ,in Indicates the first i Vector representation of each word in the context semantic space. Text sequence features. It integrates global semantic information and local temporal features, providing a unified semantic foundation for entity recognition and relationship prediction in the subsequent bidirectional interactive decoding layer.

[0042] By combining the global semantic modeling capabilities of the pre-trained DeBERTa-v3-large model with the local feature extraction capabilities of the BiLSTM bidirectional sequence network, it is possible to effectively capture deep semantic information in unstructured text, providing richer representations for subsequent entity and relation generation.

[0043] Next, the bidirectional interactive decoder receives the text sequence features output by the encoder module and outputs entities using an autoregressive sequence generation method. Relational triples: A single triple is generated sequentially in the order of head entity, tail entity, and relation type. Different triples are distinguished by a special delimiter, which is a preset identifier used to delineate the boundaries of adjacent triples in the autoregressive generation sequence.

[0044] The relation embedding pool stores all learnable embedding vectors of predefined relation types, represented as , Indicates the first A relation type, This represents the total number of predefined relation types; The entity memory is used to dynamically store the span representation vector and entity type information of entities generated during the decoding process, represented as: , Indicates the first A decoded entity, This indicates the number of entities that have been decoded so far.

[0045] The same input text shares the entity memory throughout the relation generation process and is reinitialized after text decoding is complete. The relation embedding pool is used to provide semantic priors of potential relations during the entity generation stage, while the entity memory is used to provide information support for identified entities during the relation prediction stage.

[0046] At each autoregressive generation time step, combined with Figure 1 and Figure 2 As shown, the bidirectional interactive decoder executes the following steps: Step 3.3: Represent the currently generated sequence using a self-attention layer. Perform feature modeling, where Indicates the first i One generated word element, t -1 represents the total number of tokens generated before the current time step. The self-attention output is calculated, and then the current decoding state is obtained through residual connections and layer normalization. Step 3.4: Input the current decoding state into the entity-relationship cross-attention layer, such as... Figure 3 As shown, the following processing will be performed based on the current generation stage: When the decoder is in the entity generation stage, based on the current decoding state For the query vector of cross-attention, the relation embedding pool Attention query is performed; the relation embedding vectors corresponding to each relation type stored in the relation embedding pool are mapped to form the query vector, key, and value in the entity generation stage:

[0047] In the formula, This is the learnable parameter matrix corresponding to the entity generation stage. This is the query vector for the entity generation phase. and These are the key matrix and value matrix obtained from relation embedding pool mapping, respectively; The cross-attention weights for semantic enhancement of relations are calculated and represented as follows:

[0048] In the formula, This is the cross-attention weight matrix for relational semantic enhancement. This represents the transpose of the key matrix obtained from the relation embedding pool mapping. Represents the hidden dimension of a vector; Use the Softmax activation function; After obtaining the context representation with enhanced relational semantics, the decoding state of the entity generation stage is obtained using a residual update method:

[0049] In the formula, This indicates the decoding state of the entity generation stage after relational semantic enhancement; Through this mechanism, the model no longer relies solely on local contextual information of the current location when generating entities. Instead, it can explicitly enhance the current state using latent relational semantics from the relation embedding pool. Since relation types themselves contain semantic connection patterns that may exist between entities, this interaction method can provide additional semantic references for entity boundary recognition and entity type determination. With the introduction of the relation embedding pool, the entity generation process can incorporate latent relational semantic information for state updates, thereby alleviating, to some extent, the ambiguity caused by relying solely on local context and improving the accuracy of entity recognition.

[0050] When the decoder is in the relation prediction phase, based on the current decoding state For the query vector of cross-attention, the entity memory is retrieved in reverse. The span representations of each generated entity stored in the entity memory, after mapping, constitute the query vector, key, and value for the relation prediction stage.

[0051] In the formula, This is the learnable parameter matrix corresponding to the relationship prediction stage. This is the query vector for the relation prediction phase. and These are the key matrix and value matrix obtained from mapping from the entity memory, respectively; The cross-attention weights for entity semantic enhancement are calculated and expressed as follows:

[0052] In the formula, Cross-attention weight matrix for entity semantic enhancement; This represents the transpose of the key matrix obtained by mapping from the entity memory; After obtaining the semantically enhanced contextual representation of the entities, the decoding state in the relation prediction stage is obtained using a residual update method:

[0053] In the formula, This represents the decoding state of the relation prediction stage after entity semantic enhancement. Through this reverse query, relation prediction no longer depends solely on the decoding state at the current position, but can explicitly utilize the contextual representation of the generated entities to enhance the modeling ability of the correspondence between the head and tail entities and the relation type.

[0054] Step 3.5: Input the enhanced decoding state obtained through the entity-relationship cross-attention layer into the feedforward network for feature transformation to obtain the output representation of the current time step; Step 3.6: The output representation is converted into the prediction score in the global output vocabulary space through the linear mapping layer. The candidate words are screened and their probabilities are adjusted by the structured prior constraint module. After Softmax normalization, the generation probability of the next word is obtained. The word with the highest probability is selected as the generation result of the current time step. Repeat steps 3.3 to 3.6 until the head entity, tail entity, and relation type are generated, forming a complete entity-relation triplet.

[0055] Although generative models can jointly extract entities and relations within a unified framework, their data-driven nature can lead to type-mismatched or structurally unreasonable triples in complex text scenarios. To address this, this invention proposes a structured prior constraint module to guide triple generation. Specifically, the structured prior constraint module includes a step-gated mask unit and a type constraint matrix unit; in step 3.6, after the linear mapping layer outputs the prediction score, the structured prior constraint module performs constraint processing on the prediction score, specifically including: The step-gated masking unit executes a step-gated decoding strategy. The essence of this strategy is to dynamically select the candidate space according to the generation stage, ensuring that the decoder can only select from the valid candidate range of the current stage at different generation stages. This effectively guarantees the consistency between the generation order and the joint extraction task structure, and reduces the invalid search space during the decoding process. Specifically, the decoder first constructs a global output vocabulary, which is formed by concatenating the entity candidate span set extracted from the input text with a predefined set of relation types. The entity candidate span set is a dynamically constructed span representation set after encoding all possible continuous segments in the input text.

[0056] Before each time step is generated, the step-gated masking unit constructs a mask vector of the same length as the global output vocabulary based on the current generation stage: When the decoder is in the entity generation stage, the index position of the corresponding relation type in the mask vector is assigned a value of negative infinity, and the index position of the corresponding entity candidate span is assigned a value of zero. When the decoder is in the relation prediction stage, the index position of the corresponding entity candidate span in the mask vector is assigned a value of negative infinity, and the index position of the corresponding relation type is assigned a value of zero. Subsequently, the mask vector and the original prediction score output by the linear mapping layer are subjected to tensor addition, and then normalized by Softmax. Due to the exponential property of the Softmax function, the generation probability of candidate positions assigned to negative infinity is zero, thereby physically shielding candidate words that are not in the current generation stage, so that the decoder can only select output from the legal candidate range in the current stage.

[0057] For example, given the input text "Steve Jobs founded Apple", all entity spans are: ["Jobs", "found", "found", "Apple", "Apple Inc."]. These are followed by predefined relation types: ["created", "employed", "located", "invested"]. The output vocabulary has a total length of 9, with the first 5 being entities and the last 4 being relation types. When generating entities, the constructed mask vector is [0, 0, 0, 0, 0, -∞, -∞, -∞, -∞]. The first 5 positions are 0 (allowed to be selected), and the last 4 are -∞ (masked). After tensor addition and Softmax normalization, the model can only select from the first 5 entity segments, such as selecting "Jobs", and will never select relation type tokens, thus ensuring the legality of the triple structure order at the generation position level.

[0058] Type constraint matrix elements only apply to the relation prediction stage, such as Figure 4 As shown, the process includes the following: First, construct the type constraint matrix. ,in, and These represent the number of entity types and the number of relationship types, respectively. The type constraint matrix is ​​a three-dimensional matrix, with the first dimension corresponding to the head entity type, the second dimension to the tail entity type, and the third dimension to the relationship type. The type constraint matrix is ​​constructed based on predefined entity type and relationship type constraint rules, and the head entity type is encoded by statistically analyzing the combination relationships of entity types in the training data. With tail entity type The prior scores for the rationality of each relation type under different combinations are calculated. Specifically, for each labeled entity relation triple in the training data, the co-occurrence frequency of its head entity type, tail entity type and relation type is calculated, and the co-occurrence frequency or its normalized value is used as the prior score of the corresponding position in the type constraint matrix; for entity type combinations and relation type pairings that have never appeared in the training data, the corresponding position is assigned a lower prior score or a zero value.

[0059] In the relationship prediction phase, based on the generated head entity types... Ending entity type From the predefined type constraint matrix The corresponding compatibility bias vector is obtained from the index. and compatibility bias vector The adjusted relationship prediction score is obtained by superimposing it onto the original relationship prediction score, as shown below:

[0060] In the formula, This is the adjusted score after adding compatibility bias. This represents the original prediction score corresponding to the relation type candidate output by the bidirectional interactive decoder after the linear mapping layer. (Compatibility bias vector) In this context, the relation positions that satisfy the entity type and relation type constraints are assigned a value of 0, while the relation positions that do not satisfy the constraints are assigned a negative value to reduce their prediction probability.

[0061] The bias-adjusted score Input the Softmax normalization function to obtain the probability distribution for each relation type. The relation type with the highest probability is selected as the relation prediction result for the current time step.

[0062] Through the above method, relation types incompatible with the current head and tail entity type combinations have smaller values ​​at corresponding positions in the compatibility bias vector, resulting in a negative bias and a lower prediction score. Conversely, relation types compatible with the current entity type combinations have larger values ​​at corresponding positions, leading to a higher prediction score. Compared to relying solely on textual context semantics for relation determination, this method further explicitly injects type knowledge into the relation generation process. This ensures that relation prediction is driven not only by semantic information but also by structural rules, effectively reducing the generation of illegal triples due to mismatched entity and relation types and improving the accuracy of relation extraction.

[0063] It should be noted that the type constraint matrix unit and the step-gated mask unit work together: the step-gated mask unit is responsible for physically masking candidate words that are not in the current stage at the generation stage level, while the type constraint matrix unit further refines the scores of each relation type based on entity type compatibility within the range of legal relation candidates after the step-gated mask screening. Together, they constitute structured prior constraints, which constrain the generation of triples layer by layer from coarse-grained stage selection to fine-grained type matching.

[0064] Example 5: Based on Example 4, step 4 specifically includes the following steps: Step 4.1: Perform normalization processing on the output entity-relation triplet set. This includes: standardizing the format of the entity names in the triples to eliminate inconsistencies in the text representation of the same entity caused by differences in abbreviations, symbols, and writing styles; merging duplicate triples with identical head entities, relations, and tail entities to ensure the uniqueness of the triplet set; and filtering out incomplete triples with missing entity or relation information to ensure that each retained triple contains all three elements: head entity, relation type, and tail entity. The final result is a standardized entity-relation triplet set.

[0065] Step 4.2: Map the standardized entity relation triple set to a graph structure: Map the head entity and tail entity in the triple to nodes in the knowledge graph, with each node storing the entity name, entity type, and source text identifier attributes; map the relations in the triple to directed edges from the head entity node to the tail entity node, and all nodes and edges together form a directed heterogeneous attribute graph with entities as nodes and semantic relations as edges.

[0066] like Figure 5 As shown, this knowledge graph structure uses circular nodes to represent entities, directed lines between nodes to represent semantic relationships between entities, and labels on the lines to indicate relationship types. For example, the directed edge between nodes "Oswald" and "Ruby" is labeled "Kill," indicating that there is a "Kill" relationship between entities "Oswald" and "Ruby." The graph includes various entity types such as people, locations, and organizations, as well as various relationship types such as Kill, Live_In, and Located_In, intuitively demonstrating the structural features of a directed heterogeneous attribute graph with entities as nodes and semantic relationships as edges.

[0067] Step 4.3: Use a graph database based on the attribute graph model as the storage engine. Organize entity nodes, node attributes, relation edges, and edge attributes according to preset mapping rules to realize the structured storage of the knowledge graph. Specifically, each node in the graph database corresponds to an entity and stores all attribute information of the entity; each edge corresponds to a relation and stores the type and related attributes of the relation; nodes and edges are associated through unique identifiers.

[0068] Graph databases based on attribute graph models can efficiently store knowledge graph data simply by inserting nodes and edges. They also support CRUD operations on the stored knowledge graph using graph query languages, enabling efficient execution of graph computation tasks such as multi-hop association queries, path analysis, and subgraph matching. The knowledge graph stored in the graph database provides a structured data foundation for downstream intelligent applications such as knowledge retrieval, association analysis, intelligent question answering, and recommendation systems.

[0069] Example 6: To verify the effectiveness of the method of this invention, experiments were conducted using the public dataset CoNLL04. CoNLL04 is a commonly used English public dataset for entity and relation extraction tasks, with its corpus derived from news texts, primarily designed for entity and relation extraction tasks in general domains. This dataset contains 1441 annotated texts, including entity types such as people, organizations, and locations, and relation types such as work and residence.

[0070] The performance of the entity-relation triplet extraction model in this invention is measured from different levels using three types of indicators: entity recognition, strict relation evaluation, and lenient relation evaluation.

[0071] (1) Entity Recognition (Ent). An entity is considered correctly identified only when both the predicted boundary and the entity type are correct.

[0072] (2) Relaxed Relation Evaluation (Rel). This indicator only requires that the boundaries of the two entities are correct and the relationship type is correct, but does not require that the entity types be correct.

[0073] (3) Strict Relation Evaluation (Rel+). A triple is considered a correct prediction only if the boundaries, entity types, and relation types of the two entities are all correct.

[0074] For the three indicators mentioned above, this invention uses accuracy. Recall rate and The values ​​serve as specific performance evaluation metrics for the model. The specific calculation methods for each metric are as follows:

[0075]

[0076]

[0077] In the formula, This represents the number of samples correctly identified as positive by the model. This represents the number of negative class samples that were misclassified as positive by the model. This represents the number of positive class samples that were misclassified as negative by the model.

[0078] Furthermore, this simulation experiment was programmed using Python 3.8 and implemented using the PyTorch 1.10.0 framework. The experiment used an NVIDIA A100 40GB GPU for training, with a batch size of 16, a learning rate of 2e-5, and 100 training epochs.

[0079] Experimental results To verify the effectiveness of the model of this invention, the dataset CoNLL04 was divided into training set, validation set and test set in a ratio of 7:2:1 for experimentation.

[0080] Six representative methods in the current field of joint entity relation extraction were selected as comparison models: SPERT, SPN4RE, HGERE, SINET, ATG, and SPIB. The model performance was evaluated using three metrics: entity recognition metric, relaxed relation evaluation metric, and strict relation evaluation metric.

[0081] Table 1 shows a performance comparison between the entity relation extraction graph construction method of this invention under prior constraints and the comparative method on the CoNLL04 dataset. The values ​​of the three indicators Ent, Rel, and Rel+ in the table are the corresponding indicators. value.

[0082] Table 1. Performance comparison of the method of this invention and the comparative model on the CoNLL04 dataset.

[0083] The data comparison in Table 1 reveals that, on the CoNLL04 dataset, the proposed method achieves 77.31% for the relaxed relation evaluation metric Rel and 76.94% for the strict relation evaluation metric Rel+, both exceeding the performance of other comparative models. Specifically, compared to the currently superior SPIB method, the proposed method improves the Rel and Rel+ metrics by 1.19% and 1.12%, respectively, indicating that the proposed bidirectional interactive decoder module effectively enhances the semantic association between entity and relation information, thereby improving the accuracy of relation prediction.

[0084] In terms of entity extraction, the entity recognition metric Ent of this invention achieves a result of 89.28%, which is basically equivalent to the 89.70% of the ATG method. Although slightly lower than the 90.89% of the SPIB method, it maintains a high overall level. This indicates that the method of this invention maintains good entity recognition ability while enhancing relation modeling capabilities. Compared with traditional generative joint extraction methods, this invention introduces bidirectional information interaction between entities and relations during the decoding stage, enabling the model to use relational semantics to assist in entity generation. At the same time, it combines structured prior constraints to reduce candidate results that do not conform to type rules, thereby improving the overall extraction effect of entity-relation triples.

[0085] In summary, the experimental results of the method in this invention on the public dataset CoNLL04 show that the proposed prior constraint-guided bidirectional interactive decoding mechanism can alleviate the problem of insufficient interaction between entity and relation information in the traditional autoregressive generation process to a certain extent, and improve the accuracy of generated triples through structured prior constraints, thus verifying the effectiveness of the method in the joint entity and relation extraction task.

Claims

1. A method for constructing an entity relation extraction graph under prior constraints and bidirectional interactive decoding, characterized in that, include: Step 1: Construct an entity relation triple extraction model; the entity relation triple extraction model includes an encoder module, a bidirectional interactive decoder module, and a structured prior constraint module; Step 2: Perform end-to-end parameter optimization on the entity relation triplet extraction model using R-Drop consistency regularization and FGM adversarial training strategy to obtain the trained entity relation triplet extraction model; Step 3: Input the unstructured text to be processed into the trained entity relation triple extraction model, and output a set of entity relation triples; Step 4: Normalize the set of entity relation triples, map and construct a knowledge graph, and store the knowledge graph in a graph database.

2. The entity relation extraction graph construction method under prior constraints and bidirectional interactive decoding according to claim 1, characterized in that, The encoder module includes a word segmentation unit, an embedding layer, a pre-trained language model encoder, and a BiLSTM bidirectional temporal network. The word segmentation unit is used to segment unstructured text into words. The embedding layer is used to generate word content embeddings and relative position embeddings, which are then superimposed to obtain the initial sequence encoding. The pre-trained language model encoder uses a DeBERTa-v3-large model with a built-in multi-layer bidirectional Transformer encoder to extract global semantic features of the text. The BiLSTM bidirectional temporal network is used to supplement local context and temporal dependency features, and output text sequence features. The bidirectional interactive decoder is composed of multiple stacked Transformer decoding layers, with an output layer at the end of the stack. Each Transformer decoding layer includes a self-attention layer, an entity-relation cross-attention layer, and a feedforward network layer. Furthermore, each Transformer decoding layer is configured with a relation embedding pool and an entity memory for bidirectional entity-relation cross-queries. The relation embedding pool stores all learnable relation embedding vectors of predefined relation types, and the entity memory is used to dynamically store the span representation vectors and entity type information of entities generated during the decoding process; the output layer includes a linear mapping layer and a Softmax normalization unit, and the linear mapping layer is used to map the hidden state output by the last decoding layer to the global output vocabulary space to obtain the original prediction score of each candidate word. The Softmax normalization unit is used to normalize the prediction score to a probability distribution on the vocabulary, and select the word with the highest probability as the generation result of the current time step. The structured prior constraint module includes a step-gating mask unit and a type constraint matrix unit; the step-gating mask unit is used to mask invalid prediction positions at different stages of decoding, and the type constraint matrix unit is used to constrain the legality of relation prediction based on the correspondence between entity type and relation type. The structured prior constraint module acts between the linear mapping layer and the Softmax normalization unit. After applying step-gated mask constraints and type constraint matrix constraints to the original prediction score output by the linear mapping layer, it is then input into the Softmax normalization unit.

3. The entity relation extraction graph construction method under prior constraints and bidirectional interactive decoding according to claim 2, characterized in that, In step 2, R-Drop consistency regularization is used to optimize the model parameters, specifically including: Applying two independent forward propagations to the same input text sequence, and leveraging the randomness of Dropout, yields two sets of sub-model output distributions, denoted as follows: and ,in, This represents the input text sequence. This represents the output sequence of the target triplet; Indicates the given input Time model prediction output The probability distribution; Based on the supervised loss of the conventional sequence generation task, a consistency regularization loss is defined by minimizing the bidirectional KL divergence between the output distributions of the two sub-models. , is represented as: In the formula, This represents the calculated consistency regularization loss. and This indicates the calculation of the KL divergence between the output distributions of the two sub-models; The total training loss of the entity relation triple extraction model is the weighted sum of the task supervision loss and the consistency regularization loss, expressed as: In the formula, This represents the total training loss of the model. This represents the supervised loss for a regular sequence generation task. The weighting coefficients for consistency regularization loss.

4. The entity relation extraction graph construction method under prior constraints and bidirectional interactive decoding according to claim 2, characterized in that, In step 2, FGM adversarial training is used in the embedding layer of the encoder module to optimize the model's execution parameters, specifically including: In each training iteration, the input word embeddings are first calculated. Regarding the present gradient And based on gradient Construct the adversarial perturbation vector as follows: In the formula, For the adversarial perturbation vector, This is the disturbance amplitude control coefficient. Represents the L2 norm; Subsequently, the word embeddings after superimposing adversarial perturbation vectors are... The input model undergoes forward propagation to calculate the adversarial loss, which is the word embeddings of the model after superimposing adversarial perturbation vectors. When used as input, the task loss between the predicted output and the labeled ground truth is calculated; and the model parameters are updated based on the gradient of the adversarial loss.

5. The entity relation extraction graph construction method under prior constraints and bidirectional interactive decoding according to claim 2, characterized in that, In step 3, the unstructured text to be processed is input into the trained entity relation triple extraction model. The encoder module first performs the following processing: Step 3.1: Perform word segmentation on the acquired unstructured text to obtain the corresponding input word sequence. ,in, Indicates the first i Each word element, The total number of lexical units is represented; lexical content embeddings and relative position embeddings are generated from the lexical sequence, and the two types of embeddings are superimposed to form the initial sequence encoding. The initial sequence encoding is fed into the pre-trained DeBERTa-v3-large model, and the text global context semantic encoding is completed through the multi-layer bidirectional Transformer encoder inside the model, and the text global semantic features are output. Step 3.2: Input the feature sequence composed of the global semantic features of the text into a BiLSTM bidirectional temporal network, extract features from the feature sequence from both the forward and reverse directions, process the bidirectional temporal dependencies between words, capture local contextual information, and output text sequence features that fuse global and local contextual information. ,in Indicates the first i The vector representation of each word in the context semantic space.

6. The entity relation extraction graph construction method under prior constraints and bidirectional interactive decoding according to claim 5, characterized in that, In step 3, the bidirectional interactive decoder receives the text sequence features output by the encoder module and outputs entities using an autoregressive sequence generation method. Relational triples; a single triple is generated sequentially in the order of head entity, tail entity, and relation type. Different triples are distinguished by a special delimiter, which is a preset identifier used to delineate the boundaries of adjacent triples in the autoregressive generation sequence. The relation embedding pool stores all learnable embedding vectors of predefined relation types, represented as follows: , Indicates the first A relation type, This represents the total number of predefined relation types; The entity memory is used to dynamically store the span representation vector and entity type information of entities generated during the decoding process, represented as follows: , Indicates the first A decoded entity, This indicates the number of entities that have been decoded so far. At each autoregressive generation time step, the bidirectional interactive decoder executes the following steps: Step 3.3: Model the features of the currently generated sequence representation through a self-attention layer, calculate the self-attention output, and then obtain the current decoding state through residual connections and layer normalization; Step 3.4: Input the current decoding state into the entity-relationship cross-attention layer, and perform processing according to the current generation stage: When the decoder is in the entity generation stage, based on the current decoding state For the query vector of cross-attention, the relation embedding pool Attention query is performed; the relation embedding vectors corresponding to each relation type stored in the relation embedding pool are mapped to form the query vector, key, and value in the entity generation stage: In the formula, This is the learnable parameter matrix corresponding to the entity generation stage. This is the query vector during the entity generation phase. and These are the key matrix and value matrix obtained from relation embedding pool mapping, respectively; The cross-attention weights for semantic enhancement of relations are calculated and represented as follows: In the formula, This is the cross-attention weight matrix for relational semantic enhancement. This represents the transpose of the key matrix obtained from the relation embedding pool mapping. Represents the hidden dimension of a vector; Use the Softmax activation function; The decoding state during the entity generation stage is obtained using a residual update method: In the formula, This indicates the decoding state of the entity generation stage after relational semantic enhancement; When the decoder is in the relation prediction phase, based on the current decoding state For the query vector of cross-attention, the entity memory is retrieved in reverse. ; The span representation vectors of each generated entity stored in the entity memory are mapped to form the query vector, key, and value in the relation prediction stage: In the formula, This is the learnable parameter matrix corresponding to the relationship prediction stage. This is the query vector for the relation prediction phase. and These are the key matrix and value matrix obtained from mapping from the entity memory, respectively; The cross-attention weights for entity semantic enhancement are calculated and represented as follows: In the formula, Cross-attention weight matrix for entity semantic enhancement; This represents the transpose of the key matrix obtained by mapping from the entity memory; The decoding state in the relation prediction stage is obtained using a residual update method: In the formula, This indicates the decoding state during the relation prediction stage after entity semantic enhancement; Step 3.5: Input the enhanced decoding state obtained through the entity-relationship cross-attention layer into the feedforward network for feature transformation to obtain the output representation of the current time step; Step 3.6: The output representation is converted into a prediction score in the global output vocabulary space through a linear mapping layer. The candidate words are screened and their probabilities are adjusted by the structured prior constraint module. After Softmax normalization, the generation probability of the next word is obtained. The word with the highest probability is selected as the generation result of the current time step. Repeat steps 3.3 to 3.6 until the head entity, tail entity, and relation type are generated, forming a complete entity-relation triplet.

7. The entity relation extraction graph construction method under prior constraints and bidirectional interactive decoding according to claim 6, characterized in that, The structured prior constraint module includes a step-gated mask unit and a type constraint matrix unit; in step 3.6, after the linear mapping layer outputs the prediction score, the structured prior constraint module performs constraint processing on the prediction score, specifically including: The step-gated masking unit executes a step-gated decoding strategy: It constructs a global output vocabulary, which is formed by concatenating the entity candidate span set extracted from the input text with a predefined set of relation types; the entity candidate span set is a dynamically constructed span representation set after encoding all possible continuous segments in the input text; before each time step, it constructs a mask vector of the same length as the global output vocabulary based on the current generation stage. When in the entity generation stage, the index position of the corresponding relation type in the mask vector is assigned a value of negative infinity, and the index position of the corresponding entity candidate span is assigned a value of zero. When in the relation prediction stage, the index position of the corresponding entity candidate span in the mask vector is assigned a value of negative infinity, and the index position of the corresponding relation type is assigned a value of zero; The mask vector is added to the original prediction score output by the linear mapping layer using tensor addition, and then normalized by Softmax to make the generation probability of candidate positions assigned a value of negative infinity zero, thereby physically masking candidate words that are not in the current generation stage.

8. The entity relation extraction graph construction method under prior constraints and bidirectional interactive decoding according to claim 7, characterized in that, The type constraint matrix unit only applies to the relation prediction stage: First, construct the type constraint matrix. ,in, and These represent the number of entity types and the number of relation types, respectively. The type constraint matrix is ​​constructed based on predefined entity type and relation type constraint rules, and the header entity type is encoded by statistically analyzing the combination relationships of entity types in the training data. With tail entity type The prior scores of the rationality of each relation type under different combinations; In the relationship prediction phase, based on the generated head entity types... Ending entity type From the predefined type constraint matrix The corresponding compatibility bias vector is obtained from the index. and compatibility bias vector The adjusted relationship prediction score is obtained by superimposing it onto the original relationship prediction score, as shown below: In the formula, This is the adjusted score after adding compatibility bias. This represents the original prediction score corresponding to the relation type candidate output by the bidirectional interactive decoder through the linear mapping layer.

9. The entity relation extraction graph construction method under prior constraints and bidirectional interactive decoding according to claim 1, characterized in that, Step 4 specifically includes the following steps: Step 4.1: Perform normalization processing on the output entity-relation triplet set, specifically including: standardizing the format of the entity name text of the triplet, eliminating differences in abbreviations, symbols, and writing formats; merging duplicate triplets with identical head entities, relations, and tail entities; filtering out incomplete triplets with missing entity or relation information to obtain a standardized entity-relation triplet set; Step 4.2: Map the standardized entity relation triple set to a graph structure: Map the head entity and tail entity in the triple to nodes in the knowledge graph, with each node storing the entity name, entity type, and source text identifier attributes; map the relations in the triple to directed edges from the head entity node to the tail entity node, and all nodes and edges together form a directed heterogeneous attribute graph with entities as nodes and semantic relations as edges; Step 4.3: Use a graph database based on the attribute graph model as the storage engine, organize entity nodes, node attributes, relationship edges and edge attributes according to preset mapping rules to realize the structured storage of the knowledge graph, support the CRUD operations on the knowledge graph through query language, and provide a data foundation for downstream knowledge retrieval, association analysis and intelligent applications.

Images