Relation extraction method, device and equipment based on depth prompt tuning

Abstract

The embodiment of the invention relates to the field of natural language processing, and discloses a relation extraction method, device and equipment based on depth prompt tuning, and the method comprises the steps: introducing an inference chain as a context prompt, and obtaining a learnable prompt vector; freezing parameters of the pre-training language model, and inserting the learnable prompt vector into each layer of the pre-training language model for adjustment and optimization; performing coding extraction on the original input text to obtain context sequence features; converting the context sequence features into a graph structure, and constructing a graph attention network model to output entity relationship features; and performing data enhancement processing on the entity relation characteristics based on a comparative learning mechanism to obtain relation characteristics after characteristic distribution optimization. According to the relationship extraction method disclosed by the invention, the problems of low model training efficiency and poor semantic representation capability are solved, so that the training efficiency is improved, the semantic relationship representation capability is enhanced, and the accuracy of a prediction result and the interpretability of the model are improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of natural language processing, and in particular to a method, apparatus and device for relation extraction based on deep prompting optimization. Background Technology

[0002] Relation extraction is one of the core technologies for constructing knowledge graphs, improving question-answering systems, and supporting downstream tasks such as information retrieval. With the development of deep learning, especially the widespread application of pre-trained language models (such as BERT and RoBERTa), neural network-based relation extraction methods have made significant progress.

[0003] Currently, there are two main approaches: fine-tuning-based pre-trained model methods and cue-based learning methods. Fine-tuning-based pre-trained model methods feed input text into pre-trained language models such as BERT or RoBERTa, learning the relation classification task by fine-tuning individual model parameters; typically, a classification layer is added to the output of the [CLS] tag for relation prediction. This approach fully utilizes the semantic understanding capabilities of the pre-trained model and performs well on standard datasets, but it requires updating all model parameters, resulting in high training costs. It is also prone to overfitting, especially in low-resource scenarios, and lacks explicit modeling of the inference process. Cue-based learning methods design artificial templates, transforming the relation classification task into a masked language modeling task. Simultaneously, learnable continuous cue vectors are introduced into the input layer, optimizing only these vectors while freezing the backbone model. This approach is parameter-efficient, suitable for low-resource scenarios, and better utilizes pre-trained knowledge; however, the cue usually only applies to the input layer and cannot guide the process in the deeper semantic space. Summary of the Invention

[0004] The purpose of this invention is to provide at least one method, apparatus, and device for relation extraction based on deep cue tuning, which can at least solve the problem of inaccurate prediction results caused by low model training efficiency and poor semantic representation ability in the prior art. It can at least improve training efficiency and enhance semantic relation representation ability, thereby improving the accuracy of prediction results and the interpretability of the model.

[0005] To address the aforementioned technical problems, at least one embodiment of this application provides a relationship extraction method based on depth-based prompting optimization, comprising: Introducing inference chains as contextual cues, and combining them with the original input text input into a pre-trained language model, yields learnable cue vectors for the pre-trained language model; The parameters of the pre-trained language model are frozen, and the learnable cue vectors are inserted into each layer of the pre-trained language model for optimization based on a deep cue tuning strategy, resulting in a tuned pre-trained language model. The original input text is encoded and extracted using the optimized pre-trained language model to obtain the context sequence features corresponding to the original text. The context sequence features are converted into a graph structure, and a graph attention network model is constructed based on the graph structure to capture long-range dependencies and dynamic interactions between entities, and output entity relationship features that integrate global interaction information. The entity relationship features are augmented using a contrastive learning mechanism to obtain relationship features with optimized feature distribution.

[0006] At least one embodiment of this application also provides a relationship extraction apparatus based on depth-cue tuning, comprising: The input module is used to introduce the inference chain as contextual cues, which are then used in conjunction with the original input text to input into the pre-trained language model to obtain the learnable cue vector of the pre-trained language model. The feature extraction module is used to freeze the parameters of the pre-trained language model and insert the learnable cue vectors into each layer of the pre-trained language model for optimization based on the deep cue tuning strategy to obtain the tuned pre-trained language model; and use the tuned pre-trained language model to encode and extract the original input text to obtain the context sequence features corresponding to the original text. The attention module is used to convert the context sequence features into a graph structure, and construct a graph attention network model based on the graph structure to capture long-range dependencies and dynamic interactions between entities, and output entity relationship features that integrate global interaction information. The data augmentation module is used to perform data augmentation processing on the entity relationship features based on a contrastive learning mechanism to obtain relationship features with optimized feature distribution.

[0007] At least one embodiment of this application also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described relation extraction method based on depth cue tuning.

[0008] At least one embodiment of this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described relation extraction method based on depth-hint tuning.

[0009] The embodiments of this application provide a relation extraction method, apparatus, and device based on deep cue optimization. By introducing inference chains as contextual cues, it provides interpretable reasoning basis for the model. Then, by freezing the parameters of the pre-trained language model, learnable cue vectors are inserted into each layer of the pre-trained language model for optimization. Only the learnable cue vectors are optimized, which significantly improves the model training efficiency and reduces resource consumption. Subsequently, the optimized pre-trained language model encodes the text and extracts semantically rich contextual features. Then, graph attention network modeling is introduced to capture long-range dependencies and dynamic interactions between entities. Combined with a contrastive learning mechanism, the relation representation ability is further enhanced, the semantic representation ability is enhanced, and the relation discriminativeness is improved, achieving a balance between high efficiency and high performance, thereby improving the accuracy of prediction results and the interpretability of the model.

[0010] In some optional embodiments, the method further includes: The semantic consistency detection framework is used to constrain the relation features after the feature distribution is optimized, so as to filter out the semantically unreasonable relation types and obtain the final relation features that are consistent with the logic of the original input text.

[0011] By using a semantic consistency detection framework to perform logical consistency verification and ranking of relation features after feature distribution optimization, and filtering out semantically unreasonable relation types, the accuracy and semantic rationality of prediction results can be further improved.

[0012] In some optional embodiments, the introduction of the inference chain as a contextual cue includes: Based on the analysis of real labels, a reasonable reasoning process for relation extraction tasks is obtained, and the relation extraction reasoning chain is obtained. The inference chain is used as a contextual cue, and analyzed in conjunction with the original input text input to the pre-trained language model to obtain the learnable cue vector of the pre-trained language model.

[0013] In some optional embodiments, converting the context sequence features into a graph structure includes: An adjacency matrix of entity nodes is constructed using edge indexes, and the context sequence features are converted into a graph structure using a dynamic adjacency matrix. The graph attention layer of the graph attention network model is constructed based on the graph structure.

[0014] In some optional embodiments, the adjacency matrix weight formula is:

[0015] In the formula, q is the total number of edges; Indicates the position of an element in the adjacency matrix; The normalized attention coefficients that incorporate the adjacency matrix weights are expressed as follows:

[0016] In the formula, M represents the adjacency matrix; LeakyReLU represents the nonlinear activation function; e su Indicates the importance of node u to node s; e sk This indicates the importance of node u to node k; N is the number of nodes; The output of the graph attention network model is represented as follows: ; In the formula, This represents a non-linear activation function, employing a multi-head attention mechanism and using the average value as the output; Z represents the number of heads in the attention mechanism; N represents the number of nodes; α su It is the normalized attention coefficient of node u to introduce the adjacency matrix weights to node s; W is the weight matrix; g u This represents the node features of the input node u.

[0017] In some optional embodiments, the data augmentation processing of the entity relationship features based on the contrastive learning mechanism includes: Determine the semantic representation of the relationship corresponding to the entity relationship feature based on the entity relationship feature; Based on the data augmentation strategy, the entity relationship features are perturbed to generate augmented features of the samples corresponding to the entity relationship features, and positive sample pairs are generated by combining the relationship semantic representation. Based on the positive examples, the relationships between other samples belonging to the same training batch as the positive examples are used to construct a set of negative examples within the batch; The positive sample pairs and the negative sample sets are subjected to contrastive loss analysis using the contrastive loss function to obtain the relational features after feature distribution optimization.

[0018] In some optional embodiments, the step of performing contrastive loss analysis on the positive sample pairs and the negative sample set using a contrastive loss function includes: The feature similarity of the positive samples is determined based on the positive samples, and the feature similarity of the positive samples is controlled based on a preset temperature coefficient to obtain the similarity of the positive sample pairs with a smooth distribution. The sum of similarities between the current sample and all enhanced features in the batch is determined based on the negative sample set to obtain the similarity of the negative sample set with suppressed similarity. The similarity between the positive sample pairs and the negative sample sets is calculated to obtain the maximum positive similarity ratio. The maximum positive similarity ratio is then solved based on the logarithmic function and the negative sign, guiding the graph attention network model to learn more discriminative relation features, and thus obtaining relation features with optimized feature distribution.

[0019] In some optional embodiments, a feature decoding module is also included, which is used to constrain the relation features after feature distribution optimization based on a semantic consistency detection framework, so as to filter out semantically unreasonable relation types and obtain the final relation features that are consistent with the logic of the original input text. Attached Figure Description

[0020] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, and these illustrative descriptions do not constitute a limitation on the embodiments.

[0021] Figure 1 This is a flowchart of a relationship extraction method based on depth-based prompting optimization provided in one embodiment of this application. Figure 1 ; Figure 2 This is a flowchart of a relationship extraction method based on depth-based prompting optimization provided in one embodiment of this application. Figure 2 ; Figure 3 This is a diagram of the depth-hint optimization structure in a relation extraction method based on depth-hint optimization provided in an embodiment of this application; Figure 4 This is a flowchart of a relationship extraction method based on depth-based prompting optimization provided in one embodiment of this application. Figure 3 ; Figure 5 The flowchart of the relationship extraction method based on depth prompting optimization provided in one embodiment of this application is shown in Figure 4. Figure 6 This is a diagram of the GAT (Graph Attention Network) structure in a relation extraction method based on deep cue tuning provided in one embodiment of this application; Figure 7 This is a flowchart of a relationship extraction method based on depth-based prompting optimization provided in one embodiment of this application. Figure 5 ; Figure 8 This is a flowchart of a relationship extraction method based on depth-based prompting optimization provided in one embodiment of this application. Figure 6 ; Figure 9 This is a schematic diagram of a relationship extraction device based on depth-cue tuning provided in another embodiment of this application; Figure 10 This is a flowchart illustrating the implementation of a relation extraction device algorithm based on depth-hint tuning, provided in another embodiment of this application. Figure 11 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application; Figure 12 This is a schematic diagram of the structure of a storage medium provided in another embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this application to help readers better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.

[0023] To facilitate understanding of the embodiments of this application, relevant content regarding relation extraction will be introduced first.

[0024] Relation extraction is one of the core technologies for constructing knowledge graphs, improving question-answering systems, and supporting downstream tasks such as information retrieval. With the development of deep learning, especially the widespread application of pre-trained language models (such as BERT and RoBERTa), neural network-based relation extraction methods have made significant progress. However, how to efficiently train models with limited labeled data, effectively model long-distance dependencies, improve the discriminative power of relation representations, and ensure the semantic rationality of prediction results remain key challenges.

[0025] Currently, there are two main approaches: fine-tuning-based pre-trained model methods and cue-based learning methods. Fine-tuning-based pre-trained model methods feed input text into pre-trained language models such as BERT or RoBERTa, learning relation classification tasks by fine-tuning individual model parameters; typically, a classification layer is added to the output of the [CLS] tag for relation prediction. This approach fully utilizes the semantic understanding capabilities of the pre-trained model and performs well on standard datasets, but it requires updating all model parameters, resulting in high training costs. It is also prone to overfitting, especially in low-resource scenarios, and lacks explicit modeling of the reasoning process. Cue-based learning methods design artificial templates, transforming the relation classification task into a masked language modeling task. Typically, learnable cue vectors are injected only into the input layer. Single-layer cueing is difficult to adapt to the feature extraction needs of different semantic levels; shallow layers focus on lexical and syntactic information, while deeper layers focus on semantics and reasoning. A single cue cannot adapt across layers, and the cue vector is gradually diluted during propagation, making it difficult for deep networks to perceive cue guidance.

[0026] To address the technical problems of inaccurate prediction results due to low model training efficiency and poor semantic representation capabilities in existing technologies, this invention proposes a relation extraction method based on deep cue optimization. The implementation details of the relation extraction method based on deep cue optimization in this embodiment are described below. The following content is only for ease of understanding and is not necessary for implementing this solution.

[0027] Example 1: The relation extraction method based on deep prompting optimization in this embodiment can be applied to electronic devices with communication, computing, and data storage capabilities. Its specific process can be as follows: Figure 1 As shown, it includes: Step 101: Introduce the inference chain as a contextual cue, and input it into the pre-trained language model along with the original input text to obtain the learnable cue vector of the pre-trained language model; Specifically, the inference chain constructed by multi-step derivation logic is transformed into a practical prompt text, enabling the model to extract relations according to a preset inference path. By introducing the inference chain as a contextual prompt during the input stage, and further combining it with the original input text input into the pre-trained language model, the inference chain can be used as part of the prompt to construct a learnable prompt vector, thereby improving the accuracy of relation extraction and the interpretability of the model.

[0028] In some examples, the pre-trained language model provided in this embodiment is a BERT-optimized pre-trained language model, namely RoBERTa (A Robustly Optimized BERTPretraining Approach) proposed by Facebook AI in 2019, which improves model performance by optimizing training strategies.

[0029] Step 102: Freeze the parameters of the pre-trained language model, and insert the learnable cue vectors into each layer of the pre-trained language model for optimization based on the deep cue tuning strategy to obtain the tuned pre-trained language model. Specifically, by freezing the backbone model parameters of the pre-trained language model, learnable cue vectors are injected into each layer of the pre-trained language model. The inserted learnable cue vectors are continuously optimized in each layer of the pre-trained language model, enabling the pre-trained language model to better complete downstream tasks, narrowing the performance gap with fine-tuning, and optimizing only the cue vectors, thereby significantly improving training efficiency and reducing resource consumption.

[0030] In some examples, deep suggestion tuning strategies are advanced solutions in the field of efficient fine-tuning of large language model parameters (PEFT). The core of this approach is to overcome the limitations of basic suggestion tuning, which only adds soft suggestions at the input layer. By embedding trainable suggestion information into the deep network of the model and optimizing the suggestion learning mechanism, the model's adaptability to complex downstream tasks is further improved while freezing the main parameters. Specifically, the structure of deep suggestion tuning is as follows: Figure 3 As shown in the diagram, the left area represents the multi-layered prompt input area (Layer 1 Prompts to Layer N Prompts). The yellow rectangles represent the prompt embeddings of different network layers, which are passed layer by layer after parameterization (an optional step) to form a hierarchical prompt structure. The right area represents the text encoding and classification area. Taking the sentence "Amazing movie!" as an example, the blue squares represent the embedding vectors (e (...)) of each token (such as [CLS], Amazing, movie, !). These vectors, along with the multi-layered prompt embeddings on the left, are input into the model, and finally, the class label is output through the linear head.

[0031] By decomposing prompts into multi-level embeddings and fusing them with text encoding, precise control over the model's intermediate layers is achieved. This overcomes the limitation of traditional prompt optimization, which only affects the input layer, and improves the performance of complex tasks such as text classification and sentiment analysis. The dashed box marking "Optimization" indicates that the entire process supports end-to-end optimization and allows for dynamic adjustment of prompt embedding parameters.

[0032] Step 103: Use the optimized pre-trained language model to encode and extract the original input text to obtain the context sequence features corresponding to the original text; Specifically, the optimized pre-trained language model is used to accurately extract the encoding and obtain the context sequence features, so as to extract entity relationships through the context sequence features. In this embodiment, the extracted context features are semantically rich context features to enhance the semantic relationship representation ability.

[0033] Step 104: Convert the context sequence features into a graph structure, and construct a graph attention network model based on the graph structure to capture long-range dependencies and dynamic interactions between entities, and output entity relationship features that integrate global interaction information. Specifically, by transforming sequence features into graph structures and introducing graph attention networks to model complex long-range dependencies and interactions between entities, the model can effectively model long-range dependencies and complex interactions, thereby improving the accuracy and semantic rationality of the model's prediction results.

[0034] In some cases, context sequence features are converted into graph structures by constructing dynamic adjacency matrices. Dynamic adjacency matrices are adjacency matrices that adaptively adjust as the model is trained or the input data changes. Their core function is to dynamically characterize the strength of associations between nodes, thus overcoming the limitation of static adjacency matrices in capturing complex interactions between entities.

[0035] Step 105: Perform data augmentation processing on the entity relationship features based on the contrastive learning mechanism to obtain relationship features with optimized feature distribution.

[0036] Specifically, the contrastive learning mechanism generates positive sample pairs through feature perturbation in key stages such as encoding and graph aggregation, and performs comparative optimization by combining negative samples within the batch, thereby further enhancing the data, improving semantic representation ability, and improving relation discriminability.

[0037] In some embodiments, after data augmentation processing, such as Figure 2 As shown, the method also includes: Step 106: Constrain the optimized relation features based on the semantic consistency detection framework to filter out semantically unreasonable relation types and obtain the final relation features that are consistent with the original input text logic.

[0038] Specifically, the semantic consistency detection framework, based on the idea of ​​natural language inference (NLI), judges whether the candidate relation features are consistent with the logic of the original text, thereby filtering out semantically unreasonable relation types and improving the accuracy and semantic rationality of the prediction results.

[0039] In some cases, the semantic consistency detection framework adopts the RECENT paradigm (Relation-aware Entailment and Consistency Testing): a semantic consistency detection framework used to verify the reasonableness of relation predictions, so as to accurately determine whether the candidate relation features are consistent with the original logic.

[0040] In this embodiment, an inference chain is introduced as a contextual cue to provide the model with an interpretable reasoning basis. Then, by freezing the parameters of the pre-trained language model, learnable cue vectors are inserted into each layer of the pre-trained language model for optimization. Optimizing only the learnable cue vectors significantly improves model training efficiency and reduces resource consumption. Subsequently, the optimized pre-trained language model encodes the text, extracting semantically rich contextual features. A graph attention network is then introduced to model and capture long-range dependencies and dynamic interactions between entities. A contrastive learning mechanism is then combined to further enhance relation representation capabilities, semantic representation capabilities, and relation discriminative ability, achieving a balance between high efficiency and high performance, thereby improving the accuracy of prediction results and the interpretability of the model. Simultaneously, a semantic consistency detection framework is used to perform logical consistency verification and ranking of the optimized relation features, filtering out semantically unreasonable relation types, which further improves the accuracy and semantic rationality of the prediction results.

[0041] In some embodiments, inference chains are introduced as contextual hints, which can be achieved through methods such as... Figure 4 The steps shown are implemented as follows: Step 201: Analyze the reasonable reasoning process of the relation extraction task based on real labels to obtain the relation extraction reasoning chain; Step 202: The inference chain is used as a contextual cue and analyzed in conjunction with the original input text input to the pre-trained language model to obtain the learnable cue vector of the pre-trained language model.

[0042] Specifically, to guide the model to focus on the relation extraction task and effectively leverage the knowledge inherent in the pre-trained language model, real labels are used to generate a reasonable reasoning process, which is then used as part of the prompts to construct structured input, thereby improving the model's interpretability and accuracy. The structured input is as follows:

[0043] [CLS] is a special classification label, and its corresponding final hidden layer state is used as the aggregate semantic representation of the entire input sequence. The decision for relation extraction is based on this vector. [SEP] is used to separate different semantic segments, helping the model to clearly identify the boundaries between the context and the prompt, thereby improving structural understanding and processing accuracy. context represents the original input text, and entity1 and entity2 represent entity mentions of the relations to be extracted.

[0044] Ground truth labels are accurate annotations of the true attributes of samples. They serve as the "target answer" learned during model training and the "gold standard" for evaluating model prediction performance, providing error reference for the model. In this embodiment, ground truth labels can be manually annotated by professional annotators (or annotation teams) according to task specifications; they can also be directly reused from existing structured data with accurate built-in labels, requiring no additional annotation; or ground truth labels can be generated in batches using preset rules or automated tools, suitable for scenarios with large amounts of data and well-defined rules; and the true attributes of the samples are objective facts that can be directly derived without manual judgment. Different sources of ground truth labels are selected based on different scenario requirements.

[0045] In some embodiments, the context sequence features are converted into a graph structure, such as... Figure 5 As shown, it includes: Step 301: Construct an adjacency matrix of entity nodes using edge indexes, and convert the context sequence features into a graph structure using a dynamic adjacency matrix; Step 302: Construct the graph attention layer of the graph attention network model based on the graph structure.

[0046] Specifically, by constructing a dynamic adjacency matrix, sequence features are transformed into a graph structure, and a graph attention network is introduced to model the complex long-range dependencies and interactions between entities.

[0047] In some examples, an edge information table of size (q, 3) is constructed, where q is the total number of edges. Each row of this table describes the start index, end index, and interaction weight values ​​of the adjacency matrix for a directed edge. An adjacency matrix M of entity nodes is constructed using the edge indices, assigning higher weights to more neighboring nodes in the sequence, with each node itself receiving the highest weight.

[0048] The adjacency matrix weight formula is:

[0049] In the formula, q is the total number of edges; Indicates the position of an element in the adjacency matrix; like Figure 6 The diagram shows a single-layer structure of the graph attention network (GAT) provided in this embodiment. Let the input of GAT be a set of node features. The output node features are , where N is the number of nodes and F is the number of features for each node.

[0050] Define the shared attention coefficient α and the weight matrix. If the vector concatenation symbol is ∥, then the importance of node u to node s is defined as:

[0051] Furthermore, the normalized attention coefficients, which incorporate the adjacency matrix weights, are expressed as:

[0052] In the formula, M represents the adjacency matrix; LeakyReLU represents the nonlinear activation function; e su Indicates the importance of node u to node s; e sk This indicates the importance of node u to node k; N is the number of nodes; The output of the graph attention network model is represented as follows: ; In the formula, This represents a non-linear activation function, employing a multi-head attention mechanism and using the average value as the output; Z represents the number of heads in the attention mechanism; N represents the number of nodes; α su It is the normalized attention coefficient of node u to introduce the adjacency matrix weights to node s; W is the weight matrix; g u This represents the node features of the input node u.

[0053] In some embodiments, data augmentation processing is performed on the entity relationship features based on a contrastive learning mechanism, such as... Figure 7 As shown, it includes: Step 401: Determine the semantic representation of the relationship corresponding to the entity relationship feature based on the entity relationship feature; Step 402: Based on the data augmentation strategy, perform feature perturbation on the entity relationship features to generate augmented features of the samples corresponding to the entity relationship features, and combine the relationship semantic representation to generate positive sample pairs; Step 403: Based on the positive sample, construct a set of negative samples within the batch by relating the positive sample to other samples belonging to the same training batch as the positive sample; Step 404: Perform contrastive loss analysis on the positive sample pairs and the negative sample set using the contrastive loss function to obtain the relational features after feature distribution optimization.

[0054] Specifically, for any sample, the entity relation features obtained through feature extraction are used to represent its relational semantics. Subsequently, a data augmentation strategy using dropout to randomly mask some feature dimensions was employed to perturb the features and generate augmented features for the sample. Simulates semantic variations of the same relation in different contexts to ensure that the generated enhanced features belong to the same category as the original features, forming a pair of positive sample pairs. .

[0055] In this embodiment, to achieve the goal of widening the distance between dissimilar samples, the enhanced features of all other samples within the same training batch are used as negative examples of the current sample. Assume the training batch size is... ,in This represents the number of labeled samples in the batch. If the number of unlabeled samples is zero, then for each positive pair of samples... The set of negative examples is These negative examples come from other samples within the batch and theoretically belong to different relationships or are not completely of the same type. The overlap of dissimilar features can be reduced through comparison optimization.

[0056] Furthermore, a contrastive loss function is used to perform contrastive loss analysis on the positive sample pairs and the negative sample set, such as... Figure 8 As shown, it includes: Step 501: Determine the feature similarity of the positive sample based on the positive sample, and control the feature similarity of the positive sample based on a preset temperature coefficient to obtain the similarity of the positive sample pairs with a smooth distribution; Step 502: Determine the sum of similarities between the current sample and all enhanced features in the batch based on the negative sample set, so as to obtain the similarity of the negative sample set with suppressed similarity. Step 503: Calculate the similarity between the positive sample pairs and the negative sample set to obtain the maximum positive similarity ratio. Solve the maximum positive similarity ratio based on the logarithmic function and the negative sign to guide the graph attention network model to learn relation features with higher discriminative power, and obtain relation features with optimized feature distribution.

[0057] Specifically, the loss function formula is as follows:

[0058] In the formula, the numerator part calculates the positive example. The feature similarity is divided by the temperature coefficient τ to control the smoothness of the similarity distribution, ensuring that the similarity of positive pairs is amplified under the exponential function; the denominator calculates the sum of the similarities between the current sample and all enhanced features in the batch, ensuring that the similarity of negative pairs is suppressed; the whole uses a logarithmic function and a negative sign to transform maximizing the proportion of positive similarity into a loss minimization problem, guiding the model to learn relational features with higher discriminative power.

[0059] In some embodiments, before inputting the data into the model, two key pieces of information, entity mentions and entity types, are first extracted from the original text. Based on the training data, a candidate relation mapping table is constructed by statistically analyzing the actual relationships that occur under each (ts,to) type (ts: Source, representing the source entity / node of the relation (e.g., the acquirer in the "acquisition" relation); to: Target, representing the target entity / node of the relation). Let R represent the set of all possible relations, S(r) represent the set of allowed subject entity types for relation r, and O(r) represent the set of allowed object entity types for relation r. Instead of using a general classifier to handle all relations, a separate classifier is trained for each (ts,to) type pair. This allows for the filtering of semantically unreasonable relation types when the relation features optimized by the semantic consistency detection framework are constrained.

[0060] The relation extraction method provided in this embodiment has the following significant advantages compared to the prior art: First, it significantly reduces training costs and improves model training efficiency. A deep cueing optimization strategy is adopted, optimizing only a small number of learnable cue vectors and freezing the backbone RoBERTa model parameters, thus drastically reducing the number of trainable parameters (typically by more than 90%). Second, enhance semantic representation capabilities and improve relation discriminative ability. A three-stage contrastive learning mechanism is introduced, which generates positive sample pairs through feature perturbation in key stages such as encoding and graph aggregation, and combines them with negative samples within the batch for comparative optimization; Third, it effectively models long-range dependencies and complex interactions. Sequence features are transformed into a graph structure by constructing a dynamic adjacency matrix, and a graph attention network (GAT) is introduced for message passing. Fourth, improve the accuracy and semantic rationality of prediction results. The RECENT paradigm is introduced to perform logical consistency checks and ranking of candidate relations, filtering out semantically unreasonable relation types; Fifth, it achieves a balance between efficiency and high performance while maintaining interpretability. The cue vector is introduced as an explicit inference chain into the input stage, providing a traceable reasoning basis for model decisions.

[0061] Example 2: Another embodiment of this application relates to a relation extraction device based on depth-hint optimization. The implementation details of this relation extraction device based on depth-hint optimization are described below. The following details are provided for ease of understanding and are not essential for implementing this solution. A schematic diagram of the relation extraction device based on depth-hint optimization in this embodiment can be seen as follows: Figure 9 and Figure 10 As shown, it includes an input module 801, a feature extraction module 802, an attention module 803, and a data augmentation module 804.

[0062] The input module 801 is used to introduce the inference chain as a contextual cue, which is then used in conjunction with the original input text to input the pre-trained language model to obtain the learnable cue vector of the pre-trained language model. The feature extraction module 802 is used to freeze the parameters of the pre-trained language model, and insert the learnable cue vector into each layer of the pre-trained language model for optimization based on the deep cue tuning strategy to obtain the tuned pre-trained language model; and use the tuned pre-trained language model to encode and extract the original input text to obtain the context sequence features corresponding to the original text. Attention module 803 is used to convert the context sequence features into a graph structure, and construct a graph attention network model based on the graph structure to capture long-range dependencies and dynamic interactions between entities, and output entity relationship features that integrate global interaction information. The data augmentation module 804 is used to perform data augmentation processing on the entity relationship features based on the contrastive learning mechanism to obtain relationship features with optimized feature distribution.

[0063] The output of the input module 801 is connected to the input of the feature extraction module 802, the output of the feature extraction module 802 is connected to the input of the attention module 803, and the output of the attention module 803 is connected to the input of the data augmentation module 804.

[0064] In some embodiments, such as Figure 9 As shown, the relation extraction device based on deep prompting optimization also includes a feature decoding module 805. The feature decoding module 805 is used to constrain the relation features after feature distribution optimization based on a semantic consistency detection framework, in order to filter out semantically unreasonable relation types and obtain the final relation features that are logically consistent with the original input text. The output of the data augmentation module 804 is connected to the input of the feature decoding module 805.

[0065] In some embodiments, the input module 801 is further configured to analyze the reasonable reasoning process of the relation extraction task based on real labels to obtain the relation extraction reasoning chain; and to use the reasoning chain as a contextual cue, and to analyze it in conjunction with the original input text input to the pre-trained language model to obtain the learnable cue vector of the pre-trained language model.

[0066] In some embodiments, the attention module 803 is further configured to construct an adjacency matrix of entity nodes through edge indexing, and convert the context sequence features into a graph structure through a dynamic adjacency matrix; and construct a graph attention layer of the graph attention network model based on the graph structure.

[0067] In some embodiments, the adjacency matrix weight formula is:

[0068] In the formula, q is the total number of edges; Indicates the position of an element in the adjacency matrix; The normalized attention coefficients that incorporate the adjacency matrix weights are expressed as follows:

[0069] In the formula, M represents the adjacency matrix; LeakyReLU represents the nonlinear activation function; e su Indicates the importance of node u to node s; e sk This indicates the importance of node u to node k; N is the number of nodes; The output of the graph attention network model is represented as follows: ; In the formula, This represents a non-linear activation function, employing a multi-head attention mechanism and using the average value as the output; Z represents the number of heads in the attention mechanism; N represents the number of nodes; α su It is the normalized attention coefficient of node u to introduce the adjacency matrix weights to node s; W is the weight matrix; g u This represents the node features of the input node u.

[0070] In some embodiments, the data enhancement module 804 is further configured to determine the relation semantic representation corresponding to the entity relation features based on the entity relation features; Based on the data augmentation strategy, the entity relationship features are perturbed to generate augmented features of the samples corresponding to the entity relationship features, and positive sample pairs are generated by combining the relationship semantic representation. Based on the positive examples, the relationships between other samples belonging to the same training batch as the positive examples are used to construct a set of negative examples within the batch; The positive sample pairs and the negative sample sets are subjected to contrastive loss analysis using the contrastive loss function to obtain the relational features after feature distribution optimization.

[0071] In some embodiments, the data augmentation module 804 is further configured to determine the feature similarity of positive samples based on the positive samples, and control the feature similarity of the positive samples based on a preset temperature coefficient to obtain a smooth distribution of positive sample pair similarity. The sum of similarities between the current sample and all enhanced features in the batch is determined based on the negative sample set to obtain the similarity of the negative sample set with suppressed similarity. The similarity between the positive sample pairs and the negative sample sets is calculated to obtain the maximum positive similarity ratio. The maximum positive similarity ratio is then solved based on the logarithmic function and the negative sign, guiding the graph attention network model to learn more discriminative relation features, and thus obtaining relation features with optimized feature distribution.

[0072] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed in this application; however, this does not mean that other units are absent in this embodiment.

[0073] Example 3: Another embodiment of this application relates to an electronic device, such as... Figure 11 As shown, it includes: at least one processor 901; and a memory 902 communicatively connected to the at least one processor 901; wherein the memory 902 stores instructions executable by the at least one processor 901, the instructions being executed by the at least one processor 901 to enable the at least one processor 901 to execute the relationship extraction method based on depth hint tuning in the above embodiments.

[0074] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0075] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0076] Example 4: Another embodiment of this application relates to a computer-readable storage medium, such as... Figure 12 As shown, a computer program 31 is stored. When the computer program 31 is executed by the processor, it implements the above-described method embodiment.

[0077] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0078] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.

Claims

1. A relation extraction method based on depth-based prompting optimization, characterized in that, include: Introducing inference chains as contextual cues, and combining them with the original input text input into a pre-trained language model, yields learnable cue vectors for the pre-trained language model; The parameters of the pre-trained language model are frozen, and the learnable cue vectors are inserted into each layer of the pre-trained language model for optimization based on a deep cue tuning strategy, resulting in a tuned pre-trained language model. The original input text is encoded and extracted using the optimized pre-trained language model to obtain the context sequence features corresponding to the original text. The context sequence features are converted into a graph structure, and a graph attention network model is constructed based on the graph structure to capture long-range dependencies and dynamic interactions between entities, and output entity relationship features that integrate global interaction information. The entity relationship features are augmented using a contrastive learning mechanism to obtain relationship features with optimized feature distribution.

2. The relationship extraction method based on depth-based prompting optimization according to claim 1, characterized in that, The method further includes: The semantic consistency detection framework is used to constrain the relation features after the feature distribution is optimized, so as to filter out the semantically unreasonable relation types and obtain the final relation features that are consistent with the logic of the original input text.

3. The relationship extraction method based on depth-based prompting optimization according to claim 1, characterized in that, The introduction of inference chains as contextual hints includes: Based on the analysis of real labels, a reasonable reasoning process for relation extraction tasks is obtained, and the relation extraction reasoning chain is obtained. The inference chain is used as a contextual cue, and analyzed in conjunction with the original input text input to the pre-trained language model to obtain the learnable cue vector of the pre-trained language model.

4. The relationship extraction method based on depth-based prompting optimization according to claim 1, characterized in that, The step of converting the context sequence features into a graph structure includes: An adjacency matrix of entity nodes is constructed using edge indexes, and the context sequence features are converted into a graph structure using a dynamic adjacency matrix. The graph attention layer of the graph attention network model is constructed based on the graph structure.

5. The relationship extraction method based on depth-based prompting optimization according to claim 4, characterized in that, The adjacency matrix weight formula is as follows: In the formula, q is the total number of edges; Indicates the position of an element in the adjacency matrix; The normalized attention coefficients that incorporate the adjacency matrix weights are expressed as follows: In the formula, M represents the adjacency matrix; LeakyReLU represents the nonlinear activation function; e su Indicates the importance of node u to node s; e sk This indicates the importance of node u to node k; N is the number of nodes; The output of the graph attention network model is represented as follows: ； In the formula, This represents a non-linear activation function, employing a multi-head attention mechanism and using the average value as the output; Z represents the number of heads in the attention mechanism; N represents the number of nodes; α su It is the normalized attention coefficient that introduces the adjacency matrix weights of node u to node s; W is the weight matrix; g u This represents the node features of the input node u.

6. The relationship extraction method based on depth-based prompting optimization according to claim 1, characterized in that, The data augmentation processing of the entity relationship features based on the contrastive learning mechanism includes: Determine the semantic representation of the relationship corresponding to the entity relationship feature based on the entity relationship feature; Based on the data augmentation strategy, the entity relationship features are perturbed to generate augmented features of the samples corresponding to the entity relationship features, and positive sample pairs are generated by combining the relationship semantic representation. Based on the positive examples, the relationships between other samples belonging to the same training batch as the positive examples are used to construct a set of negative examples within the batch; The contrastive loss function is used to perform contrastive loss analysis on the positive sample pairs and the negative sample set to obtain the relational features after feature distribution optimization.

7. The relationship extraction method based on depth-based prompting optimization according to claim 6, characterized in that, The step of performing contrastive loss analysis on the positive sample pairs and the negative sample set using a contrastive loss function includes: The feature similarity of the positive samples is determined based on the positive samples, and the feature similarity of the positive samples is controlled based on a preset temperature coefficient to obtain a smooth distribution of positive sample pair similarity. The sum of similarities between the current sample and all enhanced features in the batch is determined based on the negative sample set to obtain the similarity of the negative sample set with suppressed similarity. The similarity between the positive sample pairs and the negative sample sets is calculated to obtain the maximum positive similarity ratio. The maximum positive similarity ratio is then solved based on the logarithmic function and the negative sign, guiding the graph attention network model to learn more discriminative relation features, and thus obtaining relation features with optimized feature distribution.

8. A relationship extraction device based on depth-based cueing optimization, characterized in that, include: The input module is used to introduce the inference chain as contextual cues, which are then used in conjunction with the original input text to input into the pre-trained language model to obtain the learnable cue vector of the pre-trained language model. The feature extraction module is used to freeze the parameters of the pre-trained language model and insert the learnable cue vectors into each layer of the pre-trained language model for optimization based on the deep cue tuning strategy to obtain the tuned pre-trained language model; and use the tuned pre-trained language model to encode and extract the original input text to obtain the context sequence features corresponding to the original text. The attention module is used to convert the context sequence features into a graph structure, and construct a graph attention network model based on the graph structure to capture long-range dependencies and dynamic interactions between entities, and output entity relationship features that integrate global interaction information. The data augmentation module is used to perform data augmentation processing on the entity relationship features based on a contrastive learning mechanism to obtain relationship features with optimized feature distribution.

9. A relationship extraction device based on depth-cue optimization according to claim 8, characterized in that, It also includes a feature decoding module, which is used to constrain the relation features after the feature distribution is optimized based on the semantic consistency detection framework, so as to filter out the semantically unreasonable relation types and obtain the final relation features that are consistent with the logic of the original input text.

10. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the relationship extraction method based on depth hint tuning as described in any one of claims 1 to 7.

Images