A document-level relation extraction method fusing three-dimensional sparse semantics and multi-head expert mechanism
Patent Information
- Application Number
- CN202610760420.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-29
- Publication Date
- 2026-08-18
AI Technical Summary
尽管这些方法取得了显著进展，但在处理长文档时仍面临困境：基于图神经网络的方法计算复杂度高，且性能高度依赖外部工具构建图结构的精度；而基于语义增强的方法则受限于模型的输入长度，在长文档中容易出现注意力稀释，导致跨句证据的表达不完整，影响关系推理的稳定性
[0059] This invention provides a document-level relation extraction method based on three-dimensional sparse semantic enhancement and a multi-head expert mechanism. It utilizes a lightweight relation pre-filtering module combined with a dynamic threshold strategy to compress the candidate space while ensuring high recall, thus alleviating sample imbalance. A three-dimensional sparse semantic enhancement module is constructed to explicitly model the interaction between entities and relations, accurately locating evidence sentences to address the attention dilution problem in long documents. A multi-head heterogeneous low-rank expert module is introduced, enabling efficient parameter fine-tuning, significantly reducing training costs, and improving the stability and accuracy of relation extraction.
Smart Images
Figure CN122596058A_ABST
Abstract
Description
Technical Field
[0001] This invention discloses a document-level relation extraction method that integrates three-dimensional sparse semantics and multi-head expert mechanism, which relates to the fields of natural language processing, knowledge engineering and information extraction technology. Background Technology
[0002] Document-level relation extraction aims to identify semantic relationships between entities from complete documents and is a core foundational technology for building knowledge graphs, supporting intelligent question answering, and professional information retrieval. With the rapid development of information technology, massive amounts of text data place higher demands on automated knowledge extraction. Unlike traditional sentence-level relation extraction, document-level relation extraction faces severe challenges such as evidence scattered across multiple sentences, difficulty in modeling long-range dependencies, a large number of candidate entity pairs, and a high proportion of negative examples (unrelated entity pairs).
[0003] In recent years, researchers have explored two main technical paths to address these challenges. On one hand, graph neural network-based methods explicitly construct graphs of relationships between entities, mentions, and sentences within a document, transforming the reasoning process into information propagation on a graph. For example, the EoG model uses entity mentions and sentences as nodes, employing graph convolutional networks for relation classification. On the other hand, semantic feature enhancement methods based on pre-trained language models, such as ATLOP and SSAN, improve self-attention mechanisms or introduce entity mention bias terms, enabling the model to better focus on key evidence. Despite these significant advancements, challenges remain when processing long documents: graph neural network-based methods have high computational complexity and their performance is highly dependent on the accuracy of external tools constructing the graph structure; while semantic enhancement methods are limited by the model's input length, easily leading to attention dilution in long documents, resulting in incomplete expression of cross-sentence evidence and affecting the stability of relational reasoning. Therefore, how to efficiently and accurately model scattered evidence in long documents while controlling computational overhead remains a core problem to be solved in the field of document-level relation extraction. Summary of the Invention
[0004] To address the aforementioned challenges, this invention proposes a document-level relation extraction method based on three-dimensional sparse semantic enhancement and a multi-head expert mechanism. The aim is to improve the stability and accuracy of document-level relation extraction through structured semantic enhancement and efficient parameter fine-tuning.
[0005] The document-level relation extraction method of this invention, which integrates three-dimensional sparse semantics and multi-head expert mechanism, includes the following steps:
[0006] (1) Enumerate all possible entity pairs based on the pre-extracted entity list provided by the dataset to form an initial candidate entity pair set;
[0007] (2) The candidate entity pairs in the initial candidate entity pair set are filtered by the candidate entity pair pre-screening module based on the semantic interaction strength between entities, and unrelated entity pairs are eliminated to obtain the filtered candidate entity pair set.
[0008] (3) Using the three-dimensional sparse semantic enhancement module, construct a three-dimensional semantic embedding for each entity pair in the filtered candidate entity pair set, locate key evidence sentences based on the three-dimensional semantic embedding, and generate structured key prompt information;
[0009] (4) By using a multi-head heterogeneous low-rank expert module, while keeping the main parameters of the pre-trained language model frozen, the outputs of multiple low-rank adapter sub-modules are dynamically fused through a gating mechanism based on the key evidence sentences and the structured key prompt information to classify the candidate entity pairs.
[0010] Furthermore, the implementation process of step (1) is as follows:
[0011] The dataset is trained and tested using the DocRED and Re-DocRED datasets. All possible entity pairs are enumerated based on the pre-extracted entity lists provided by the DocRED and Re-DocRED datasets, and an initial semantic embedding vector is generated for each entity using a pre-trained language model.
[0012] Furthermore, the implementation process of step (2) is as follows:
[0013] Based on the multi-head attention mechanism, the head entity is calculated. With tail entity The degree of overlap in attention distributions across multiple attention heads yields a joint contextual semantic representation:
[0014] (1)
[0015] in, For the total number of attention heads, and Representing the head entity respectively With tail entity The attention distribution vector at the t-th attention head. This represents an element-wise multiplication operation;
[0016] The joint context representation is fused with the initial semantic embedding vector of the entity to obtain the enhanced entity pair representation:
[0017] (2)
[0018] in, For entities The initial semantic embedding vector, and The learnable parameter matrix;
[0019] Based on the enhanced entity pair representation, the probability that an entity pair is unrelated is calculated using a bilinear transformation:
[0020] (3)
[0021] in It is a bilinear weight matrix. For bias terms, It is the sigmoid function;
[0022] Remove unrelated entity pairs from the candidate set using dynamic thresholds:
[0023] (4)
[0024] in, and These represent the number of negative and positive samples in the current training batch, respectively. It is a very small constant.
[0025] Furthermore, the method for constructing the three-dimensional semantic embedding in step (3) is as follows:
[0026] According to the head entity With tail entity and predefined relationship types Constructing 3D Embedding Each element Represents head entity With tail entity In relation types The strength of semantic interaction below:
[0027] (5)
[0028] in, and These are the semantic vectors of the head and tail entities, respectively. For relation type The prototype vector, GeLU is the Gaussian error linear unit activation function, and d is the vector dimension.
[0029] Furthermore, the method for locating key evidence sentences based on the three-dimensional semantic embedding in step (3) is as follows:
[0030] For each candidate entity pair, a relevance score is calculated for each sentence in the document based on the three-dimensional semantic embedding. :
[0031] (6)
[0032] in, and For sentences Middle head entity Tail-end entity word frequency For sentences Position number in the document The core sentence position of the entity pair is identified; and several sentences are selected as key evidence sentences according to the aforementioned relevance score, from high to low:
[0033] (7).
[0034] Furthermore, the structured key prompt information in step (3) is generated in the following way:
[0035] For candidate entity pairs First, the matching score vector for each relation type is calculated based on the three-dimensional semantic embedding, where the k-th component is calculated as follows:
[0036] (8)
[0037] By introducing temperature-regulated softmax normalization, the matching score vector is transformed into a probability distribution:
[0038] (9)
[0039] Where T is the temperature hyperparameter; the top three relation types with the highest probability values and their corresponding probability values are selected, and structured key prompt information is generated according to the preset prompt template. The structured key prompt information includes entity pair identifiers, relation type names and their matching probabilities.
[0040] Furthermore, the implementation process of step (4) is as follows:
[0041] While keeping the main parameters of the pre-trained language model frozen, the multi-head heterogeneous low-rank expert module includes multiple low-rank adapter submodules and an evidence-enhancing gating network; the low-rank adapter submodules are used to learn parameter updates for specific domains or relational patterns in a low-dimensional subspace, and their output is:
[0042] (10)
[0043] in, For the first Input to the layer network, For the kth, For the shared output mapping matrix, The gating network is then used to dynamically calculate the fusion weights of each low-rank adapter submodule based on the input semantics, the key evidence sentences, and the structured key prompt information.
[0044] Furthermore, the process of calculating the fusion weights by the gated network is as follows:
[0045] The process of calculating the fusion weights in the gated network is as follows: The input vector of the current layer of the large language model is... The semantic encoding vector of the key evidence sentence and the semantic encoding vector of the structured key prompt information. The weights of each low-rank adapter submodule are concatenated and generated using a fully connected layer and a Softmax function:
[0046] (11)
[0047] in, , , The projection matrix is learnable. This represents a vector concatenation operation. For the gated parameter matrix, This is a bias term.
[0048] Furthermore, the training method for the multi-head heterogeneous low-rank expert module is as follows:
[0049] When training the multi-head heterogeneous low-rank expert module, an expert load balancing loss is introduced:
[0050] (12)
[0051] The module's total loss function is a weighted sum of the relation classification loss and the load balancing loss:
[0052] (13)
[0053] in, For relationship classification loss, To balance the hyperparameters.
[0054] Furthermore, the final output of the relation classification is as follows:
[0055] The final output of the relation classification is a weighted sum of the output of the main network of the large language model and the outputs of each low-rank adapter submodule. :
[0056] (14)
[0057] in, This represents the general semantic features provided by the pre-trained model. This represents the domain-specific features generated by the expert module.
[0058] Compared with the prior art, the technical solution implemented in this invention has the following beneficial effects:
[0059] This invention provides a document-level relation extraction method based on three-dimensional sparse semantic enhancement and a multi-head expert mechanism. It utilizes a lightweight relation pre-filtering module combined with a dynamic threshold strategy to compress the candidate space while ensuring high recall, thus alleviating sample imbalance. A three-dimensional sparse semantic enhancement module is constructed to explicitly model the interaction between entities and relations, accurately locating evidence sentences to address the attention dilution problem in long documents. A multi-head heterogeneous low-rank expert module is introduced, enabling efficient parameter fine-tuning, significantly reducing training costs, and improving the stability and accuracy of relation extraction. Attached Figure Description
[0060] Figure 1 This is a flowchart of a document-level relation extraction method that integrates three-dimensional sparse semantics and multi-head expert mechanism according to the present invention;
[0061] Figure 2 This is a flowchart of the three-dimensional sparse semantic enhancement module of the present invention;
[0062] Figure 3 This is a flowchart of the multi-head heterogeneous low-rank expert module of the present invention;
[0063] Figure 4 This is a diagram illustrating the structured key prompts in this invention. Detailed Implementation
[0064] The present invention will be further described below.
[0065] (1) Enumerate all possible entity pairs based on the pre-extracted entity list provided by the dataset to form an initial candidate entity pair set.
[0066] The datasets used DocRED and Re-DocRED for training and testing. The DocRED dataset is a benchmark dataset for document-level relation extraction based on Wikipedia articles, covering 97 relation types, with an average of 26 entities per document. The Re-DocRED dataset is an optimized version that corrects the shortcomings of DocRED annotation. Through multiple rounds of cross-validation and supplementation of missing relations, it significantly reduced the false negative rate in the original dataset, improved the accuracy of data annotation, and provided higher-quality dataset support for model training.
[0067] First, based on the pre-extracted entity list provided by the dataset, all possible entity pairs are enumerated to form an initial candidate entity pair set.
[0068] (2) The candidate entity pairs in the initial candidate entity pair set are filtered by the candidate entity pair pre-screening module based on the semantic interaction strength between entities, and unrelated entity pairs are eliminated to obtain the filtered candidate entity pair set.
[0069] The document is encoded using a pre-trained language model to obtain the initial semantic embedding vector for each entity. This invention introduces a joint context representation mechanism based on multi-head attention, and then calculates the head entity based on the multi-head attention mechanism. With tail entity The degree of overlap in the attention distributions across multiple attention heads yields a joint semantic representation:
[0070] (1)
[0071] in For the total number of attention heads, and Representing the head entity respectively With tail entity The attention distribution vector at the t-th attention head. This indicates an element-wise multiplication operation.
[0072] The entity semantic representation is enhanced by a learnable nonlinear transformation layer: the joint context representation is fused with the initial semantic embedding vector of the entity to obtain the enhanced entity pair representation.
[0073] (2)
[0074] in and For learnable parameter matrix, This is the hyperbolic tangent activation function. This transformation effectively fuses the static semantic information of an entity with its dynamic interaction information in a specific context, generating a more discriminative entity pair representation.
[0075] Based on the enhanced entity pair representation, a bilinear transformation is used to calculate the probability that an entity pair belongs to the "no relation" category.
[0076] (3)
[0077] in It is a bilinear weight matrix. For bias terms, This is the sigmoid function. This bilinear interaction mechanism can capture the structured compatibility of head and tail entities in the relational semantic space, providing a reliable basis for determining the existence of relations.
[0078] To address the problem of a severely imbalanced ratio of positive to negative samples in document-level relation extraction, this invention designs a dynamic threshold adjustment mechanism based on sample distribution awareness:
[0079] (4)
[0080] in and These represent the number of negative and positive samples in the current training batch, respectively. This is a minimal constant. This threshold is dynamically adjusted based on the actual sample distribution. > When this happens, the corresponding entity pairs are determined to be unrelated and removed from the candidate set; otherwise, they are retained for subsequent processing. After processing by this module, the number of candidate entity pairs increases from the original number. Reduced to ,in This significantly reduces the computational complexity of subsequent modules, while a carefully designed semantic interaction mechanism ensures the recall rate of high-confidence relation candidates.
[0081] (3) Using the three-dimensional sparse semantic enhancement module, construct a three-dimensional semantic embedding for each entity pair in the filtered candidate entity pair set, locate key evidence sentences based on the three-dimensional semantic embedding, and generate structured key prompt information.
[0082] Let the number of header entities retained after the pre-filtering module be... The number of tail entities is The number of predefined relation types is To characterize entity interaction patterns from different relational semantic perspectives, a three-dimensional semantic embedding is constructed. Each element Represents head entity With tail entity In relation types The intensity of semantic interaction.
[0083] (5)
[0084] in and These are the semantic vectors of the head and tail entities, respectively. For relation type The prototype vector, GeLU is the Gaussian error linear unit activation function. The dimension is vector. This calculation uses relation prototype vectors to semantically constrain entity interactions, enabling entity pairs under different relation types to have differentiated semantic representations, providing discriminative features for subsequent relation classification.
[0085] To determine the primary supporting evidence for each candidate entity pair in the document, for each candidate entity pair, the main supporting evidence is calculated for each sentence in the document. Relevance score This scoring system comprehensively considers three key factors: relational semantic matching based on three-dimensional semantic embedding, entity co-occurrence strength based on word frequency statistics, and contextual relevance based on sentence distance decay. This multi-dimensional evaluation accurately identifies the key sentences that most strongly support the judgment of entity pair relationships.
[0086] (6)
[0087] in and For sentences Middle head entity Tail-end entity word frequency For sentences Position number in the document The core sentence position of the entity pair; and the highest-scoring entity is selected according to the relevance score from high to low. The following sentences serve as key evidence:
[0088] (7)
[0089] Then, by generating structured key prompts, the numerical semantic features are transformed into text prompts suitable for language model understanding. The core of generating structured key prompts lies in extracting the tendency distribution of entity pairs across different relation types from the three-dimensional semantic embedding.
[0090] The structured key hint information is generated as follows: for candidate entity pairs First, the matching score vector for each relation type is calculated based on the three-dimensional semantic embedding. The k-th component is calculated as follows:
[0091] (8)
[0092] Then, a temperature-regulated softmax normalization is introduced to transform the matching score vector into a probability distribution:
[0093] (9)
[0094] Where T is a temperature hyperparameter used to control the sharpness of the probability distribution. Then, the three relation types with the highest probability values and their corresponding probability values are selected, and structured key prompt information is generated according to a preset prompt template. This structured key prompt information includes entity pair identifiers, relation type names, and their matching probabilities. This prompt generation method quantifies the matching strength of different relation types through explicit probability values, providing reliable prior knowledge guidance for subsequent relation classification in the language model. (Relation type set) Relationships derived from predefined tasks This includes, but is not limited to, relationship types in specific areas such as geographical location, organizational affiliation, and personal relationships.
[0095] To ensure effective modeling while controlling computational costs, a block sparse attention mechanism is used to encode the evidence sentences:
[0096] ;
[0097] in, , , These represent the query, key, and value matrices in the b-th subspace, respectively, with ⊕ indicating a concatenation operation. The total number of subspaces. This mechanism significantly reduces computational complexity while preserving key semantic dependencies by decomposing the complete attention computation into multiple independent subspaces.
[0098] After processing by this module, each candidate entity pair obtains a rich semantic representation containing three-dimensional semantic embedding, a set of key evidence sentences, and structured key prompt information. This information will serve as an important input for the subsequent relationship classification module, providing multi-level semantic support for the final relationship determination.
[0099] (4) By using a multi-head heterogeneous low-rank expert module, while keeping the main parameters of the pre-trained language model frozen, the outputs of multiple low-rank adapter sub-modules are dynamically fused through a gating mechanism based on the key evidence sentences and the structured key prompt information to classify the candidate entity pairs.
[0100] While keeping the main parameters of the pre-trained language model frozen, the multi-head heterogeneous low-rank expert module includes multiple low-rank adapter submodules and an evidence-enhancing gating network; the low-rank adapter submodules are used to learn parameter updates for specific domains or relational patterns in a low-dimensional subspace, and their output is:
[0101] (10)
[0102] in, For the first Input to the layer network, For the kth, For the shared output mapping matrix, The gating network is then used to dynamically calculate the fusion weights of each low-rank adapter submodule based on the input semantics, the key evidence sentences, and the structured key prompt information.
[0103] Furthermore, the process of calculating the fusion weights by the gated network is as follows:
[0104] The process of calculating the fusion weights in the gated network is as follows: The input vector of the current layer of the large language model is... The semantic encoding vector of the key evidence sentence and the semantic encoding vector of the structured key prompt information. The weights of each low-rank adapter submodule are concatenated and generated using a fully connected layer and a Softmax function:
[0105] (11)
[0106] in, , , The projection matrix is learnable. This represents a vector concatenation operation. For the gated parameter matrix, This is a bias term.
[0107] Furthermore, the training method for the multi-head heterogeneous low-rank expert module is as follows:
[0108] When training the multi-head heterogeneous low-rank expert module, an expert load balancing loss is introduced:
[0109] (12)
[0110] The module's total loss function is a weighted sum of the relation classification loss and the load balancing loss:
[0111] (13)
[0112] in, For relationship classification loss, To balance the hyperparameters.
[0113] The multi-head heterogeneous low-rank expert module comprises multiple low-rank adapter submodules and an evidence-enhancing gating network. The algorithm flow is attached. Figure 3 The details are shown below, with pseudocode as follows:
[0114]
[0115] Furthermore, after obtaining the outputs of each expert submodule, they are fused with the frozen pre-trained backbone network output through a weighted summation. The final output of the relation classification is as follows:
[0116] The final output of the relation classification is a weighted sum of the output of the main network of the large language model and the outputs of each low-rank adapter submodule. :
[0117] (14)
[0118] in, This represents the general semantic features provided by the pre-trained model. This represents the domain-specific features generated by the expert module. This aspect maintains the model's basic language understanding capabilities, ensuring that the introduction of expert modules does not impair the original language modeling abilities. This represents the domain-specific features generated by the expert module. This multi-level feature fusion mechanism ensures that relation classification decisions are based on both deep semantic understanding and full consideration of specific evidence support and relation priors.
[0119] Obviously, those skilled in the art should understand that the above-described method steps of this invention can be implemented using a general-purpose computing device, and the tools used are not limited to those provided by this invention; other related tools can also implement the steps of this invention. Although embodiments of this invention have been described previously, they are only for illustrating the technical solutions and main features of this invention, and are not intended to limit this invention. Those skilled in the art, once they understand the basic inventive concept, can make other changes and modifications to these embodiments, or make equivalent substitutions for some of the technical features. Any variations or substitutions easily conceived by those skilled in the art within the scope of the technology disclosed in this invention, within the principles of this invention, are covered within the protection scope of this invention.
Claims
1. A document-level relation extraction method integrating three-dimensional sparse semantics and multi-head expert mechanisms, characterized in that, Includes the following steps: (1) Enumerate all possible entity pairs based on the pre-extracted entity list provided by the dataset to form an initial candidate entity pair set; (2) The candidate entity pairs in the initial candidate entity pair set are filtered by the candidate entity pair pre-screening module based on the semantic interaction strength between entities, and unrelated entity pairs are eliminated to obtain the filtered candidate entity pair set. (3) Using the three-dimensional sparse semantic enhancement module, construct a three-dimensional semantic embedding for each entity pair in the filtered candidate entity pair set, locate key evidence sentences based on the three-dimensional semantic embedding, and generate structured key prompt information; (4) By using a multi-head heterogeneous low-rank expert module, while keeping the main parameters of the pre-trained language model frozen, the outputs of multiple low-rank adapter sub-modules are dynamically fused through a gating mechanism based on the key evidence sentences and the structured key prompt information to classify the candidate entity pairs.
2. The method according to claim 1, characterized in that, The implementation process of step (1) is as follows: The dataset is trained and tested using the DocRED and Re-DocRED datasets. All possible entity pairs are enumerated based on the pre-extracted entity lists provided by the DocRED and Re-DocRED datasets, and an initial semantic embedding vector is generated for each entity using a pre-trained language model.
3. The method according to claim 1, characterized in that, The implementation process of step (2) is as follows: Based on the multi-head attention mechanism, the head entity is calculated. With tail entity The degree of overlap in attention distributions across multiple attention heads yields a joint contextual semantic representation: (1) in, For the total number of attention heads, and Representing the head entity respectively With tail entity The attention distribution vector at the t-th attention head. This represents an element-wise multiplication operation; The joint context representation is fused with the initial semantic embedding vector of the entity to obtain the enhanced entity pair representation: (2) in, For entities The initial semantic embedding vector, and The learnable parameter matrix; Based on the enhanced entity pair representation, the probability that an entity pair is unrelated is calculated using a bilinear transformation: (3) in It is a bilinear weight matrix. For bias terms, It is the sigmoid function; Remove unrelated entity pairs from the candidate set using dynamic thresholds: (4) in, and These represent the number of negative and positive samples in the current training batch, respectively. It is a very small constant.
4. The method according to claim 1, characterized in that, The method for constructing the three-dimensional semantic embedding in step (3) is as follows: According to the head entity With tail entity and predefined relationship types Constructing 3D Embedding Each element Represents head entity With tail entity In relation types The strength of semantic interaction below: (5) in, and These are the semantic vectors of the head and tail entities, respectively. For relation type The prototype vector, GeLU is the Gaussian error linear unit activation function, and d is the vector dimension.
5. The method according to claim 4, characterized in that, The method for locating key evidence sentences based on the three-dimensional semantic embedding in step (3) is as follows: For each candidate entity pair, a relevance score is calculated for each sentence in the document based on the three-dimensional semantic embedding. : (6) in, and For sentences Middle head entity Tail-end entity word frequency For sentences Position number in the document The core sentence position of the entity pair is identified; and several sentences are selected as key evidence sentences according to the aforementioned relevance score, from high to low: (7)。 6. The method according to claim 4, characterized in that, The structured key prompt information in step (3) is generated in the following way: For candidate entity pairs First, the matching score vector for each relation type is calculated based on the three-dimensional semantic embedding, where the k-th component is calculated as follows: (8) By introducing temperature-regulated softmax normalization, the matching score vector is transformed into a probability distribution: (9) Where T is the temperature hyperparameter; the top three relation types with the highest probability values and their corresponding probability values are selected, and structured key prompt information is generated according to the preset prompt template. The structured key prompt information includes entity pair identifiers, relation type names and their matching probabilities.
7. The method according to claim 1, characterized in that, The implementation process of step (4) is as follows: While keeping the main parameters of the pre-trained language model frozen, the multi-head heterogeneous low-rank expert module includes multiple low-rank adapter submodules and an evidence-enhancing gating network; the low-rank adapter submodules are used to learn parameter updates for specific domains or relational patterns in a low-dimensional subspace, and their output is: (10) in, For the first Input to the layer network, For the kth, For the shared output mapping matrix, The gating network is then used to dynamically calculate the fusion weights of each low-rank adapter submodule based on the input semantics, the key evidence sentences, and the structured key prompt information.
8. The method according to claim 7, characterized in that, The process of calculating the fusion weights in the gated network is as follows: The process of calculating the fusion weights in the gated network is as follows: The input vector of the current layer of the large language model is... The semantic encoding vector of the key evidence sentence and the semantic encoding vector of the structured key prompt information. The weights of each low-rank adapter submodule are concatenated and generated using a fully connected layer and a Softmax function: (11) in, , , The projection matrix is learnable. This represents a vector concatenation operation. For the gated parameter matrix, This is a bias term.
9. The method according to claim 7, characterized in that, The training method for the multi-head heterogeneous low-rank expert module is as follows: When training the multi-head heterogeneous low-rank expert module, an expert load balancing loss is introduced: (12) The module's total loss function is a weighted sum of the relation classification loss and the load balancing loss: (13) in, For relationship classification loss, To balance the hyperparameters.
10. The method according to claim 7, characterized in that, The final output of the relation classification is as follows: The final output of the relation classification is a weighted sum of the output of the main network of the large language model and the outputs of each low-rank adapter submodule. : (14) in, This represents the general semantic features provided by the pre-trained model. This represents the domain-specific features generated by the expert module.