Bilingual network threat intelligence relation extraction-oriented large language model optimization method and system

Abstract

The invention relates to a bilingual network threat intelligence relation extraction-oriented large language model optimization method and system, and the method comprises the steps: generating an enhanced data set according to an example set, and combining the example set and the enhanced data set to form a training set; the performance of a plurality of LLMs models is tested under various conditions, task performance and instruction following ability are extracted based on the relation of the models on a Chinese and English data set, and a basic model is screened out. And carrying out fine tuning on the screened model by adopting LoRA, and correcting the output by the LoRA through two trainable weight matrixes and by introducing an increment updating item containing task specific knowledge. Generating a dynamic instruction with a cluster exclusive feature for each test sample based on cluster affiliation, retaining a part of candidate examples with the highest correlation, screening out a part of examples with the highest correlation from the training set, constructing a sample subset under redundancy constraints, and performing cluster classification; and finally, fusing the cluster exclusive instruction with the demonstration examples for enhancing diversity in the sample subset by the cue word.

Description

Technical Field

[0001] This application relates to the field of large model technology, and in particular to a method and system for optimizing large language models for bilingual network threat intelligence relationship extraction. Background Technology

[0002] With the rapid development of network information technology, cyberattacks are becoming increasingly complex. New attack methods such as Advanced Persistent Threats (APTs) are constantly emerging, posing a serious threat to network security. Against this backdrop, the "proactive defense and attribution" cybersecurity strategy has received increasing attention. The concept of Cyber ​​Threat Intelligence (CTI) provides new theoretical support for cyberattack attribution, enabling the identification of the source of attacks in large-scale incidents. By extracting actionable intelligence from relevant reports and constructing a cyber threat intelligence knowledge graph that characterizes their internal relationships, security professionals can develop more resilient defense strategies.

[0003] To help researchers quickly understand the correlation between emerging and historical threat events, relation extraction (RE) has become a core task in constructing cyber threat intelligence knowledge graphs. Relation extraction mainly comprises two key tasks: relation classification and entity relation extraction. Traditional methods treat named entity recognition and relation extraction as two independent tasks, but this approach is prone to error propagation problems. Therefore, recent research has begun to employ joint entity relation extraction techniques to improve performance.

[0004] Relation extraction based on LLMs has become a popular new paradigm. However, existing methods have three key limitations: (1) Entities and their relationships in network threat intelligence data often exhibit rare types and highly complex features, making it difficult for existing general-purpose LLMs to be directly applied in practice; (2) Current context learning methods mainly rely on sentence similarity-based retrieval to select the top k examples, but this approach often results in sample redundancy and insufficient diversity after model fine-tuning; (3) Finally, current research on relationship extraction in network threat intelligence mainly focuses on monolingual (English) scenarios, with insufficient coverage of other languages. Summary of the Invention This invention aims to provide a large language model optimization method and system for bilingual network threat intelligence relationship extraction, in order to overcome the shortcomings of existing technologies. The technical problem to be solved by this invention is achieved through the following technical solutions.

[0005] A method for optimizing a large language model for bilingual network threat intelligence relation extraction includes the following steps: Step 1: Long-tail relationship data augmentation The kNN algorithm is used to retrieve the k most similar examples for each target relation type from the original dataset. The example set is selected according to the relation type, and an augmented dataset is generated based on the example set. The example set and the augmented dataset are then merged to form the training set. Step Two: Model Evaluation and Selection The performance of multiple LLMs models was tested under zero-sample and small-sample conditions. Based on the model's performance in relation extraction tasks and instruction compliance on Chinese and English datasets, a base model was selected. Step 3: Fine-tuning of bilingual instructions For a selected base model, LoRA is used for model fine-tuning. LoRA corrects the output by using two trainable weight matrices and introducing incremental update terms that contain task-specific knowledge. Step 4: Cluster-guided dynamic redundancy-aware context learning A clustering algorithm is used to generate dynamic instructions with cluster-specific features for each test sample based on cluster affiliation. The kNN algorithm is used to retain a subset of candidate examples with the highest relevance. Then, k examples with the highest relevance are selected from the training set to construct a sample subset under redundancy constraints. Finally, the prompt words are integrated with the cluster-specific instructions and the enhanced diversity demonstration examples in the sample subset.

[0006] Preferably, in step one, the sample selection and template design process is described as follows:

[0007] in, Indicates based on relation type Selected sample set, Represents the original dataset. It is the target relation type. Indicates the number of nearest neighbors.

[0008] Preferably, in step one, the augmented dataset is generated in the following manner.

[0009] in, Indicates based on relation type The generated augmented dataset, where ∪ represents the union operation on all generated results. This is a relation type A collection of design prompt templates, Represents a set Each prompt template in the file, Indicates based on the selected example set and prompt template New labeled data can be generated using LLMs.

[0010] Preferably, in step two, in the case of zero samples, given the input text and set of relation types The model outputs the predicted set of triples. , recorded as Its expression is:

[0011] in, This represents the model's prediction function. Indicates pre-trained parameters; With a small number of samples, the model utilizes a limited number of labeled examples. To enhance the model's ability to follow task instructions, its expression is:

[0012]

[0013] in, Here are a few examples with annotations. For a complete training set, Enter a sentence for the current test. Indicates the number of fewer sample examples. This represents the model's prediction function. This represents the pre-trained parameters.

[0014] Preferably, in step three, given a pre-trained weight matrix The two trainable weight matrices of LoRA are represented as follows: and ,in d , m Represent the matrix dimension and rank, respectively. LoRA introduces incremental update items that include task-specific knowledge. Correct the output:

[0015] in, Indicates the output. Indicates input, This represents the scaling factor.

[0016] Preferably, in step four, the clustering guidance includes the following steps: Training Embedded into the semantic space, and divided into groups using a clustering algorithm. There are several clusters, and their formal objective functions are as follows:

[0017] in, This indicates that the embedded sample Clustering Clusters, For the sample with cluster center of mass The square of the Euclidean distance between them; For each test sample We need to determine the cluster closest to it and determine its semantic cluster affiliation by minimizing the embedding distance:

[0018] in, The index of the optimal cluster. arg min is the minimum value operation. Used to calculate the embedding vector of the test sample With each cluster center of mass The distance between them; Generate dynamic instructions with cluster-specific characteristics based on cluster affiliation. :

[0019] in, This represents predefined basic instructions. For clusters Generate task-specific supplementary information unique to this cluster. This indicates a string concatenation operation. Preferably, in step four, dynamic redundancy sensing includes the following steps: use The NN algorithm selects a subset of candidate examples that are most relevant to the test examples; From the training set in the following way Precisely selected The most relevant examples form the candidate set. :

[0020] in, T Enter a sentence for the current test; The threshold is dynamically adjusted based on the similarity distribution of the actual candidate set in the following manner.

[0021]

[0022] in, Indicates the baseline similarity threshold. Embedded vectors for test samples and Embedded vectors of retrieved candidate samples The first cosine similarity score between percentile; It is the adaptive similarity threshold ultimately used for selecting redundant-aware examples. It is the minimum allowed similarity threshold. It is an adjustable offset; From the candidate set under redundancy constraints, the following method is used. Constructing a sample subset that preserves diversity ;

[0023] in, For sample subset The samples in For candidate sample embedding vectors, For sample subset The embedding vector; The prompt words combine cluster-specific instructions with demonstration examples that enhance diversity, and their formal objective function is as follows:

[0024] in, To predict the results of the test samples, Cluster-specific instructions This represents a subset of samples that preserves diversity. This indicates a string concatenation operation.

[0025] This invention also provides a large language model optimization system for bilingual network threat intelligence relation extraction, comprising: The long-tail relation data augmentation module uses the kNN algorithm to retrieve the k most similar examples for each target relation type from the original dataset, selects the example set according to the relation type, generates the augmented dataset based on the example set, and then merges the example set and the augmented dataset to form the training set. The model evaluation and selection module tests the performance of multiple LLMs models under zero-sample and small-sample conditions, and selects a base model based on the model's performance in relation extraction tasks and instruction compliance on Chinese and English datasets. The bilingual instruction fine-tuning module uses LoRA to fine-tune a selected base model. LoRA uses two trainable weight matrices and introduces incremental update terms containing task-specific knowledge to correct the output. The cluster-guided dynamic redundancy-aware context learning module uses a clustering algorithm to generate dynamic instructions with cluster-specific features for each test sample based on cluster affiliation. It retains a subset of candidate examples with the highest relevance based on the kNN algorithm, and then selects k examples with the highest relevance from the training set. Under redundancy constraints, a sample subset is constructed. Finally, the prompt words integrate the cluster-specific instructions with the enhanced diversity demonstration examples in the sample subset.

[0026] The large language model optimization method and system for bilingual network threat intelligence relation extraction of the present invention have the following beneficial effects: 1. We augmented the open-source Chinese and English cyber threat intelligence datasets, improving their quality and diversity. Based on the augmented datasets, we conducted a systematic evaluation of mainstream LLMs under zero-sample and few-sample settings to select the best-performing models.

[0027] 2. Based on the evaluation results, the instructions were fine-tuned and the bilingual network threat intelligence relationship extraction was fine-tuned to enhance the model's understanding and compliance with task instructions, which significantly improved the model's performance in zero-shot scenarios.

[0028] 3. The cluster-guided dynamic redundancy-aware context learning method of this application can achieve dynamic filtering of context examples and adaptive organization of prompts, thereby effectively enhancing the model's ability to extract relation triples in a few-sample setting. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0030] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0032] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0033] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0034] In the description of this invention, it should be noted that the terms "upper," "lower," "inner," "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the product of this invention is usually placed when in use. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0035] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," and "connect" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0036] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0037] Combination Figure 1 As shown, using - Neighbors ( The NN algorithm retrieves the most similar top-order relation for each target relation type from the original dataset. We provided several examples to achieve precise matching. Subsequently, to accommodate differences in dataset format and content, we designed two customized prompt templates to guide the generation of high-quality labeled data. We describe the sample selection and template design process as follows:

[0038] in, Represented as relation type Selected sample set, Represents the original dataset. It is the target relation type. Indicates the number of nearest neighbors. NN This represents the retrieval function used for similarity measurement.

[0039]

[0040] in, Represented by relation type The generated augmented dataset, where ∪ represents the union operation on all generated results. This is a relation type A collection of design prompt templates, Represents a set Each prompt template in the file, This indicates the use of LLMs, based on the selected sample set. and prompt template A function to generate newly labeled data. Then, the generated data is merged with the original dataset to obtain the final augmented dataset, the size of which is as follows: As shown in Table 1.

[0041] Table 1 Statistics of the original and augmented datasets

[0042] For example, this application selected five LLMs for evaluation. These models not only demonstrated strong performance in NLP tasks but also achieved excellent results on multiple benchmarks. The selected models include Meta-Llama-3-8B-Instruct, Mistral-7B-Instruct-v0.3, Zephyr-7B-Alpha, Qwen2.5-7B-Instruct-1M, and GLM-4-9B-Chat-HF. Specifically, we conducted a systematic evaluation of these models under both zero-shot and few-shot experimental settings.

[0043] In the zero-shot setting, since no relevant samples are provided, the model must directly predict unseen test data triples based on pre-trained knowledge. The task is defined as follows: Given input text... and set of relation types The model outputs the predicted set of triples. , recorded as Its expression is:

[0044] in, This represents the model's prediction function. This represents the pre-trained parameters.

[0045] In a few-sample setting, the model will utilize a small number of labeled examples. This enhances the model's ability to follow task instructions. Its expression is:

[0046]

[0047] in, Indicates the number of few-sample examples, for example, , For a complete training set, Enter a sentence for the current test, that is, enter text.

[0048] Based on the model's performance in relation extraction tasks and its ability to follow instructions on Chinese and English datasets, a basic model was selected for subsequent fine-tuning.

[0049] Evaluation results show that the basic model has limited instruction compliance capabilities and struggles to consistently generate relation triples that meet the required format. Furthermore, relation extraction tasks in the field of cyber threat intelligence often involve complex structures and diverse semantics, further increasing the difficulty of the generation task. To address this issue, an enhanced dataset is used to construct instruction fine-tuning content. By injecting domain-specific knowledge, the model can better grasp the semantics of the cyber threat intelligence domain, thereby improving its overall cyber threat intelligence performance.

[0050] Based on the bilingual evaluation results, Mistral-7B-Instruct v0.3 was selected as the base model. To reduce training costs, LoRA was used for model fine-tuning. A pre-trained weight matrix was provided. LoRA introduces two trainable weight matrices. and , where rank .set up This represents input; normally, it is done through... The output obtained is In contrast, LoRA introduces incremental update items that contain task-specific knowledge. Correct the output:

[0051] in This represents the scaling factor. During the initial training phase... Initialization is performed using a random Gaussian distribution, while Initialize as a zero matrix to ensure Initially, it is a zero matrix. The number of trainable parameters in LoRA is... Far fewer than the number of updates required in a full-scale fine-tuning process. One parameter.

[0052] To address the issue of homogeneity and redundancy in the training and retrieval processes and enhance the diversity of content with limited samples, this paper proposes an innovative context learning method—cluster-guided dynamic redundancy-aware context learning. This method comprises two core operational stages: semantic clustering-guided cue adaptation and dynamic redundancy-aware example selection.

[0053] In the semantic clustering-guided prompt adaptation phase, the training data is first... Embedded into the semantic space, and divided into groups using a clustering algorithm. There are several clusters, and their formal objective functions are as follows:

[0054] in, This indicates that the embedded sample Clustering Clusters, For the sample with cluster center of mass The square of the Euclidean distance between them.

[0055] Next, for each test sample We need to determine the cluster closest to it and determine its semantic cluster affiliation by minimizing the embedding distance:

[0056] in, The index of the optimal cluster. Used to calculate the embedding vector of the test sample With each cluster center of mass The distance between them.

[0057] Finally, dynamic instructions with cluster-specific characteristics are generated based on cluster affiliation:

[0058] in, This represents predefined basic instructions. For clusters Generate task-specific supplementary information unique to this cluster. This represents a string concatenation operation, that is, by using the basic instruction... Clustering-guided knowledge Combine to form new dynamic instructions This allows for the generation of personalized tasks or semantic guidance for each test sample.

[0059] In the selection phase of dynamic redundancy sensing examples, firstly utilize - Neighbors ( The NN algorithm retrieves candidate examples, quickly filtering out training examples with low relevance to the test examples, and retaining the batch of candidate examples with the highest relevance. Then, from the training set... Precisely selected The most relevant examples form the candidate set. .

[0060] Secondly, it is necessary to ensure that the context has reasonable redundancy control and to dynamically adjust the threshold according to the similarity distribution of the actual candidate set:

[0061]

[0062] in, This represents the baseline similarity threshold, which is calculated from the test sample embedding vector. and Embedded vectors of retrieved candidate samples The first cosine similarity score between percentile; It is the minimum allowed similarity threshold, used to avoid including too many irrelevant samples; It is an adjustable offset used to relax the baseline threshold; It is the adaptive similarity threshold ultimately used for selecting redundant awareness examples.

[0063] Next, under redundancy constraints, from the candidate set Constructing a sample subset that preserves diversity :

[0064] Under this constraint, the retained samples not only satisfy The high relevance requirement of NN selection also ensures the semantic diversity of the selected examples, effectively reducing redundancy in few-sample demonstrations.

[0065] The final prompt combines cluster-specific instructions with demonstration examples that enhance diversity, and its formal objective function is as follows:

[0066] in, Cluster-specific instructions This represents a set of examples for preserving diversity. The input text for the test sample.

[0067] The model's reasoning ability is enhanced by semantic clustering-guided cue adaptation and dynamic redundancy awareness in example selection.

[0068] It should be noted that the above detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0069] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments described in this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0070] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that the embodiments of this application described herein can be implemented in sequences other than those illustrated or described herein.

[0071] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or apparatus.

[0072] For ease of description, spatial relative terms such as "above," "on top of," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the figures to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation beyond the orientation of the device as described in the figures. For example, if the device in the figures were inverted, a device described as "above" or "on top of" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways, such as rotated 90 degrees or in other orientations, and the spatial relative descriptions used herein will be interpreted accordingly.

[0073] In the detailed description above, reference has been made to the accompanying drawings, which form part of this document. In the drawings, similar symbols typically identify similar parts unless the context otherwise indicates otherwise. The illustrated embodiments described in the detailed specification, drawings, and claims are not intended to be limiting. Other embodiments may be used and other changes may be made without departing from the spirit or scope of the subject matter presented herein.

[0074] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for optimizing a large language model for bilingual network threat intelligence relation extraction, characterized in that: Includes the following steps: Step 1: Long-tail relationship data augmentation The kNN algorithm is used to retrieve the k most similar examples for each target relation type from the original dataset. The example set is selected according to the relation type, and an augmented dataset is generated based on the example set. The example set and the augmented dataset are then merged to form the training set. Step Two: Model Evaluation and Selection The performance of multiple LLMs models was tested under zero-sample and small-sample conditions. Based on the model's performance in relation extraction tasks and instruction compliance on Chinese and English datasets, a base model was selected. Step 3: Fine-tuning of bilingual instructions For a selected base model, LoRA is used for model fine-tuning. LoRA corrects the output by using two trainable weight matrices and introducing incremental update terms that contain task-specific knowledge. Step 4: Cluster-guided dynamic redundancy-aware context learning A clustering algorithm is used to generate dynamic instructions with cluster-specific features for each test sample based on cluster affiliation. The kNN algorithm is used to retain a subset of candidate examples with the highest relevance. Then, k examples with the highest relevance are selected from the training set to construct a sample subset under redundancy constraints. Finally, the prompt words are integrated with the cluster-specific instructions and the enhanced diversity demonstration examples in the sample subset.

2. The method for optimizing a large language model for bilingual network threat intelligence relationship extraction according to claim 1, characterized in that: In step one, the sample selection and template design process is described as follows: in, Indicates based on relation type Selected sample set, Represents the original dataset. It is the target relation type. Indicates the number of nearest neighbors.

3. The method for optimizing a large language model for bilingual network threat intelligence relationship extraction according to claim 1, characterized in that: In step one, the augmented dataset is generated in the following manner. in, Indicates based on relation type The generated augmented dataset, where ∪ represents the union operation on all generated results. This is a relation type A collection of design prompt templates, Represents a set Each prompt template in the file, Indicates based on the selected example set and prompt template New labeled data can be generated using LLMs.

4. The method for optimizing a large language model for bilingual network threat intelligence relationship extraction according to claim 1, characterized in that: In step two, given the input text, in the case of zero samples... and set of relation types The model outputs the predicted set of triples. , recorded as Its expression is: in, The prediction function of the model. Indicates pre-trained parameters; With a small number of samples, the model utilizes a limited number of labeled examples. To enhance the model's ability to follow task instructions, its expression is: in, Here are a few examples with annotations. For a complete training set, Enter a sentence for the current test. Indicates the number of fewer sample examples. The prediction function of the model. This represents the pre-trained parameters.

5. The method for optimizing a large language model for bilingual network threat intelligence relationship extraction according to claim 1, characterized in that: In step three, given the pre-trained weight matrix The two trainable weight matrices of LoRA are represented as follows: and ,in d , m Represent the matrix dimension and rank, respectively. LoRA introduces incremental update items that include task-specific knowledge. Correct the output: in, Indicates the output. Indicates input, This represents the scaling factor.

6. The method for optimizing a large language model for bilingual network threat intelligence relationship extraction according to claim 1, characterized in that: In step four, clustering guidance includes the following steps: Training Embedded into the semantic space, and divided into groups using a clustering algorithm. There are several clusters, and their formal objective functions are as follows: in, This indicates that the embedded sample Clustering Clusters, arg min is the minimum value operation. For the sample with cluster center of mass The square of the Euclidean distance between them; For each test sample We need to determine the cluster closest to it and determine its semantic cluster affiliation by minimizing the embedding distance: in, The index of the optimal cluster. arg min is the minimum value operation. Used to calculate the embedding vector of the test sample With each cluster center of mass The distance between them; Generate dynamic instructions with cluster-specific characteristics based on cluster affiliation. : in, This represents predefined basic instructions. For clusters Generate task-specific supplementary information unique to this cluster. This indicates a string concatenation operation.

7. The method for optimizing a large language model for bilingual network threat intelligence relationship extraction according to claim 6, characterized in that: In step four, dynamic redundancy sensing includes the following steps: use The NN algorithm selects a subset of candidate examples that are most relevant to the test examples; From the training set in the following way Precisely selected The most relevant examples form the candidate set. : in, T Enter a sentence for the current test; The threshold is dynamically adjusted based on the similarity distribution of the actual candidate set in the following manner: in, Indicates the baseline similarity threshold. Embedded vectors for test samples and Embedded vectors of retrieved candidate samples The first cosine similarity score between percentile; It is the adaptive similarity threshold ultimately used for selecting redundant-aware examples. It is the minimum allowed similarity threshold. It is an adjustable offset; From the candidate set under redundancy constraints, the following method is used. Constructing a sample subset that preserves diversity ; in, For sample subset The samples in For candidate sample embedding vectors, For sample subset The embedding vector; The prompt words combine cluster-specific instructions with demonstration examples that enhance diversity, and their formal objective function is as follows: in, To predict the results of the test samples, Cluster-specific instructions This represents a subset of samples that preserves diversity. This indicates a string concatenation operation.

8. A large language model optimization system for bilingual network threat intelligence relationship extraction, characterized in that: include: The long-tail relation data augmentation module uses the kNN algorithm to retrieve the k most similar examples for each target relation type from the original dataset, selects the example set according to the relation type, generates the augmented dataset based on the example set, and then merges the example set and the augmented dataset to form the training set. The model evaluation and selection module tests the performance of multiple LLMs models under zero-sample and small-sample conditions, and selects a base model based on the model's performance in relation extraction tasks and instruction compliance on Chinese and English datasets. The bilingual instruction fine-tuning module uses LoRA to fine-tune a selected base model. LoRA uses two trainable weight matrices and introduces incremental update terms containing task-specific knowledge to correct the output. The cluster-guided dynamic redundancy-aware context learning module uses a clustering algorithm to generate dynamic instructions with cluster-specific features for each test sample based on cluster affiliation. It retains a subset of candidate examples with the highest relevance based on the kNN algorithm, and then selects k examples with the highest relevance from the training set. Under redundancy constraints, a sample subset is constructed. Finally, the prompt words integrate the cluster-specific instructions with the enhanced diversity demonstration examples in the sample subset.

Images