Method and system for predicting relations of science and education resources based on large model zero sample prediction

Abstract

The present application provides a science and education resource relationship prediction method and system based on large model zero sample prediction, comprising: obtaining unlabeled science and education resource data, dividing the unlabeled science and education resource data into simple samples and difficult samples based on the zero sample prediction and self-check of the large model, and constituting a few sample training set; labeling the few sample training set by artificial means; selecting a corresponding large model instruction fine-tuning algorithm according to different application scenarios, training the model on the labeled few sample training set, and obtaining a trained model; loading the trained model to perform automatic relationship extraction and complete relationship prediction. By obtaining unlabeled science and education resource data and using the zero sample prediction capability of the large model to preliminarily process these data, the dependence on a large amount of labeled data in the traditional method is avoided, so that effective relationship prediction can be performed in the science and education field where labeled data is scarce.

Description

Technical Field

[0001] This invention belongs to the field of knowledge graph construction technology for science and education resources, and specifically relates to a method and system for predicting the relationship between science and education resources based on zero-sample prediction of a large model. Background Technology

[0002] Software-defined science and education resources are a collection of resources consisting of educational resources, carriers of educational resources and their interrelationships. Software breaks the traditional integrated science and education hardware resources, virtualizes, standardizes and modularizes them to adapt to frequently changing application needs. In order to efficiently, accurately and comprehensively process massive, multi-source heterogeneous, and integrated science and education resources, it is necessary to organize and manage resources through knowledge graphs (KG) [1], and fully consider their multi-dimensional and multi-level data characteristics. Traditional knowledge graph automatic construction methods based on pre-trained model fine-tuning require a large amount of labeled data, so it is difficult to achieve satisfactory results in the field of science and education where labeled samples are scarce. Therefore, it is necessary to design a knowledge graph construction method for few-sample scenarios. Recently, large language models (LLM) have provided strong support for few-sample learning with their high-dimensional representation, context awareness and cross-domain applicability. By combining instruction fine-tuning technology, large models can quickly adapt to the knowledge graph construction task in the field of science and education on a small number of samples.

[0003] Knowledge graphs, as a novel knowledge organization method, can effectively manage massive amounts of complex, multi-source, and heterogeneous data, attracting widespread attention from industry and academia. This has led many large companies and teams to build large-scale knowledge bases, including Wikidata, a multilingual knowledge base with links to Wikipedia's database. A basic method for automatically constructing knowledge graphs involves fine-tuning a small-scale pre-trained model to implement three sub-tasks: entity discovery, relation extraction, and knowledge graph completion. However, this method requires a large amount of labeled data, making it unsuitable for fields like science and education where labeled samples are scarce.

[0004] As the parameter scale of large language models continues to increase, the large amount of common-sense knowledge they contain endows these models with a certain degree of knowledge reasoning ability, allowing them to quickly adapt to tasks in vertical domains with only a small amount of labeled data. A popular approach to applying large models to few-shot knowledge graph construction is through direct reasoning or context learning. However, due to limitations in input length and comprehension capacity, large models typically only accommodate 5–30 correct examples, failing to fully utilize the few-shot training set. Furthermore, the performance of ICL (Independent Reasoning and Context Learning) is highly unstable and easily affected by multiple factors, such as instruction, example order, and output format. In conclusion, applying large models to few-shot knowledge graph construction tasks through direct reasoning or ICL either fails to adapt to highly specialized fields like science and education or fails to fully utilize the training samples, exhibiting multiple unstable factors. Therefore, these solutions cannot be directly applied to few-shot knowledge graph construction tasks in the science and education fields.

[0005] Instruction fine-tuning can also be used to construct few-shot vertical domain knowledge graphs based on large models. Lilong et al. proposed an open knowledge graph construction framework, AutoRE, based on the large model fine-tuning algorithm (QLoRA). However, these methods typically use random strategies or sample few samples according to a certain data distribution, ignoring the powerful zero-shot capability inherent in large models. That is, the training benefits of samples that are relatively easy for large models are limited. Therefore, how to adaptively select the few training samples that maximize training benefits based on the zero-shot capability of large models is crucial for constructing few-shot knowledge graphs in the science and education domain.

[0006] In summary, the existing methods mainly have the following problems:

[0007] 1. The automatic knowledge graph construction method based on pre-trained model fine-tuning requires a large amount of labeled data, and its performance is poor in the science and education field where labeled samples are scarce.

[0008] 2. Methods based on direct reasoning or contextual learning, which mostly utilize the general reasoning capabilities of large models, cannot be generalized to highly specialized fields such as science and education, resulting in poor performance. Furthermore, due to the inherent limitations of large models, these methods exhibit poor stability.

[0009] 3. The method based on fine-tuning of large model instructions fails to consider the zero-sample capability inherent in the large model itself in the few-sample selection mechanism, thus failing to maximize the benefits of model training in a limited number of samples, resulting in poor knowledge graph construction performance. Summary of the Invention

[0010] The purpose of this invention is to provide a method and system for predicting the relationship between science and education resources based on zero-sample prediction of a large model, so as to solve the above-mentioned problems.

[0011] To achieve the above objectives, the present invention adopts the following technical solution:

[0012] In a first aspect, this invention provides a method for predicting the relationship between science and education resources based on zero-sample prediction using a large model, including:

[0013] We acquire unlabeled science and education resource data, and based on zero-shot prediction and self-testing of a large model, we divide the unlabeled science and education resource data into simple samples and difficult samples to form a few-sample training set.

[0014] The training set with a small number of samples is labeled manually.

[0015] Select the appropriate large model instruction fine-tuning algorithm according to different application scenarios, train the model on the labeled small sample training set, and obtain the trained model.

[0016] The pre-trained model is loaded to automatically extract relationships and predict them.

[0017] Furthermore, acquiring unlabeled science and education resource data includes: obtaining multi-source heterogeneous science and education data through educational resources, and converting uneditable resources into editable resources through OCR recognition and storing them in a corpus.

[0018] Furthermore, based on the zero-shot prediction and self-testing of the large model, the unlabeled science and education resource data is divided into simple samples and difficult samples, forming a few-shot training set, including:

[0019] Zero-shot prediction: Existing unlabeled relation extraction dataset ,for Each sample Use a large model to make zero-shot predictions on it, i.e. ,in Indicates that for the first Predictive hints for individual sample designs;

[0020] Sample partitioning based on self-testing: Subsequently, based on the zero-shot prediction results of the large model, each sample... Relationship Predictions Design large model self-check prompts And perform a large model self-check, that is Based on the results of the large model self-check Unlabeled dataset Divided into and Two sets, where Indicates a difficult sample. Represents a simple sample;

[0021] Uncertainty ranking: Uncertainty is used to measure... The degree of difficulty of the medium-difficulty samples, i.e. ,in express The maximum probability distribution is then determined; subsequently, the difficult samples are ranked according to the uncertainty of each self-test result, and the top ones are selected. One sample is used as the training sample to be labeled;

[0022] Construction of high-quality training sets: [Sets have been selected] Given a set of difficult samples, the easy samples are ranked from low to high uncertainty, and the top samples are selected. A simple sample is added as a regularization sample for training, and finally, the sample is selected. A number of high-quality training samples awaiting labeling.

[0023] Furthermore, the training set with a small number of samples is labeled manually, including:

[0024] General large model automatic annotation: For training samples to be annotated, automatic annotation is performed first using existing general large models;

[0025] Manual corrections are performed on the automatically labeled data;

[0026] The manually corrected data will be manually reviewed. If the review fails, the data will be corrected again.

[0027] Furthermore, the appropriate large model instruction fine-tuning algorithm can be selected based on different application scenarios, including:

[0028] Select the appropriate large model instruction fine-tuning algorithm according to different application scenarios, including full fine-tuning, efficient fine-tuning LoRA and model quantization fine-tuning QLoRA, and train the model on a labeled small sample training set.

[0029] Furthermore, the model is trained on a labeled small-sample training set to obtain a trained model, including:

[0030] After determining the fine-tuning method, the cross-entropy of each category is calculated and summed to obtain the final loss of the model. , where K represents the number of training samples and M represents the sequence length of each training sample.

[0031] Furthermore, the pre-trained model is loaded to automatically extract relations and perform relation prediction, including:

[0032] Model loading: For models under full fine-tuning, load the fine-tuned large model; for models under efficient fine-tuning, first load the parameters of the base model, and then load the model weights added during the fine-tuning process; for models under quantized fine-tuning, first load the quantized base model, and then load the additional trained model parameters.

[0033] Relation extraction hint construction: When applying a large model to a relation extraction task, design hints to guide the large model to complete the relevant task;

[0034] Relationship prediction: Automatically extract relationships from the pre-constructed prompts and loaded model.

[0035] Secondly, this invention provides a science and education resource relationship prediction system based on large-scale model zero-sample prediction, comprising:

[0036] The data acquisition module is used to acquire unlabeled science and education resource data. Based on the zero-shot prediction and self-check of the large model, the unlabeled science and education resource data is divided into simple samples and difficult samples to form a few-sample training set.

[0037] The data annotation module is used to manually annotate a small number of training samples.

[0038] The model training module is used to select the appropriate large model instruction fine-tuning algorithm according to different application scenarios, and train the model on the labeled small sample training set to obtain the trained model.

[0039] The prediction output module is used to load the trained model, perform automated relation extraction, and complete relation prediction.

[0040] Thirdly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method for predicting the relationship between science and education resources based on zero-sample prediction of a large model.

[0041] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method for predicting the relationship between science and education resources based on zero-sample prediction of a large model.

[0042] Compared with the prior art, the present invention has the following technical effects:

[0043] This technical solution proposes a method for predicting the relationship between science and education resources based on zero-sample prediction using a large model. It aims to address the problem of scarce labeled data in the field of science and education and improve the accuracy of relationship prediction.

[0044] This method first acquires unlabeled science and education resource data and then uses the zero-shot prediction capability of a large model to perform preliminary processing on this data. This avoids the dependence on large amounts of labeled data in traditional methods, enabling effective relationship prediction even in the science and education field where labeled data is scarce.

[0045] By leveraging the self-checking capabilities of a large model, unlabeled data is categorized into easy and difficult samples, and high-quality training samples are further selected through uncertainty ranking. This mechanism ensures that the training set includes both challenging difficult samples and a sufficient number of easy samples, thereby improving the efficiency and accuracy of model training. The application of uncertainty ranking makes the selected training samples more representative, better reflecting the overall distribution of the data, and thus enhancing the model's generalization ability.

[0046] By using a general large model for automatic annotation, followed by manual correction and review, both annotation efficiency and quality are ensured. This semi-automated annotation method reduces the burden of manual annotation while improving the accuracy of annotation.

[0047] The appropriate large model instruction fine-tuning algorithm can be selected based on different application scenarios, such as full fine-tuning, efficient fine-tuning LoRA, and model quantization fine-tuning QLoRA. This flexibility allows the method to adapt to different hardware resources and time requirements, improving its practicality and applicability.

[0048] By calculating the cross-entropy of each category and summing them as the model's final loss, this method effectively measures the model's performance during training and guides its optimization. This helps ensure that the trained model has high accuracy in relation prediction tasks.

[0049] After loading the trained model, relation extraction can be automated by constructing relation extraction prompts and inputting them into the model. This automated process greatly improves the efficiency of relation prediction, enabling the method to quickly and accurately extract relationships between scientific and educational resources in practical applications.

[0050] This method comprehensively utilizes the zero-sample prediction and self-testing capabilities of large models, along with flexible fine-tuning strategies, to construct an efficient method for predicting relationships among science and education resources. This is of great significance for constructing high-quality science and education knowledge graphs and can promote the improvement of knowledge management and application in the field of science and education.

[0051] In summary, this technical solution effectively addresses the problem of scarce labeled data in the field of science and education through a series of innovative methods and techniques, improves the accuracy of relationship prediction, and provides strong support for constructing high-quality science and education knowledge graphs. Attached Figure Description

[0052] Figure 1 This is a flowchart of the present invention.

[0053] Figure 2 This is a schematic diagram of the automatic knowledge graph construction system in the field of science and education of this invention. Detailed Implementation

[0054] The present invention will be further described below with reference to the accompanying drawings:

[0055] Example 1, please refer to Figure 1 This invention provides a method for predicting the relationship between science and education resources based on zero-sample prediction using a large model, comprising:

[0056] We acquire unlabeled science and education resource data, and based on zero-shot prediction and self-testing of a large model, we divide the unlabeled science and education resource data into simple samples and difficult samples to form a few-sample training set.

[0057] The training set with a small number of samples is labeled manually.

[0058] Select the appropriate large model instruction fine-tuning algorithm according to different application scenarios, train the model on the labeled small sample training set, and obtain the trained model.

[0059] The pre-trained model is loaded to automatically extract relationships and predict them.

[0060] In the field of science and education, labeled data is often scarce and difficult to obtain. This invention utilizes the zero-shot prediction capability of a large model to perform preliminary processing on unlabeled science and education resource data without relying on any labeled data, thus providing a basis for subsequent model training.

[0061] By leveraging the self-checking capabilities of large models, unlabeled data is categorized into easy and difficult samples, further filtering out high-quality training samples to form a few-sample training set. This solves the problem that traditional machine learning methods require a large amount of labeled data for effective training.

[0062] This invention selects the most representative training samples from simple and difficult samples using methods such as uncertainty ranking, ensuring the quality of the training set. This helps improve the accuracy of the model in relation prediction tasks.

[0063] By selecting appropriate large-scale model instruction fine-tuning algorithms based on different application scenarios, the model can be finely adjusted. This personalized fine-tuning approach enables the model to better adapt to specific science and education resource relationship prediction tasks, further improving prediction accuracy.

[0064] By using carefully selected training samples and a large model instruction fine-tuning algorithm, the model can achieve good prediction results in fewer training rounds, thus improving training efficiency.

[0065] Thanks to the careful selection of training samples and the personalized choice of fine-tuning algorithms, the model can maintain stable prediction performance when facing different science and education resource relationship prediction tasks, reducing prediction fluctuations caused by data changes or task differences.

[0066] The method provided by this invention can efficiently extract relationships between science and education resources, providing strong support for constructing high-quality science and education knowledge graphs. This helps to improve knowledge management and application in the field of science and education, and promotes knowledge sharing and inheritance.

[0067] In summary, the method for predicting relationships between science and education resources based on zero-shot prediction of large models provided by this invention has shown significant technical effects in addressing the scarcity of labeled data, improving the accuracy of relationship prediction, enhancing model training efficiency and stability, and promoting the construction and application of science and education knowledge graphs.

[0068] Example 2: This invention provides a method for predicting the relationship between science and education resources based on zero-sample prediction using a large model, specifically including:

[0069] A knowledge graph construction system model for the science and education field: The proposed solution's framework, based on a large model with zero-shot adaptive training and few-shot training, is primarily applied to knowledge graph construction systems in the science and education field where labeled samples are scarce. With the continuous development of automatic knowledge graph construction technology, the knowledge graph construction system mainly consists of four modules: knowledge acquisition, ontology modeling, knowledge extraction, and knowledge storage. Figure 2 As shown in the diagram, the knowledge graph construction process first involves acquiring a large amount of knowledge from diverse and heterogeneous science and education resources, including structured, semi-structured, and unstructured knowledge. Secondly, ontology modeling is performed, aiming to design the conceptual layer of the acquired knowledge to standardize its representation. The knowledge extraction module typically consists of two sub-tasks: entity recognition and relation extraction. The training scheme in this paper primarily focuses on this module, aiming to fully leverage the few-shot transfer capabilities of the large model through reasonable sample selection. Knowledge storage is the final step in this system, used to manage and utilize the constructed knowledge graph. This system employs the open-source graph management tool Neo4j as its storage database.

[0070] The proposed few-shot training framework based on zero-shot adaptation of a large model runs on the aforementioned knowledge graph construction system in the science and education field. Its main functions rely on a large amount of unlabeled science and education data. The main functions are as follows: Faced with a large amount of unlabeled science and education data, the framework can adaptively select a small batch of samples that are most challenging for the current large model for manual annotation, based on the model's zero-shot capability, thereby maximizing training gains within a limited number of labeled samples. Furthermore, to prevent overfitting of the large model on difficult samples, a portion of aligned samples matching the model's zero-shot capability are also manually annotated, forming the few-shot fine-tuning training data together with the selected difficult samples. Secondly, the framework provides various large model instruction fine-tuning algorithms, including full fine-tuning, efficient fine-tuning (LoRA), and model quantization fine-tuning (QLoRA), which can be adjusted according to different application scenarios.

[0071] Operational Flow (Operational Flow of a Knowledge Graph Construction System)

[0072] First, the system acquires a large amount of heterogeneous, multi-source data in the science and education field through educational resources such as the internet, textbooks, and supplementary teaching materials. Uneditable resources are converted into editable resources using OCR and stored in the corpus. Second, based on the science and education data, ontology design is performed semi-automatically, primarily including three types of entity elements: teaching resources (such as tutorials, videos, and test questions), algorithm resources (such as frameworks, algorithms, and datasets), and computing power resources (such as GPUs and network devices). It also includes four types of relation sets: teaching-algorithm set, teaching-computing power set, algorithm-computing power set, and the entire set. Then, the system enters the knowledge extraction stage, which can be further divided into entity extraction and relation extraction based on the type of knowledge extracted. Entity extraction consists of two steps: named entity boundary recognition and entity classification. Relation extraction, based on entity extraction, predicts the potential relationships between entities. The proposed small-shot training framework based on zero-shot adaptation of a large model is applied to both the entity extraction and relation extraction stages, improving the accuracy and efficiency of knowledge extraction.

[0073] Taking relation extraction as an example, the training framework process is briefly described as follows: Based on a large amount of unlabeled corpus and extracted entities, the zero-shot capability of the large model is used to predict the relations between entity pairs in the same sample. Then, the correctness of the prediction is judged by the zero-shot self-testing capability of the large model, that is, the prediction result and the sample are reorganized into a conclusion, which is then input into the large model to ask whether it is correct. If it is correct, it means that the large model is relatively confident in the prediction result of the sample, and the training with such samples may not bring much benefit; if it is incorrect, it means that the large model is confused about the sample, which we call "hard sample". Training with hard samples can bring greater training benefits to the large model. Therefore, based on the results of the zero-shot self-testing of the large model, the unlabeled samples can be divided into two parts: simple samples and hard samples. In addition, this paper uses uncertainty to measure the reliability of the self-testing results of the large model. The smaller the uncertainty, the more confident the result. Therefore, this paper sorts the hard samples and simple samples of the self-testing results from low to high uncertainty, and selects the top K samples from the hard samples as training samples. To prevent severe overfitting during training, we select the top NK samples from the simple sample set as regularization samples for training, thus obtaining N training samples through the above steps. Next, these N training samples are manually labeled, and a suitable algorithm is selected from three fine-tuning algorithms based on different application scenarios to fine-tune the large model. At this point, the training task for relation extraction on the large model is complete. Finally, the trained large model can automatically perform relation extraction tasks.

[0074] Specific procedures:

[0075] 1. A few-shot training framework based on zero-shot adaptation of a large model, mainly including the following steps (taking relation extraction as an example):

[0076] High-quality sample selection: Based on the zero-shot prediction capability and self-testing capability of the large model, the framework divides unlabeled data into simple samples and difficult samples. By comprehensively selecting K difficult samples and NK simple samples through uncertainty, the final small sample training set is formed.

[0077] Small sample data annotation: After the above sample selection, the selected high-quality samples need to be manually annotated.

[0078] Large model few-shot fine-tuning: Select the appropriate large model instruction fine-tuning algorithm according to different application scenarios, including full fine-tuning, efficient fine-tuning (LoRA) and model quantization fine-tuning (QLoRA), and train the model on the labeled few-shot training set.

[0079] Large model prediction: Automatic relation extraction can be performed by loading a pre-trained model.

[0080] The selection of high-quality samples, as shown in Figure 1, mainly includes the following steps:

[0081] Zero-shot prediction: Existing unlabeled relation extraction dataset ,for Each sample Use a large model to make zero-shot predictions on it, i.e. ,in Indicates that for the first The prediction prompt for each sample is as follows: "Your task is to extract relations based on a given sentence, a predefined set of relations, and entity pairs.\nPredefined set of relations: 1. Teaching-Algorithm 2. Teaching-Computing Power 3. Computing Power-Algorithm 4. Other\nSentence: {... ...}\nEntity pair: {Entity 1, Entity 2}\nPlease answer the relation between {Entity 1} and {Entity 2} based on the above information."

[0082] Sample partitioning based on self-testing: Subsequently, based on the zero-shot prediction results of the large model, each sample... Relationship Predictions Design large model self-check prompts Specifically, the task is: "Your task is to determine whether a given conclusion is correct. Conclusion: In {given sentence}, the relationship between {entity 1} and {entity 2} is {relationship prediction}. Please determine whether this conclusion is correct based on the above content, answering with 'yes' or 'no'," and perform a large-scale model self-check, i.e. Based on the results of the large model self-check Unlabeled datasets can be used Divided into and Two sets, where This indicates a difficult sample, meaning the self-test results of the large model are inconsistent with its predictions, indicating an internal contradiction regarding that sample. Therefore, training on this sample can yield better training results; conversely, large models can perform better on other samples. The simple samples in the model show consistent results in both the prediction and self-testing stages, indicating that the large model is relatively confident in the results. These samples represent the capabilities of the large model itself to some extent, and training on these samples yields little benefit.

[0083] Uncertainty Ranking: To further select high-quality samples, this paper designs uncertainties to measure... The difficulty level of the medium-difficulty samples, i.e. ,in express The maximum probability distribution. Then, based on the uncertainty of each self-test result, the difficult samples are ranked, and the top... One sample is used as the training sample to be labeled.

[0084] Construction of a high-quality training set: At this point, the training set has been selected. There are several difficult samples. Furthermore, since there is an implicit distribution of difficult samples, to prevent large models from suffering severe overfitting on this distribution, which would cause them to lose their predictive ability for simple samples, this scheme will rank the simple samples from low to high uncertainty and select the top... A few simple samples were added as regularization samples for training. Finally, after the above steps, a total of [number] samples were selected. A number of high-quality training samples awaiting labeling.

[0085] As shown in Figure 1, the annotation of few-sample data mainly includes the following steps:

[0086] General-purpose large-scale model automatic annotation: To improve annotation efficiency, training samples to be annotated will be automatically annotated using existing general-purpose large-scale models, such as Tongyi Qianwen and Wenxin Yiyan. Specific prompts are designed as follows: "Your task is to extract relations based on a given sentence, a predefined set of relations, and entity pairs.\nPredefined set of relations: 1. Teaching-Algorithm 2. Teaching-Computing Power 3. Computing Power-Algorithm 4. Other\nSentence: {... ...}\nEntity pair: {Entity 1, Entity 2}\nPlease answer the relation between {Entity 1} and {Entity 2} based on the above information."

[0087] Manual correction: To improve the accuracy of annotation, manual correction is required for the automatically annotated data.

[0088] Manual review: To further improve the reliability of the labeled data, the system will submit the manually corrected data to professionals for review. If the review fails, it means that the sample is ambiguous and step (2) needs to be repeated.

[0089] As shown in Figure 1, fine-tuning of a large model with few samples mainly includes the following steps:

[0090] Fine-tuning algorithm selection: Appropriate large model instruction fine-tuning algorithms are selected based on different application scenarios, including full fine-tuning, efficient fine-tuning (LoRA), and model quantization fine-tuning (QLoRA). The model is trained on a labeled, small-sample training set. Full fine-tuning has high requirements for GPU memory, while efficient fine-tuning freezes all model parameters and trains by adding additional trainable parameters. Model quantization has lower device requirements but causes a significant loss in model accuracy.

[0091] Model fine-tuning training: After determining the fine-tuning method, the cross-entropy of each class is calculated and summed as the final loss of the model, i.e. .

[0092] As shown in Figure 1, large model predictions mainly involve the following steps:

[0093] Model Loading: For models undergoing full fine-tuning, the system only needs to load the fine-tuned large model. For models undergoing efficient fine-tuning, the system first needs to load the parameters of the base model, and then load the additional model weights added during the fine-tuning process. For models undergoing quantization fine-tuning, the system first needs to load the quantized base model, and then load the additional trained model parameters.

[0094] Relation Extraction Hint Construction: When applying a large model to a relation extraction task, it is necessary to design corresponding hints to guide the large model to complete the relevant task. The specific hint design is as follows: "Your task is to extract relations based on a given sentence, a predefined set of relations, and entity pairs.\nPredefined set of relations: 1. Teaching-Algorithm 2. Teaching-Computing Power 3. Computing Power-Algorithm 4. Other\nSentence: {... ...}\nEntity pair: {Entity 1, Entity 2}\nPlease answer the relation between {Entity 1} and {Entity 2} based on the above content."

[0095] Relationship prediction: The automatic extraction of relations can be completed by inputting the constructed prompts into the loaded model.

[0096] This invention comprehensively utilizes the zero-shot prediction and self-testing capabilities of large models to construct a high-yield training sample selection mechanism in few-shot scenarios. It does not rely on any labeled data and is successfully applied to the construction of knowledge graphs in the science and education field where labeled data is scarce, based on a knowledge graph construction system. Compared with most current few-shot solutions based on random sampling, the proposed solution improves the fine-tuning effect of large model instructions while taking into account the stability of few-shot fine-tuning, which is of great significance for constructing high-quality science and education knowledge graphs.

[0097] In another embodiment of the present invention, a science and education resource relationship prediction system based on zero-shot prediction of a large model is provided, which can be used to implement the above-mentioned science and education resource relationship prediction method based on zero-shot prediction of a large model. Specifically, the system includes:

[0098] The data acquisition module is used to acquire unlabeled science and education resource data. Based on the zero-shot prediction and self-check of the large model, the unlabeled science and education resource data is divided into simple samples and difficult samples to form a few-sample training set.

[0099] The data annotation module is used to manually annotate a small number of training samples.

[0100] The model training module is used to select the appropriate large model instruction fine-tuning algorithm according to different application scenarios, and train the model on the labeled small sample training set to obtain the trained model.

[0101] The prediction output module is used to load the trained model, perform automated relation extraction, and complete relation prediction.

[0102] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0103] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions from the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of a science and education resource relationship prediction method based on large-model zero-sample prediction.

[0104] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the science and education resource relationship prediction method based on large model zero-sample prediction in the above embodiments.

[0105] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0106] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0107] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0108] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for predicting the relationship between science and education resources based on zero-sample prediction using a large model, characterized in that, include: We acquire unlabeled science and education resource data, and based on zero-shot prediction and self-testing of a large model, we divide the unlabeled science and education resource data into simple samples and difficult samples to form a few-sample training set. The training set with a small number of samples is labeled manually. Select the appropriate large model instruction fine-tuning algorithm according to different application scenarios, train the model on the labeled small sample training set, and obtain the trained model. Load the pre-trained model to automatically extract relations and complete relation prediction; Zero-shot prediction and self-testing based on a large model divide unlabeled science and education resource data into simple and difficult samples, forming a few-shot training set, including: Zero-shot prediction: Existing unlabeled relation extraction dataset ,for Each sample Use a large model to make zero-shot predictions on it, i.e. ,in Indicates that for the first Predictive hints for individual sample designs; Sample partitioning based on self-testing: Subsequently, based on the zero-shot prediction results of the large model, each sample... Relationship Predictions Design large model self-check prompts And perform a large model self-check, that is Based on the results of the large model self-check Unlabeled dataset Divided into and Two sets, where Indicates a difficult sample. Represents a simple sample; Uncertainty ranking: Uncertainty is used to measure... The degree of difficulty of the medium-difficulty samples, i.e. ,in express The maximum probability distribution is then determined; subsequently, the difficult samples are ranked according to the uncertainty of each self-test result, and the top ones are selected. One sample is used as the training sample to be labeled; Construction of high-quality training sets: [Sets have been selected] Given a set of difficult samples, the easy samples are ranked from low to high uncertainty, and the top samples are selected. A simple sample is added as a regularization sample for training, and finally, the sample is selected. One high-quality training sample to be labeled; Depending on the different application scenarios, the appropriate large model instruction fine-tuning algorithm can be selected, including: Select the appropriate large model instruction fine-tuning algorithm according to different application scenarios, including full fine-tuning, efficient fine-tuning LoRA and model quantization fine-tuning QLoRA, and train the model on a labeled small sample training set.

2. The method for predicting the relationship between science and education resources based on zero-sample prediction of a large model according to claim 1, characterized in that, Acquiring unlabeled science and education resource data includes: obtaining multi-source heterogeneous science and education data through educational resources, and converting uneditable resources into editable resources through OCR recognition and storing them in a corpus.

3. The method for predicting the relationship between science and education resources based on zero-sample prediction of a large model according to claim 1, characterized in that, Manual annotation of a small training set includes: Automatic annotation of general large models: For training samples to be labeled, automatic annotation is performed first using existing general large models; Manual corrections are then performed on the automatically labeled data. The manually corrected data will be manually reviewed. If the review fails, the data will be corrected again.

4. The method for predicting the relationship between science and education resources based on zero-sample prediction of a large model according to claim 1, characterized in that, The model is trained on a labeled small sample training set to obtain a trained model, including: After determining the fine-tuning method, the cross-entropy of each category is calculated and summed to obtain the final loss of the model. , where K represents the number of training samples and M represents the sequence length of each training sample.

5. The method for predicting the relationship between science and education resources based on zero-sample prediction of a large model according to claim 1, characterized in that, Load the pre-trained model to automatically extract relations and perform relation prediction, including: Model loading: For models under full fine-tuning, load the fine-tuned large model; for models under efficient fine-tuning, first load the parameters of the base model, and then load the model weights added during the fine-tuning process; for models under quantized fine-tuning, first load the quantized base model, and then load the additional trained model parameters. Relation extraction hint construction: When applying a large model to a relation extraction task, design hints to guide the large model to complete the relevant task; Relationship prediction: Automatically extract relationships from the pre-constructed prompts and loaded model.

6. A science and education resource relationship prediction system based on large-scale model zero-sample prediction, characterized in that, include: The data acquisition module is used to acquire unlabeled science and education resource data. Based on the zero-shot prediction and self-check of the large model, the unlabeled science and education resource data is divided into simple samples and difficult samples to form a few-sample training set. The data annotation module is used to manually annotate a small number of training samples. The model training module is used to select the appropriate large model instruction fine-tuning algorithm according to different application scenarios, and train the model on the labeled small sample training set to obtain the trained model. The prediction output module is used to load the trained model to automatically extract relations and complete relation prediction. Zero-shot prediction and self-testing based on a large model divide unlabeled science and education resource data into simple and difficult samples, forming a few-shot training set, including: Zero-shot prediction: Existing unlabeled relation extraction dataset ,for Each sample Use a large model to make zero-shot predictions on it, i.e. ,in Indicates that for the first Predictive hints for individual sample designs; Sample partitioning based on self-testing: Subsequently, based on the zero-shot prediction results of the large model, each sample... Relationship Predictions Design large model self-check prompts And perform a large model self-check, that is Based on the results of the large model self-check Unlabeled dataset Divided into and Two sets, where Indicates a difficult sample. Represents a simple sample; Uncertainty ranking: Uncertainty is used to measure... The degree of difficulty of the medium-difficulty samples, i.e. ,in express The maximum probability distribution is then determined; subsequently, the difficult samples are ranked according to the uncertainty of each self-test result, and the top ones are selected. One sample is used as the training sample to be labeled; Construction of high-quality training sets: [Sets have been selected] Given a set of difficult samples, the easy samples are ranked from low to high uncertainty, and the top samples are selected. A simple sample is added as a regularization sample for training, and finally, the sample is selected. One high-quality training sample to be labeled; Depending on the different application scenarios, the appropriate large model instruction fine-tuning algorithm can be selected, including: Select the appropriate large model instruction fine-tuning algorithm according to different application scenarios, including full fine-tuning, efficient fine-tuning LoRA and model quantization fine-tuning QLoRA, and train the model on a labeled small sample training set.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method for predicting the relationship between science and education resources based on zero-sample prediction of a large model as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for predicting the relationship between science and education resources based on zero-sample prediction of a large model as described in any one of claims 1 to 5.

Images