Non-abandoned text relation extraction method, system and equipment and medium

Abstract

The invention discloses a non-abandoned text relation extraction method, system and device and a medium. The method comprises the steps of obtaining a to-be-extracted non-abandoned text; filling the non-abandoned text to be extracted into a Prompt template; the Prompt template filled with the to-be-extracted non-abandoned text is input into the large language model, the semantic relation between the entities in the to-be-extracted non-abandoned text is obtained, and the semantic relation between the entities in the non-abandoned text can be extracted through the method, the system, the equipment and the medium.

Description

Technical Field

[0001] The present invention belongs to the technical field of feature extraction and relates to a method, system, device and medium for extracting intangible cultural heritage text relations. Background Art

[0002] As an important cultural resource, intangible cultural heritage (hereinafter referred to as ICH) carries rich historical and regional cultural information. Its protection and inheritance have received increasing attention in recent years. ICH texts record core information such as origin, inheritors, and categories. They contain a large number of terms and expressions with regional characteristics, reflecting the distinct regional characteristics and complex semantic structure of ICH culture. However, the digitization and semantic processing of ICH information still face many technical challenges. The unique semantic structure and rich background knowledge in these texts are beyond the scope of conventional natural language processing (NLP) tasks. In particular, a large number of regional vocabulary and unique cultural expressions pose huge challenges to the task of relationship extraction.

[0003] Traditional relationship extraction methods mostly rely on predefined templates or rules. Although they are applicable to some general fields, they are often difficult to accurately identify and extract key entity relationships when faced with the language diversity and complex semantic structure of intangible cultural heritage texts. In recent years, with the rapid development of large-scale pre-trained language models, their capabilities in semantic understanding and text generation have provided new possibilities for relationship extraction of intangible cultural heritage texts. However, directly applying general large models to relationship extraction tasks in the field of intangible cultural heritage may lead to performance degradation due to semantic differences. At the same time, traditional fine-tuning methods have high computational costs on large models and are difficult to adapt to resource-constrained scenarios. Therefore, how to effectively fine-tune large models under the condition of limited annotated data to adapt them to the specific needs of the field of intangible cultural heritage has become an urgent problem to be solved.

[0004] Relation Extraction (RE) is an important task in natural language processing, which aims to automatically identify and extract semantic relationships between entities from unstructured text. Since the rise of information extraction research, relation extraction has always been the focus of attention in the field and plays an important role in applications such as intelligent question answering and knowledge graph construction. In recent years, with the rapid development of deep learning and large-scale pre-training models, relation extraction technology has made significant progress.

[0005] Judging from the application fields of the above-mentioned relationship extraction related research, most of them are concentrated in the fields of medicine, agriculture, public security, etc., while there are fewer studies on relationship extraction in the field of intangible cultural heritage. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a method, system, device and medium for extracting relations between intangible cultural heritage texts. The method, system, device and medium can extract semantic relations between entities in intangible cultural heritage texts.

[0007] In order to achieve the above object, the present invention adopts the following technical scheme:

[0008] In one aspect, the present invention provides a method for extracting intangible cultural heritage text relations, comprising:

[0009] Get the intangible cultural heritage text to be extracted;

[0010] Fill the intangible cultural heritage text to be extracted into the Prompt template;

[0011] The Prompt template filled with the intangible cultural heritage text to be extracted is input into the large language model to obtain the semantic relationship between the entities in the intangible cultural heritage text to be extracted.

[0012] The method for extracting intangible cultural heritage text relations described in the present invention is further improved in that:

[0013] Furthermore, before the Prompt template filled with the intangible cultural heritage text to be extracted is input into the large language model, the following steps are also included:

[0014] Constructing an intangible cultural heritage dataset;

[0015] Build large language models;

[0016] Based on the few-sample learning strategy, the large language model is trained using the intangible cultural heritage dataset to obtain the trained large language model.

[0017] Furthermore, the process of constructing the intangible cultural heritage dataset is as follows:

[0018] Acquire text data in the form of a web crawler, and perform noise data removal, non-text data deletion, and format conversion on the text data to form an initial intangible cultural heritage text data set;

[0019] Sorting out the intangible cultural heritage entity relationships of the initial intangible cultural heritage text data set;

[0020] According to the combing results, the data in the initial intangible cultural heritage text data set is labeled to obtain an intangible cultural heritage data set.

[0021] Furthermore, the process of labeling the data in the initial intangible cultural heritage text data set is:

[0022] Set intangible cultural heritage entity tags;

[0023] Set the intangible cultural heritage entity relationship label;

[0024] The data in the initial intangible cultural heritage text dataset are labeled according to the set intangible cultural heritage entity labels and intangible cultural heritage entity relationship labels.

[0025] Furthermore, the process of constructing a large language model is as follows:

[0026] Fine-tune LoRA on the large language model;

[0027] Design a Prompt template for a large language model.

[0028] Furthermore, the Prompt template of the large language model is:

[0029]

[0030] The present invention discloses a non-legacy text relationship extraction system, comprising:

[0031] The first building module is used to construct the intangible cultural heritage dataset;

[0032] The second building module is used to build a large language model;

[0033] The training acquisition module is used to train the large language model based on the few-sample learning strategy and use the intangible cultural heritage dataset to obtain the trained large language model.

[0034] An acquisition module is used to obtain the intangible cultural heritage text to be extracted;

[0035] A filling module is used to fill the intangible cultural heritage text to be extracted into the Prompt template;

[0036] The extraction module is used to input the Prompt template filled with the intangible cultural heritage text to be extracted into the large language model to obtain the semantic relationship between the entities in the intangible cultural heritage text to be extracted.

[0037] The present invention discloses a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the intangible cultural heritage text relationship extraction method are implemented.

[0038] The present invention discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the intangible cultural heritage text relationship extraction method are implemented.

[0039] The present invention has the following beneficial effects:

[0040] The intangible cultural heritage text relationship extraction method, system, device and medium of the present invention use a large language model to extract semantic relationships during specific operations, and the operation is simple and convenient. Specifically, the intangible cultural heritage text to be extracted is filled into a Prompt template; the Prompt template filled with the intangible cultural heritage text to be extracted is input into the large language model to obtain the semantic relationships between entities in the intangible cultural heritage text to be extracted, and the practicability is extremely strong. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0042] Figure 1 A diagram of the research framework based on large language models and few-sample prompt learning;

[0043] Figure 2 A flowchart for sorting out the relationships in the seven-step method;

[0044] Figure 3 Label example images for intangible cultural heritage entities;

[0045] Figure 4 Label example graphs for entity relationships;

[0046] Figure 5 Schematic diagram of ChatGLM self-attention mask mechanism;

[0047] Figure 6 This is the LoRA structure diagram;

[0048] Figure 7 This is a graph showing the proportion of different intangible cultural heritage relationship categories in the dataset;

[0049] Figure 8 This is a comparison chart of the extraction results of the ChatGLM model on each relationship;

[0050] Fig. 9 Comparison chart of relationship extraction results for each model;

[0051] Fig.10 This is a comparison chart of the relationship extraction results of the ablation experiment. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only an embodiment of a part of the present invention, not all embodiments, and is not intended to limit the scope of the present invention. In addition, in the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concepts disclosed in the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present invention.

[0053] The accompanying drawings show schematic diagrams of structures according to embodiments disclosed in the present invention. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0054] Embodiment 1

[0055] It should be noted that the present invention uses LoRA fine-tuning technology and few-sample learning strategy to enhance the performance of large language models in relation recognition in the field of intangible cultural heritage. Figure 1 As shown. First, by collecting and preprocessing text data from intangible cultural heritage websites and offline resources, an intangible cultural heritage dataset is constructed, and the entity relationships in the intangible cultural heritage dataset are sorted and annotated. Then, the LoRA technology is used to fine-tune the large language model, and the parameter adjustment of low-rank decomposition is used to improve the large language model's recognition ability for specific entity relationships in the intangible cultural heritage field. Subsequently, a Prompt template is designed to guide the large language model to better adapt to the semantic expression characteristics of the intangible cultural heritage field and improve the recognition accuracy of common relationship structures. In the few-shot learning stage, based on the few-shot idea, three schemes of 1-shot, 2-shot and 3-shot are gradually adopted to provide the model with different numbers of labeled samples to enhance the large language model's recognition ability for diversified relationships in the intangible cultural heritage field. The above steps are used to optimize the performance of the large language model in the intangible cultural heritage relationship extraction task.

[0056] Specifically, the intangible cultural heritage text relationship extraction method of the present invention comprises the following steps:

[0057] 1) Construct an intangible cultural heritage dataset;

[0058] 11) Data acquisition and processing;

[0059] The intangible cultural heritage resources are rich, including more than 100,000 intangible cultural heritage items in 10 categories such as folk literature and traditional music. Text data is obtained through web crawlers, and a series of operations such as noise data cleaning, non-text data deletion, and format conversion are performed on the text data to form an initial intangible cultural heritage text data set.

[0060] 12) Sorting out the intangible cultural heritage entity relationships of the initial intangible cultural heritage text data set;

[0061] The present invention draws on the seven-step construction strategy of Stanford University to analyze all entity relationships contained in the initial intangible cultural heritage text dataset. The steps are as follows: Figure 2 shown.

[0062] Specifically, the entire initial intangible cultural heritage text data set is text data related to the intangible cultural heritage field, including project name, inheritor, project category, project level, region, project batch and project characteristics. The reuse of the ontology model can save time and cost, improve the consistency and quality of the model, and support expansion and sustainable development. Therefore, the reuse of the existing ontology model is considered. CIDOC CRM provides a set of standardized ontology vocabulary and models for describing various entities and relationships in the field of cultural heritage. The CIDOC CRM ontology model is selected to construct the intangible cultural heritage ontology model of the present invention.

[0063] List the important terms related to intangible cultural heritage. Taking the traditional opera Qinqiang as an example, the key terms involved are: Qinqiang (project name), Shaanxi Province (region), traditional opera (category), Ma Youxian (inheritor), etc. These terms will serve as the basic knowledge elements of Qinqiang to support further sorting of entity relationships.

[0064] Then, the listed terms are constructed into classes and their hierarchical structure is defined. The attributes and value ranges are defined for each class, and specific data are filled into the instances of each class. The final entity relationship is shown in Table 1 below.

[0065] Table 1

[0066]

[0067] 13) Annotate the data in the initial intangible cultural heritage text dataset;

[0068] Specifically, in the task of relation extraction, data annotation can enhance the understanding of context and semantic dependencies of the large language model. The present invention uses the annotation criteria BIO commonly used in the field of natural language processing to annotate the initial intangible cultural heritage data set. With the help of Label Studio, an open source data annotation tool, manual annotation is performed. The data annotation process is as follows:

[0069] 131) Define intangible cultural heritage entity labels. For example, the intangible cultural heritage entity categories in Shaanxi Province include: traditional folk art, traditional drama, traditional skills, traditional dance, traditional medicine, traditional sports and acrobatics, traditional folk customs, traditional music, folk literature, and traditional fine arts. Based on these entity categories, design 10 corresponding intangible cultural heritage entity labels.

[0070] 132) Define the intangible cultural heritage entity relationship labels. According to the results of sorting the entity relationships obtained in step 12), define 6 entity relationship labels, including belonging to, including, inherited, distributed in, graded and batched.

[0071] 133) Data annotation is performed to obtain the intangible cultural heritage text dataset. Each text in the initial intangible cultural heritage text dataset is annotated with BIO, where the "BX" label indicates that the entity to which an element belongs is of type X and the element is the starting part of the entity; the "IX" label indicates that the element is located in the middle of the X-type entity; and the "O" label indicates that the element does not belong to any specific entity type. Figure 3 and Figure 4 Examples of entity and entity relationship annotation based on BIO criteria are shown respectively.

[0072] 2) Fine-tune the large language model using LoRA;

[0073] The ChatGLM large language model contains a large number of parameters in its structure, mainly composed of weight matrices in the self-attention layer and the feedforward network layer. The large language model has strong contextual understanding and generation capabilities in general text understanding, thanks to its unique attention mask mechanism. Figure 5 The specific implementation process of this mechanism is shown in the following. First, the large language model takes the input sequence {x 1 ,x 2 ,x 3 ,x 4 ,x 5 ,x 6} randomly select two fragments, such as Figure 5 The x shown in a 3 and x 5 、x 6 .exist Figure 5 In b, these extracted fragments are replaced by the mask marker [MASK] to form two parts: Part A as the context prompt part and Part B as the generation target.

[0074] exist Figure 5In c, the model autoregressively predicts the content of Part B through two position encoding mechanisms: Position 1 and Position 2. Among them, Position 1 encodes the absolute position of the entire input sequence, and Position 2 is used for the internal relative position encoding of each masked segment, so that the model can accurately capture the order information between segments during the generation process.

[0075] Figure 5 d shows the attention mask design of ChatGLM. In the Part A area (blue box), each token can access each other. This part serves as a prompt, providing contextual information for the input, and is not affected by the content of Part B, which enhances the model's contextual understanding ability. Part B is the part that the model needs to generate, which is divided into two sub-areas: the first segment x 5 ,x 6 (yellow box) and the second segment x 3 (Green box). The first segment uses an autoregressive masking mechanism (lower triangle structure) to ensure that each token can only see itself and the previous token. 3 When the model can access not only itself, but also the previously generated x 5 ,x 6 This design helps GLM to make predictions step by step when generating complex texts, improving its performance in natural language understanding and generation tasks.

[0076] Although ChatGLM has powerful general capabilities, it is challenging to directly fine-tune all model parameters. Since the model parameters are large and distributed in multiple weight matrices, directly adjusting each parameter of these matrices consumes a lot of computing resources and it is difficult to train ideal results on limited data. Therefore, some optimization methods are usually introduced during fine-tuning to adapt to specific downstream task requirements.

[0077] To solve the above problems, LoRA simplifies the fine-tuning process through matrix decomposition. By introducing two low-rank matrices, it avoids adjusting the entire weight matrix. It only needs to learn the parameters of two smaller matrices, which greatly reduces the computational requirements during fine-tuning. Figure 6 Based on the idea of ​​matrix decomposition of LoRA, we can see that the weight of the large language model is a d×d dimensional matrix, and LoRA uses the multiplication of two matrices (d,r)*(r,d) to get d×d, and then the two low-rank sub-matrix features are added together. Among them, r represents the hyperparameter of the model, which is the rank of the W matrix in the blue sub-figure on the left.

[0078] The relevant calculation formula is:

[0079] h=W 0x+ΔW x (1)

[0080] ΔW x =BA x (2)

[0081] Among them, h represents the output of the fine-tuned model; W 0 represents the original weight matrix, all parameters in this matrix have been learned in pre-training and remain unchanged in the fine-tuning process; x represents the input to the model to the data; ΔW represents the weight adjustment matrix, B and A represent two low-rank decomposition matrices, where B∈R d×r , A∈R r×k .

[0082] When fine-tuning, for W 0 Freeze the parameters in, initialize A with a random Gaussian, initialize B with 0, calculate the loss gradient through back propagation, and apply it to the parameter update on matrices A and B, and adjust the values ​​of these two matrices to adapt to the task requirements.

[0083] 3) Design the Prompt template for the ChatGLM large language model;

[0084] As a generative model, the performance of the ChatGLM large language model depends greatly on the output format that can understand and adapt to different task requirements. In order to make the model adapt to the unique expression characteristics of intangible cultural heritage texts and enable it to achieve better relationship extraction effects in a small number of sample data learning scenarios, a Prompt prompt message adapted to this task is designed. The construction of the entire Prompt template is divided into the following steps:

[0085] 31) Clarify the model role, set the role identity of ChatGLM and provide it with the answer requirements from the perspective of a relationship extraction researcher. The specific input content is: "You are a natural language processing expert who is very good at relationship extraction tasks, especially extracting relationship information in a specified format from intangible cultural heritage text corpus."

[0086] 32) Design a suitable Prompt template. Considering that there is more than one set of entity relationships in an intangible cultural heritage text sentence, design a combination of prompts to guide the model, clearly tell the model which categories of entity relationships need to be identified, and the content format of the relationship extraction results output to the user. The specific input content is: "Now you need to help me complete the entity relationship extraction task. When I give you a sentence containing an intangible cultural heritage project, you need to help me extract the relationship triples in the sentence, including: the relationship between the name of the intangible cultural heritage project and its corresponding category, project batch, project level, region, inheritor and project characteristics, and output it in JSON format, with values ​​inside the relationship separated by ','."

[0087] Use contextual cues to illustrate the model in the initial stages of training. Give the model examples to further clarify the tasks it needs to complete.

[0088] The designed relation extraction prompt template is shown in Table 2:

[0089] Table 2

[0090]

[0091]

[0092] 4) Few-shot learning strategy to train large language models;

[0093] Few-shot learning is an important strategy for in-context prompting. As an efficient learning paradigm, its core lies in using a very small number of labeled samples to enable the model to generalize to new and unseen instances. The learning process is usually divided into a variety of typical settings according to the number of samples. Among them, 1-shot learning refers to the model completing a specific task under extreme conditions (i.e., relying only on a single training sample); while k-shot learning extends this concept, allowing the model to use k samples (such as 2-shot or 3-shot) for more robust learning to enhance its generalization ability.

[0094] In the task of extracting relations in the field of intangible cultural heritage, based on the principle of few-shot learning, three learning strategies including 1-shot, 2-shot and 3-shot are designed to improve the model's ability to recognize six types of intangible cultural heritage entity relations. These six types of relations include "belonging to", "batch division", "level division", "distributed in", "inherited" and "included". The process is as follows:

[0095] 1-shot learning strategy: In this strategy, a single sample sentence containing only the "belongs to" relationship is first input into ChatGLM for learning. Then, sample sentences containing other types of relationships ("batch division", "level division", "distributed in", "inherited by", and "contains") are gradually added and input into the model in sequence for learning to achieve the model's adaptation to different relationships.

[0096] 2-shot learning strategy: Based on the 1-shot strategy, in order to further enhance the generalization ability of the model, a second sample sentence that is not repeated with the previous sample is added. During the training process, the relationship types of the newly added sample sentences follow the increasing order in the 1-shot strategy. Through the complementary effect of double samples, the learning effect of the model on different relationship categories is improved.

[0097] 3-shot learning strategy: Based on the 2-shot strategy, a third different sample sentence is added to ensure that the sample is not repeated with the previous two. The relationship categories in the sample continue to increase in the previous order and are input into ChatGLM for training to further improve the model's relationship recognition ability.

[0098] By gradually increasing the number of samples and introducing different types of entity relationships, the few-shot learning strategy can enhance the model’s generalization ability for multiple relationships and significantly reduce the ChatGLM model’s dependence on large-scale labeled data. Table 3 shows the specific implementation details of the above learning strategy.

[0099] Table 3

[0100]

[0101]

[0102]

[0103] 5) Obtain the intangible cultural heritage text to be extracted;

[0104] 6) Fill the intangible cultural heritage text to be extracted into the Prompt template;

[0105] 7) Inputting the Prompt template filled with the intangible cultural heritage text to be extracted into the large language model to obtain the semantic relationship between the entities in the intangible cultural heritage text to be extracted.

[0106] Confirmatory test

[0107] The experimental environment for this experiment is based on Python 3.10 and PyTorch 2.3.0. The ChatGLM3-6b model is used as the large language model. The model has 6.2 billion parameters (6B) and is designed based on the general language model architecture GLM. The LoRA method is used for model fine-tuning. During fine-tuning, the maximum input length is set to 512, the batch size is set to 8, and the initial learning rate is 2e-5.

[0108] The intangible cultural heritage text dataset is used as the experimental dataset for model training and verification. The dataset covers 91 national and 766 provincial intangible cultural heritage items, including 6 types of intangible cultural heritage entity relationships and 10,667 entity relationships. The intangible cultural heritage entity relationships are annotated for both the training set and the verification set, while the test set remains unannotated for model evaluation. Figure 7 The number and proportion of relationships in different categories.

[0109] When evaluating the performance of the model in the relation extraction task, this experiment uses three common evaluation indicators: precision (P), recall (R), and F 1Value (F 1 -score). Precision focuses on the proportion of instances predicted by the model as positive samples that are actually positive samples. Recall measures how many of all the instances that are actually positive samples are correctly identified as positive samples by the model, which reflects the model's ability to capture positive samples. In order to comprehensively evaluate the performance of the model in terms of both precision and capture, F 1 The value is taken as the harmonic mean of precision and recall, thus providing a more comprehensive evaluation perspective. The calculation formula of the evaluation index is as follows.

[0110]

[0111]

[0112] Among them, TP (True Positive) indicates the number of samples correctly predicted by the model as positive, that is, the number of correctly identified relation instances. FP (False Positive) indicates the number of samples incorrectly predicted by the model as positive, that is, the number of samples that are not actually relation instances but are incorrectly identified as relation instances. FN (False Negative) indicates the number of samples incorrectly predicted by the model as negative, that is, the number of samples that actually have relation instances but are not identified by the model.

[0113] Comparative experiments and result analysis;

[0114] Table 4 shows the ability to extract the same type of relations based on different learning strategies based on k-shot (k = 1, 2, 3). It can be seen that as the k value increases (from 1 to 3), the model's ability to identify triples is significantly improved, especially for the extraction of complex and multi-dimensional relations. Specifically:

[0115] 1-shot learning strategy;

[0116] Under this strategy, the model only learns one sample sentence and can only extract the most basic relationship type. The recognition accuracy is low and the performance is limited when recognizing complex relationships. The relationship between the project name "Xi'an Roujiamo" and the project category "traditional skills" is incorrectly identified as [Xi'an Roujiamo, as, traditional skills]; when analyzing the "Suimi Suona" intangible cultural heritage entity, only two relationships [Suimi Suona, belongs to, traditional music category] and [Suimi Suona, batch division, second batch] are identified, and the relationship between the intangible cultural heritage project and the location is not identified. This shows that due to the insufficient number of samples provided by the 1-shot strategy, the model relies more on previous general domain knowledge and generalization ability, but is weak in the specific field or details of intangible cultural heritage.

[0117] 2-shot learning strategy;

[0118] Under this strategy, the model can learn from two sample sentences. Under the 2-shot strategy, the performance of the model is improved compared to the 1-shot recognition effect, but when identifying the intangible cultural heritage entity relationship of "Yulin Xiaoqu", it only recognizes [Yulin Xiaoqu, distributed in, Yulin City], [Yulin Xiaoqu, belongs to, Quyi category], [Yulin Xiaoqu, batch division, first batch], [Yulin Xiaoqu, level division, national level], [Yulin Xiaoqu, inherited, Gao Jinghong], and misses the "inclusion" relationship [Yulin Xiaoqu, includes, Xiaoqu and chorus].

[0119] This shows that 2-shot allows the model to learn more contextual features than 1-shot, and can more accurately extract most correct triplets, especially the basic "belongs to" and "batch division" relationships. However, more complex multi-category relationships may be missed, and the generated triplets are not complete.

[0120] 3-shot learning strategy;

[0121] The model using the 3-shot learning strategy performed the most accurately and comprehensively in identifying relation triplets. Almost all examples successfully returned correct triplets, including relation categories such as "belongs to", "batch division", "level division", "distributed in", "contains", and "inherited". This shows that more learning samples enhance the generalization ability of the model, especially in triplets involving multiple relation categories at the same time.

[0122] In particular, when extracting the entity relationship of "Zhu's Bone-setting Skills", the relationship triple between the project name and category [Zhu's Bone-setting Skills, belongs to, traditional medicine] can also be extracted for the relationship type "belongs to" that is not mentioned in the text. This is because after the previous few-shot learning, the model has been exposed to and learned many similar entities and their relationship patterns. When there are other "traditional medicine" intangible cultural heritage projects in the table (such as "longevity pills"), this pattern can help the model quickly capture similar structures and relationships and generalize them to "Zhu's Bone-setting Skills". This memory effect allows the model to infer and fill in the relationships that are not explicitly mentioned even when faced with slightly different texts.

[0123] Table 4

[0124]

[0125]

[0126]

[0127]

[0128] According to the comparative experimental results, the 3-shot strategy is selected for the few-shot prompt learning of the model. Figure 8 The results of extracting different types of relationship categories of the fine-tuned ChatGLM model are shown. It can be seen that the model performs well in the categories of "batch division", "belonging to" and "level division", especially the F1 value of the "batch division" relationship is as high as 0.9756.

[0129] To further verify the performance of the optimized model for relation extraction, we conducted comparative experiments with commonly used deep learning models and large language models for relation extraction. The comparative experimental models are introduced as follows:

[0130] BERT model: In relation extraction, this combined model is suitable for named entity recognition (NER) and sequence labeling tasks. BiLSTM improves the ability to capture long-range dependencies, while CRF optimizes the label prediction of the global sequence.

[0131] GPT-3.5-Turbo model: GPT-3.5 is one of the latest versions of the generative pre-training (GPT) model released by OpenAI. GPT-3.5 can handle relationship extraction tasks, and can automatically extract relationships from text through dialogue generation and completion, especially in the case of few samples or unsupervised learning.

[0132] Tongyi Qianwen: It is a large language model developed by Alibaba, based on the Transformer architecture, with powerful text generation and comprehension capabilities. It is widely used in Chinese NLP tasks, especially question answering, translation, text generation and other scenarios.

[0133] Wenxinyiyan: It is a Chinese language model launched by Baidu, based on the ERNIE architecture. The model combines external knowledge graphs and pre-trained language models by introducing a knowledge enhancement mechanism, thus improving the performance of the model in knowledge-intensive tasks.

[0134] Table 5 shows the comparison of the results of each model on the relation extraction task based on the test set data.

[0135] Table 5

[0136]

[0137]

[0138] From the above comparative experimental data, we can see that: in the "belongs to" relationship extraction, the F1 value of ChatGLM-6b is 0.9302, which outperforms most models, such as GPT-3.5-Turbo's 0.9268 and Tongyi Qianwen's 0.8023, indicating that ChatGLM-6b has higher generalization ability in category recognition, second only to Wenxin Yiyan.

[0139] In the "inclusion" relationship extraction experiment, the F1 value of ChatGLM-6b is 0.9248, which is slightly improved in both precision and recall compared to other models (such as GPT-3.5-Turbo's 0.9122 and Wenxinyiyan's 0.9091), showing its robustness in inclusion relationship extraction.

[0140] In the "batch division" relationship extraction, ChatGLM-6b performed particularly well in this task, with an F1 value of 0.9756, indicating that ChatGLM-6b can better cope with multi-level and multi-layered complex relationship extraction tasks. In the "level division" ChatGLM-6b's F1 value is 0.9333, indicating that it also maintains a high level in the relationship extraction task between project names and levels, which is better than GPT-3.5-Turbo's 0.8333 and Wenxin Yiyan's 0.7391, indicating that ChatGLM-6b has a strong ability to capture hierarchical level relationship extraction.

[0141] ChatGLM-6b has an F1 value of 0.8772 in the "distributed in" relationship extraction, which is slightly better than GPT-3.5-Turbo's 0.75 and Wenxinyiyan's 0.7143. This shows that it has a strong ability to extract geographical area information. ChatGLM-6b has an F1 value of 0.8571 in the "inherited" relationship, which is still at a high level. This result shows that ChatGLM-6b has a good generalization ability in identifying inheritors and related inheritance information, and can accurately extract character names.

[0142] Fig. 9 The experimental results of the comparison of relation extraction models under different evaluation indicators are presented. The experimental results show that the ChatGLM-6b model based on fine-tuning and few-shot prompt learning can maintain stable high performance in multiple relation extraction tasks, especially in hierarchical information extraction tasks such as batch division and level division. This is because the model can well capture complex contextual information and accurately identify the relationship between entities in different relation extraction tasks. In particular, when dealing with intangible cultural heritage projects with regional characteristics, it can accurately identify a variety of complex relationships.

[0143] Ablation experiment and result analysis

[0144] In order to verify the effect of the LoRA fine-tuning technology, Prompt template, and few-sample learning strategy modules proposed in this study on improving the model relationship extraction performance, four groups of ablation experiments were designed, and each module was gradually added to evaluate its impact on the final result.

[0145] Experiment 1 only uses the original ChatGLM-6b model for relation extraction without adding any fine-tuning or prompt information. It serves as a baseline experiment and aims to evaluate the basic performance of the original model in the intangible cultural heritage relation extraction task.

[0146] Experiment 2 adds LoRA fine-tuning technology to the baseline model without adding other prompt information. This step is used to evaluate the impact of LoRA fine-tuning technology on the model to understand its adaptability in the field of intangible cultural heritage texts.

[0147] Experiment 3 introduces role information prompts based on the model fine-tuned by LoRA, but does not provide task prompts. This experiment is used to test the model's basic understanding of the task with role prompt information and its impact on the accuracy of relation extraction.

[0148] Experiment 4 further adds a prompt template based on fine-tuning and role information prompts to clarify the format requirements of the relationship extraction task. By providing the model with 1-shot, 2-shot, and 3-shot examples, it demonstrates the correct relationship extraction method to help the model better understand complex relationship extraction tasks. This experiment is used to test the effect of the few-shot learning strategy on improving the performance of the model under complex tasks.

[0149] By gradually adding LoRA fine-tuning, Prompt templates, and few-shot learning strategies, we can systematically evaluate the specific contribution of each module in the relation extraction task and verify the rationality and effectiveness of the model design. The ablation experiment results are shown in Table 6.

[0150] Table 6

[0151]

[0152] The ablation experiment results in Table 6 show that the gradual introduction of LoRA fine-tuning technology, role prompt templates and Prompt prompt templates has significantly improved the performance of the ChatGLM-6b model in the relationship extraction task in the intangible cultural heritage field.

[0153] First, when only the original ChatGLM-6b model is used, the F1 values ​​of most relationships are concentrated around 0.6, especially for the "inherited" relationship, where the F1 value is only 0.5333. This type of relationship usually involves specific cultural expressions and language features, such as the semantic nuances of the "master-apprentice" or "inheritor" relationship. The basic model is weak in generalization to specific tasks and has difficulty in effectively distinguishing these types of relationships.

[0154] After the introduction of LoRA fine-tuning, the F1 values ​​of all relation types have been improved, especially those of "batch division" and "distributed in" (reaching 0.7205 and 0.7234 respectively), reflecting that a small amount of parameter adjustment can enhance the model's recognition of domain-specific semantics. For example, the improvement of the "distributed in" relation shows that the model is better at extracting specific geographical distribution information after LoRA fine-tuning.

[0155] After adding the role hint template on the basis of LoRA fine-tuning, the F1 value of most relations exceeded 0.8, and the F1 value of the "distributed in" relation reached 0.8889. The role hint template provides clear contextual information in the input, helping the model to understand the role relationship between entities more accurately.

[0156] Finally, after the introduction of the Prompt prompt template, the F1 values ​​of all relationship types were significantly improved, especially the "attribute" and "contains" relationships, which reached 0.9302 and 0.93 respectively. The Prompt template ensures that the model outputs structured results and reduces errors caused by inconsistent formats; the few-shot learning strategy further improves the model's generalization ability on specific relationships through example guidance. Fig.10 The effect of adding different modules on the efficiency of relation extraction of different categories is demonstrated.

[0157] Embodiment 2

[0158] The present invention discloses a non-legacy text relationship extraction system, comprising:

[0159] The first building module is used to construct the intangible cultural heritage dataset;

[0160] The second building module is used to build a large language model;

[0161] The training acquisition module is used to train the large language model based on the few-sample learning strategy and use the intangible cultural heritage dataset to obtain the trained large language model.

[0162] An acquisition module is used to obtain the intangible cultural heritage text to be extracted;

[0163] A filling module is used to fill the intangible cultural heritage text to be extracted into the Prompt template;

[0164] The extraction module is used to input the Prompt template filled with the intangible cultural heritage text to be extracted into the large language model to obtain the semantic relationship between the entities in the intangible cultural heritage text to be extracted.

[0165] The division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application may be integrated into a processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0166] Embodiment 3

[0167] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the intangible cultural heritage text relationship extraction method are implemented. The memory may include a memory, such as a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk memory, etc. The processor, the network interface, and the memory are interconnected through an internal bus, which may be an industrial standard architecture bus, a peripheral component interconnection standard bus, an extended industrial standard structure bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory is used to store programs. Specifically, the program may include a program code, and the program code includes computer operation instructions. The memory may include a memory and a non-volatile memory, and provide instructions and data to the processor.

[0168] Embodiment 4

[0169] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the intangible cultural heritage text relationship extraction method are implemented. Specifically, the computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. The volatile memory may include random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include read-only memory (ROM), hard disk, flash memory, optical disk, magnetic disk, etc.

[0170] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0171] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0172] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0173] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0174] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for extracting relations from intangible cultural heritage texts, characterized in that: include: Get the intangible cultural heritage text to be extracted; Fill the intangible cultural heritage text to be extracted into the Prompt template; The Prompt template filled with the intangible cultural heritage text to be extracted is input into the large language model to obtain the semantic relationship between the entities in the intangible cultural heritage text to be extracted.

2. The method for extracting intangible cultural heritage text relations according to claim 1, characterized in that: Before inputting the Prompt template filled with the intangible cultural heritage text to be extracted into the large language model, it also includes: Constructing an intangible cultural heritage dataset; Build large language models; Based on the few-sample learning strategy, the large language model is trained using the intangible cultural heritage dataset to obtain the trained large language model.

3. The method for extracting intangible cultural heritage text relations according to claim 2, characterized in that: The process of constructing the intangible cultural heritage dataset is as follows: Acquire text data in the form of a web crawler, and perform noise data removal, non-text data deletion, and format conversion on the text data to form an initial intangible cultural heritage text data set; Sorting out the intangible cultural heritage entity relationships of the initial intangible cultural heritage text data set; According to the combing results, the data in the initial intangible cultural heritage text data set is labeled to obtain an intangible cultural heritage data set.

4. The method for extracting intangible cultural heritage text relations according to claim 3, characterized in that: The process of labeling the data in the initial intangible cultural heritage text data set is: Set intangible cultural heritage entity tags; Set the intangible cultural heritage entity relationship label; The data in the initial intangible cultural heritage text dataset are labeled according to the set intangible cultural heritage entity labels and intangible cultural heritage entity relationship labels.

5. The method for extracting intangible cultural heritage text relations according to claim 2, characterized in that: The process of building a large language model is as follows: Fine-tune LoRA on large language models; Design a Prompt template for a large language model.

6. The method for extracting intangible cultural heritage text relations according to claim 5, characterized in that: The Prompt template of the large language model is:

7. A non-legacy text relationship extraction system, characterized in that: include: An acquisition module is used to obtain the intangible cultural heritage text to be extracted; A filling module is used to fill the intangible cultural heritage text to be extracted into the Prompt template; The extraction module is used to input the Prompt template filled with the intangible cultural heritage text to be extracted into the large language model to obtain the semantic relationship between the entities in the intangible cultural heritage text to be extracted.

8. The intangible cultural heritage text relationship extraction system according to claim 7, characterized in that: Before inputting the Prompt template filled with the intangible cultural heritage text to be extracted into the large language model, it also includes: The first building module is used to construct the intangible cultural heritage dataset; The second building module is used to build a large language model; The training acquisition module is used to train the large language model based on the few-sample learning strategy and use the intangible cultural heritage dataset to obtain the trained large language model.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the intangible cultural heritage text relationship extraction method as described in any one of claims 1-6 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the intangible cultural heritage text relationship extraction method as described in any one of claims 1 to 6 are implemented.

Images