A text attribute relation extraction method based on a small data set pre-trained model

Abstract

The application relates to the technical field of archive construction, and provides a text attribute relation extraction method based on a small data set pre-training model, a pre-training model based on an ERNIE structure is publicly fine-tuned to a relation extraction model in a business background through a public pre-training model UIE which can be started by fine-tuning only a small amount of labeled data, and on the basis of the model, Before_pipeline data processing before prediction and After_pipiline data processing after prediction are constructed in combination with business text features to improve result reliability, the problems of lacking a labeled data set, UIE violent sentence splitting leading to errors, and low extraction result accuracy of the UIE are solved, and the problem that an existing traditional relation extraction model depends on a large amount of high-quality data sets manually labeled is solved.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and more specifically to a model and method for extracting attribute relationships from unstructured text in the field of natural language processing. It can intelligently identify text containing multiple main entities and their multiple attributes. Based on a publicly available pre-trained model, it is fine-tuned and adds operations such as long text segmentation and result verification to improve the model's accuracy, and finally provides a complete text attribute relationship extraction service. Background Technology

[0002] In attribute relationship extraction tasks, intelligently, quickly, and effectively identifying text content and extracting specified effective information is the goal of every model, in order to realize the expectation of empowering related businesses with intelligent model services. However, when data of the same business type comes from different data sources, there are varying degrees of differences in expression, text length, and sentence complexity, making extraction more difficult.

[0003] Traditional rule-based relation extraction models use the appearance of specified keywords as a prefix of related entity types, and then determine the entity based on the rules of entity composition. For example, in the text describing the main entity and its attribute entities, the order in which the entities appear may vary, and the prefix keywords of the entities are not unique. For rule-based relation extraction models, it is difficult to exhaustively list all the rules. Moreover, if the relationship between entities is not explicit but implicit, the relation extraction task becomes even more difficult.

[0004] Therefore, we considered introducing deep learning intelligent models to solve such problems. However, due to the special nature of the business, the local area not only lacks structured data, making remote supervised learning impossible, but also lacks a large amount of labeled data. Traditional deep learning-based models require at least tens of thousands of labeled data to achieve a high accuracy rate, which requires a lot of manual and time costs. Therefore, we considered introducing the publicly available pre-trained model UIE (Universal Information Extraction), which can be fine-tuned and started with only a small amount of labeled data. Based on the model, we constructed Before_pictureline and After_pictureline data processing to improve the credibility of the results, thereby realizing the extraction of text attribute relationships. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a text attribute relationship extraction method based on a small dataset pre-trained model. By employing a pre-trained model UIE and combining it with the characteristics of business texts to construct pre-prediction data processing and post-prediction data processing, the reliability of the results is improved. This solves the problems of existing text attribute relationship extraction methods that require the collection of tens of thousands of data points and require a large amount of manual labor and time.

[0006] The specific technical solution of this invention is as follows:

[0007] A method for extracting text attribute relationships based on a pre-trained model on a small dataset, comprising the following steps:

[0008] S1: Collect and filter text data. First, collect a small amount of text data related to the business. The purpose is to select text data that contains a variety of individual information and has a variety of formats. Training data with too simple a format will result in low generalization ability of the model, and thus it will be unable to cope with different formats of text expression.

[0009] S2: Formulate annotation rules and classify the annotated data; formulate annotation rules to manually annotate a small amount of data in S1, and then process and split the annotated small amount of data to generate training set, validation set and test set data;

[0010] According to the UI E model structure, a single sample will be split into multiple samples based on the number of labeled entities. Each new sample carries only one label type and all the label information of that type in the sentence, which expands the dataset and makes the model learning task simpler and more comprehensive.

[0011] S3: The training set data is input into the UI E pre-trained model. The training set segmented in S2 is used as the input to the UI E pre-trained model, and the data is pre-processed by regular expression filtering to filter out the HTML format web page tag information that may appear in the original data.

[0012] S4: Determine the length of a single text. Design a long text processing module to address the problem of brute-force segmentation of long texts by the UI E model. Based on the maximum text length that the model can accept, use Chinese punctuation marks to reasonably segment the long text and generate a new sample list.

[0013] S5: Dictionary generation. Based on the segmentation results of the original samples in S4, generate a dictionary mapping the original samples to the new samples.

[0014] S6: Input data into the UI E model for extraction. Input the multiple data to be predicted processed in step S4 into the UI E model, start the extraction one by one, and perform legality verification on individual special data of the original extraction results of the model.

[0015] S7: Analyze the prediction results. By analyzing the prediction results in step S6, construct a positive association dictionary between entities and attribute entities, and parse the positive association dictionary to generate the final reverse dictionary with the main entity tag as the key and the attribute entity tag as the value. The result with the highest correct probability for each attribute entity is retained under the corresponding main entity.

[0016] S8: Filter the prediction results and combine them with the dictionary to extract the results. By filtering the processed prediction results and deleting invalid information, the extraction results are combined with the sample mapping dictionary in S5, and then the extraction results are output.

[0017] As a further improvement to this scheme, in step S2, the annotation rules include: one main entity as the attachment subject, eight basic information as attribute entities, used to attach relationships to the main entity, and two interference entities, which are only annotated to help the model learn, and are not attached to the attributes of the main entity.

[0018] Eight attribute relationship label types are designed to annotate the relationship between the main entity and the corresponding attribute entities.

[0019] As a further improvement to this solution, in step S3, a suitable outline is designed based on business requirements and the annotation rules in S2. After at least three rounds of iteration, the model with the highest F1 value is selected as the UI E basic model, and the data input is awaited.

[0020] As a further improvement to this scheme, in step S5, the keys in the new sample list mapping dictionary are the position indices of the original samples, and the values ​​in the dictionary are the position indices of the original samples in the new sample list.

[0021] As a further improvement to this scheme, in step S6, the legality verification refers to the legal length and legal composition of each special field. This function can help the model to verify the extraction results of special characters and increase the accuracy of the model.

[0022] As a further improvement to this scheme, in step S7, the method for constructing the forward affiliation dictionary is as follows: parse each main entity and all attribute entities, generate its own tag based on the index position of the entity and the entity string itself, the key of the forward affiliation dictionary is the tag of the attribute entity, and the value of the dictionary is all the tags of the main entities that have the attribute entity and the corresponding affiliation probability. After the forward affiliation dictionary is generated, it is necessary to sort them from high to low according to the correct probability of each key-value pair.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] 1. This invention trains an attribute relationship extraction model using a small amount of manually labeled data. First, it collects relevant business text data and performs diversified filtering on the text. Then, it uses labeled data to fine-tune the open-source pre-trained network model UIE based on the ERN IE structure, retaining the optimal model. Finally, based on the characteristics of the text data and the model performance results, it adds a series of data processing modules to form Before_pipeline and After_pipeline, increasing the reliability of the model results. This solves the problem that existing technologies require the collection of a large amount of data, as well as a lot of manpower and time, which increases the cost of text data extraction. Attached Figure Description

[0025] Figure 1 This is a diagram illustrating the overall framework of the relation extraction pre-training model of this invention.

[0026] Figure 2 This is an example of manual annotation in this invention;

[0027] Figure 3 This is a flowchart of the relation extraction and prediction process of the present invention;

[0028] Figure 4 This is a schematic diagram of the UI E model extraction principle of the present invention;

[0029] Figure 5 This is a flowchart of the long text segmentation process of this invention;

[0030] Figure 6 This is a flowchart of the entity legality verification process of this invention;

[0031] Figure 7 This is a flowchart of the attribute entity affiliation verification process of the present invention;

[0032] Figure 8 This is an example image of the text extraction results of this invention. Detailed Implementation

[0033] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0034] In this embodiment, as Figure 1-8As shown, a text attribute relation extraction method based on a small dataset pre-trained model is proposed. For unstructured text data, an attribute relation extraction model is trained using a small amount of manually labeled data. First, relevant business text data is collected and subjected to diverse filtering. Then, the labeled data is used to fine-tune the open-source pre-trained model UIE based on the ERNIE structure, retaining the optimal model. Finally, based on the characteristics of the text data and the model performance results, a series of data processing modules are added to form Before_pipeline and After_pipeline, increasing the reliability of the model results. Before_pipeline includes a regular expression filtering and long text processing module before prediction, while After_pipeline includes a module for validating the legality of attribute entities and a module for verifying multiple affiliations of attribute entities. The specific steps of the text attribute relation extraction method are as follows:

[0035] Step 1: Collect and filter text data

[0036] First, based on the business scenario, a small amount of text data (over 150 pieces) was manually selected and labeled. A relation extraction model based on the publicly available ERN IE structure, UIE, was then fine-tuned to suit the business context. The model with the highest F1 score across multiple iterations was selected as the final model. Next, based on model performance and text characteristics, Before_pictureline and After_pictureline data processing were designed to optimize model performance. The aim was to select text data containing diverse individual information and varied formats. Training data with overly uniform formats would result in low model generalization ability, making it unable to handle text expressions of different formats.

[0037] Step 2: Define annotation rules

[0038] By classifying the labeled data and formulating labeling rules, the small amount of data in step one was manually labeled, such as... Figure 2 As shown, the annotation rules include:

[0039] (1) The main entity is a subject to be attached to, the attribute entity consists of eight basic information fields, which are used to attach the relationship to the main entity, and there are two interference entities, which are only marked to help the model learn, and are not attached to the attributes of the main entity.

[0040] (2) Design eight attribute relationship label types for labeling the relationship between the main entity and the corresponding attribute entities;

[0041] Then, the small amount of labeled data is processed and split into training, validation, and test sets, and a certain number of negative samples are generated proportionally, such as... Figure 4 The working principle of the UI E model is demonstrated. According to the UI E model structure, a single sample is split into multiple samples based on the number of labeled entities. Each new sample carries only one label type and all the label information of that type in the sentence, which expands the dataset and makes the model learning task simpler and more comprehensive.

[0042] It should be noted that negative samples are a built-in feature of the UI E model; they are randomly generated to facilitate model learning.

[0043] Step 3: Input training set data into the UI E pre-trained model

[0044] The training set segmented in step two is used as the input to the UIE pre-trained model. A suitable schema is designed according to business needs and annotation rules. The schema is set within the annotation rules and includes main and sub-entities to help the model generate appropriate prompts to predict the data. After at least 3 rounds of iteration, the model with the highest F1 score is selected as the UIE base model and waits for the data input.

[0045] Meanwhile, before the data enters the model for extraction, the original data is first preprocessed with regular expression filtering to filter out any HTML-formatted web page tag information that may appear in the original data.

[0046] Step 4: Determine the length of a single text file

[0047] To address the issue of UIE models brute-force segmentation of long texts, a long text processing module is designed. Based on the maximum acceptable text length of the model, the module uses Chinese punctuation marks to reasonably segment long texts and generate a new sample list.

[0048] Specifically, such as Figure 5 As shown, if the text length exceeds 500 characters, the text is segmented using Chinese punctuation marks. Based on Chinese expression habits and the expression habits of business texts, the text is first segmented using Chinese periods ".". If the segmented subtext is still greater than 500 characters, it is then segmented using Chinese semicolons ";". If it is still greater than 500 characters, the original text is retained.

[0049] Step 5: Dictionary Generation

[0050] Based on the segmentation results of the original samples in step four, a corresponding original sample to new sample list mapping dictionary is generated. The key of the dictionary is the position index of the original sample, and the value of the dictionary is the position index of the original sample in the new sample list. For example, if the original sample 1 is segmented into 3 segments and the original sample 2 is segmented into 4 segments, then the mapping dictionary is {'1':[0,1,2],'2':[3,4,5,6]}.

[0051] Step Six: Input data into the UIE model for extraction.

[0052] By inputting the multiple data points to be predicted processed in step four above into the UIE model, the data is extracted one by one, and the validity of individual special data points in the original extraction results of the model is verified; the special data points refer to some fields with strong composition rules.

[0053] The legality verification process refers to the legal length and legal composition of each special field. This function can help the model verify the extraction results of special characters and increase the accuracy of the model.

[0054] Table 1

[0055] Dataset Domain Metric Comparable SOTA UIE ACE05-Rel News, Speech Relation Strict F1 PURE (Zhou and Chen) 65.60 66.06 CoNLL04 News Relation Strict F1 TSE (Wang and Lu) 73.70 75.00 SciERC Scinetific Relation Strict F1 PURE (Zhou and Chen) 35.60 36.53

[0056] Table 1 shows the performance of the UIE model compared to other models in relation extraction tasks. It can be seen that the UIE model has improved to varying degrees compared to other models on different datasets.

[0057] It should be noted that: under Dataset, the English term is publicly available dataset on the Internet; Domain refers to the domain of the dataset; Meta ic refers to the metric; and Comparable SOTA refers to the model name and F1 score of the model being compared.

[0058] Table 2

[0059]

[0060] Table 2 shows the performance of UIE initial models of different sizes on self-built datasets in various industries. In the table, 0-shot means that the model is started directly for prediction without fine-tuning without training data, and 5-shot means that the model is fine-tuned with 5 labeled data for each category. Experiments show that the UIE model can significantly improve its performance by fine-tuning with a small amount of vertical domain data. In this invention, UIe-base is selected as the initial model to balance model performance and model speed.

[0061] Step 7: Analyze the prediction results

[0062] By analyzing the prediction results in step 6, construct a positive association dictionary between entities and attribute entities. The dictionary construction method is as follows: parse each main entity and all attribute entities, generate tags based on the index position of the entity and the entity string itself, with the key being the tag of the attribute entity and the value being all main entity tags that have the attribute entity and the corresponding correct probability. After the positive association dictionary is generated, sort the association probability values ​​of each key-value pair from high to low.

[0063] like Figure 7 As shown, the forward-attached dictionary is parsed to generate the final reverse dictionary with the main entity tag as the key and the attribute entity tag as the value. The result with the highest probability of correctness for each attribute entity is retained under the corresponding main entity.

[0064] Step 8: Filter the prediction results and combine them with the dictionary to extract the results.

[0065] The processed prediction results are filtered, invalid information is removed, and the extraction results are merged according to the sample mapping dictionary in step five. The extraction results are then output. Figure 8 As shown, a sample extraction result adapted from real business data is given. The model successfully extracted the main entity and its respective attribute entities, and also made the correct legality judgment, without attaching incorrect attribute relationships to the main entity.

[0066] In summary, this study addresses the issues of insufficient labeled datasets, errors caused by brute-force sentence splitting in UIE, and low accuracy of UIE extraction results. Existing traditional relation extraction models rely on large batches of manually labeled, high-quality datasets. However, due to the specific nature of our business, large-scale manual data labeling is not feasible. Therefore, a pre-trained model with a small dataset, UIE, is used. In the Before_pipeline, to address the problem of erroneous extraction results caused by the model's brute-force segmentation of long texts, a module was designed to segment long texts into multiple clauses based on Chinese punctuation and then extract them separately. In the After_pipeline, to address the lack of strong rule validation for special fields in the UIE model, an entity validity verification module was designed. Furthermore, to address the issue of attribute entities being repeatedly attached to multiple main entities, a module was designed to construct a forward and reverse dictionary between main entities and attribute entities, aiming to retain attribute entities on the main entity with the highest attachment probability, thus forming an attribute entity attachment verification module. Finally, a complete attribute relationship extraction model for unstructured text data is constructed based on Before_pipeline + UIE + After_pipeline.

[0067] The embodiments of the present invention are given for the purposes of illustration and description, and are not intended to be exhaustive or to limit the invention to the forms disclosed. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features.

Claims

1. A method for extracting text attribute relationships based on a pre-trained model on a small dataset, characterized in that, The text attribute relation extraction method includes the following steps: S1: Collect and filter text data. First, collect a small amount of text data related to the business. The purpose is to select text data that contains a variety of individual information and has a variety of formats. Training data with too simple a format will result in low generalization ability of the model. S2: Formulate annotation rules and classify the annotated data; formulate annotation rules to manually annotate a small amount of data in S1, and then process and split the annotated small amount of data to generate training set, validation set and test set data; S3: The training set data is input into the UIE pre-trained model by using the training set segmented in S2 as the input to the UIE pre-trained model and performing regular expression filtering preprocessing on the data. S4: Determine the length of a single text. To address the problem of UIE model brute-force segmentation of long texts, a long text processing module is designed. Based on the maximum acceptable text length of the model, the long text is reasonably segmented using Chinese punctuation marks to generate a new sample list. S5: Dictionary generation. Based on the segmentation results of the original samples in S4, generate a dictionary mapping the original samples to the new samples. S6: Input data into the UIE model for extraction. Input the multiple data to be predicted processed in step S4 into the UIE model, start the extraction one by one, and perform legality verification on individual special data in the original extraction results of the model. S7: Analyze the prediction results. By analyzing the prediction results in step S6, construct a positive association dictionary between entities and attribute entities, and parse the positive association dictionary to generate the final reverse dictionary with the main entity tag as the key and the attribute entity tag as the value. The result with the highest correct probability for each attribute entity is retained under the corresponding main entity. In step S7, the method for constructing the forward affiliation dictionary is as follows: parse each main entity and all attribute entities, generate its own tag based on the index position of the entity and the entity string itself, the key of the forward affiliation dictionary is the tag of the attribute entity, and the value of the dictionary is all the tags of the main entities that have the attribute entity and the corresponding affiliation probability. After the forward affiliation dictionary is generated, it is necessary to sort them from high to low according to the correct probability of each key-value pair. S8: Filter the prediction results and combine them with the dictionary to extract the results. By filtering the processed prediction results and deleting invalid information, the extraction results are combined with the sample mapping dictionary in S5, and then the extraction results are output.

2. The text attribute relation extraction method based on a small dataset pre-trained model as described in claim 1, characterized in that: In step S2, the annotation rules include: one main entity as the attachment subject, eight basic information as attribute entities, used to attach relationships to the main entity, and two interference entities, which are only annotated to help the model learn, and are not attached to the main entity's attributes. Eight attribute relationship label types are designed for labeling the relationship between the main entity and the corresponding attribute entities.

3. The text attribute relation extraction method based on a small dataset pre-trained model as described in claim 1, characterized in that: In step S3, a suitable outline is designed based on actual business needs and the annotation rules in S2. After at least three rounds of iteration, the model with the highest F1 value is selected as the UIE basic model, and the data input is awaited.

4. The text attribute relation extraction method based on a small dataset pre-trained model as described in claim 1, characterized in that: In step S5, the keys in the new sample list mapping dictionary are the position indices of the original samples, and the values ​​in the dictionary are the position indices of the original samples in the new sample list.

5. The text attribute relation extraction method based on a small dataset pre-trained model as described in claim 1, characterized in that: In step S6, the validity verification refers to the valid length and valid composition of each special field. This validity verification function can help the model verify the extraction results of special characters and increase the accuracy of the model.

Images