A knowledge entity in a knowledge graph of a document set retrieval method

Abstract

The application discloses a kind of knowledge graph knowledge entity in the retrieval method of text set, using clustering model to classify all text sets, and by multi-layer clustering, text set is labeled from top to bottom according to hierarchy.Through text set is preprocessed;Using DocT5Sentence model, text set is decomposed, and the mapping relationship data of Sentence to text set is generated;Using base Transformer model, the data obtained by training T5, the result after last MLP processing is subjected to Softmax, and the text set label with the best similarity is obtained;Finally, the original power text set is tracked to the location by reverse marking, and the retrieval task of knowledge entity is completed.The application can quickly locate the power text set related to the knowledge entity object to be collected by constructing a multi-modal text set retrieval model.

Description

Technical Field

[0001] This invention belongs to the field of electric power and relates to a method for retrieving a collection of documents containing knowledge entities in a knowledge graph. Background Technology

[0002] With the country's vigorous promotion of paperless office practices, a large number of offline tasks have shifted online. Traditional paper documents are now stored as electronic files, photos, videos, and other formats. This allows for better sourcing of data, reducing reliance on traditional databases and enabling direct data retrieval from its origin. Therefore, the ability to quickly and purposefully search all types of documents is particularly essential.

[0003] Knowledge graphs not only possess the data storage, operation, and management functions of traditional databases, but also build concept entity relationships based on graph databases, providing relation search, semantic parsing, and graph-related deep learning. Knowledge acquisition is the foundation of knowledge graph construction. Industry-specific knowledge graphs have an even greater need for acquiring industry knowledge. The power industry is a highly specialized industry. Whether it's the basic functions of power generation, transmission, transformation, and storage, or the standards for power engineering construction, categories of power materials, and management requirements for power facilities, all exhibit strong industry characteristics. Summary of the Invention

[0004] The purpose of this invention is to provide a method for retrieving the document collection containing knowledge entities in a knowledge graph. By constructing a multimodal document collection retrieval model, it is possible to quickly locate the power-related document collections containing the knowledge entity objects that need to be collected.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A method for retrieving a document containing a knowledge entity in a knowledge graph, characterized by comprising the following steps:

[0007] S1. Prepare documents from various fields of the power industry, including but not limited to contract documents, bidding documents, technical specifications, maintenance work orders, financial reports, and a power industry corpus, and compile them into a collection (Corpus).

[0008] For all collections<Corpus,URL> Mapping, and then for all<Corpus,URL> Number them;

[0009] S2. Preprocess the document. Depending on the document type, use OCR, pdf2txt, and word2txt tools to parse it into a txt file, and then reduce its size by removing all whitespace, marking line breaks as sentence ends, and removing all non-Unicode characters.

[0010] S3. Using the Corpus2Sentence algorithm, generate corresponding Sentence experimental data from the file, and add the Sentence's Segment Position information from the original file to obtain... <Sentence,<CorpusID,SegmentID> Data in the form of >

[0011] S4. Construct the HuggingFaceTransformer model, using the pre-trained parameters of the T5 model from the previous step to initialize the Transformer's encoder and decoder. Set the learning rate of the encoder to 2 x 10^6. -4 The learning rate of the decoder is set to 10. -4 Training is performed to make the entire network memorize all Corpus and Sentence information;

[0012] S5. Process the knowledge entity Query in the knowledge graph to be searched into text, so that it contains only Unicode characters. Use this text as input to the model. The model calculates the Cosine Similarity between the Query and the Sentences already memorized in the model. Use the calculated similarity result as a new weight and substitute it into the next layer for calculation. Finally, obtain the Softmax result of the knowledge entity Query for the file Sentence.

[0013] S6. Using the obtained CorpusID, locate the specific file step by step from the classification results in S1 according to the hierarchical order.<Corpus,URL> It returns the corresponding URL value; based on the obtained SegmentID, it locates the specific segment number of the file where the Sentence result is located.

[0014] In step S1, the labeling method uses the HCA (Hierarchy Clustering Algorithm), as detailed below:

[0015] Set the hyperparameter of the K-Means algorithm to 10, i.e., K=10, and the threshold for the number of files in each class to 20, i.e., C=20;

[0016] 1) Perform K-Means analysis on all documents, and label the classification results with a range of [1, 10].

[0017] 2) If the number of files in a certain category in the classification results is greater than the threshold, then the K-Means algorithm is used to cluster this category again, and the classification results are re-labeled using the numbers [1,10].

[0018] 3) Repeat steps 1 and 2 until the number of files in all categories is less than the threshold or the classification results no longer change. At this point, concatenate the labels from each classification to form a tag string containing the classification results of each layer, i.e., the CorpusID corresponding to the file.

[0019] In step S3, the Corpus2Sentence algorithm is as follows:

[0020] Each term is set to a length of 32, and is determined by the terminator: comma, semicolon, period, question mark, or exclamation mark.

[0021] 1) First, take the first term of the document as a sentence;

[0022] 2) Randomly select 20 terms of length 32 from the file as sentences;

[0023] 3) Rebuild the T5 model of DocT5Sentence, transfer the model parameters that have been trained on the large dataset to the T5 model for use, and use the random sampling method to generate sentences.

[0024] 4) Save all Sentence information and the CorpusID information corresponding to each Sentence, and record the comparison information between the current Sentence and its Segment position in the original document, i.e. <Sentence,<CorpusID,SegmentID> Data in the form of >; and record the model parameters of the current T5 model.

[0025] This invention uses a clustering model to classify all document collections and then uses multi-level clustering to label the collections hierarchically from top to bottom. This is similar to the Inverted Index technique used in traditional word segmentation search. The desired document collections are preprocessed using various techniques; the DocT5Sentence model is used to decompose the collections, generating mapping data from sentences to documents; the data obtained from training the T5 model using a base Transformer model is then subjected to Softmax on the final MLP-processed results to obtain the document labels with the highest similarity; finally, reverse labeling is used to trace the location of the original power document collection, completing the task of retrieving knowledge entities.

[0026] This invention constructs a multimodal document retrieval model, which can quickly locate power-related documents related to the knowledge entity objects that need to be collected. Attached Figure Description

[0027] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0028] A method for retrieving the text collection containing knowledge entities in a knowledge graph. Figure 1 This is a flowchart of the present invention, which includes the following steps:

[0029] S1. Prepare documents from various fields of the power industry, including but not limited to contract documents, bidding documents, technical specifications, maintenance work orders, financial reports, and a power industry corpus, and compile them into a collection (Corpus).

[0030] For all collections<Corpus,URL> Mapping, and then for all<Corpus,URL> Number them.

[0031] S2. Preprocess the document. Depending on the document type, use OCR, pdf2txt, and word2txt tools to parse it into a txt file, and then reduce its size by removing all whitespace, marking line breaks as sentence ends, and removing all non-Unicode characters.

[0032] S3. Using the Corpus2Sentence algorithm, generate corresponding Sentence experimental data from the file, and add the Sentence's Segment Position information from the original file to obtain... <Sentence,<CorpusID,SegmentID> Data in the form of >

[0033] S4. Construct the HuggingFaceTransformer model, using the pre-trained parameters of the T5 model from the previous step to initialize the Transformer's encoder and decoder. Set the learning rate of the encoder to 2 x 10^6. -4 The learning rate of the decoder is set to 10. -4 Training is performed to make the entire network memorize all Corpus and Sentence information;

[0034] S5. Process the knowledge entity Query in the knowledge graph to be searched into text, so that it contains only Unicode characters. Use this text as input to the model. The model calculates the Cosine Similarity between the Query and the Sentences already memorized in the model. Use the calculated similarity result as a new weight and substitute it into the next layer for calculation. Finally, obtain the Softmax result of the knowledge entity Query for the file Sentence.

[0035] S6. Using the obtained CorpusID, locate the specific file step by step from the classification results in S1 according to the hierarchical order.<Corpus,URL> It returns the corresponding URL value; based on the obtained SegmentID, it locates the specific segment number of the file where the Sentence result is located.

[0036] Specifically as follows:

[0037] 1) Prepare a collection of documents from various sectors of the power industry, including but not limited to contract documents, bidding documents, technical specifications, maintenance work orders, and financial reports. Preprocess the documents by using tools such as OCR, pdf2txt, and word2txt to parse them into TXT files, and then reduce their size by removing all whitespace, commas, semicolons, periods, question marks, exclamation marks, and line breaks as sentence endings. Remove all non-Unicode characters.

[0038] 2) Conduct a review of all collections of texts.<Corpus,URL> Mapping, and then for all<Corpus,URL> Labeling is performed. The labeling method uses the HCA (Hierarchy Clustering Algorithm):

[0039] HCA algorithm

[0040] Set the hyperparameter of the K-Means algorithm to 10, i.e., K=10, and the threshold for the number of files per class to 20, i.e., C=20.

[0041] 1. Perform K-Means analysis on all documents, and label the classification results with a range of [1, 10].

[0042] 2. If the number of files in a certain category in the classification results exceeds the threshold, then the K-Means algorithm is used to cluster this category again, and the classification results are re-labeled using the number [1,10], denoted as .

[0043] 3. Repeat steps 1 and 2 until the number of files in all categories is less than the threshold or the classification results no longer change. At this point, concatenate the annotations from each classification to form a tag string containing the classification results of each layer, i.e., the CorpusID corresponding to the file.

[0044] For example, if K-Means is performed 3 times, and the file is labeled as class 6 in the first clustering, class 3 in the second clustering, and class 3 in the last clustering, then its CorpusID is 633.

[0045] 3) Use the Corpus2Sentence algorithm to break down the file into individual sentences.

[0046] Corpus2Sentence algorithm

[0047] Note: After extensive comparison and summarization, we found that the length of a sentence in a file generally does not exceed 32 Chinese characters. In our algorithm, we set the length of each term to 32 characters (using commas, semicolons, periods, question marks, and exclamation marks as terminators).

[0048] 1. First, treat the first term of the document as a sentence;

[0049] 2. Randomly select 20 terms of length 32 from the file as sentences;

[0050] 3. Then, build a T5 model for DocT5Sentence, transfer the model parameters that are isomorphic to our model and have been trained on a large dataset to our model for use, and use random sampling to generate sentences.

[0051] 4. Save all Sentence information and the CorpusID information corresponding to each Sentence, and record the comparison information between the current Sentence and its Segment position in the original document, i.e. <Sentence,<CorpusID,SegmentID> Data in the form of >. And record the model parameters of the current T5 model.

[0052] 4) Construct the HuggingFaceTransformer model, using the training parameters of the T5 model from the previous step to initialize the Transformer's encoder and decoder. Set the learning rate of the encoder to 2 x 10^6. -4 The learning rate of the decoder is set to 10. -4 The consistency-based regularization strategy employs the rdropouts method with a dropout ratio of 0.1. Data from S3 is sampled and used for training, allowing the entire network to memorize information from all Corpus and Sentence.

[0053] 5) The knowledge entity query from the knowledge graph to be searched is processed into text, containing only Unicode characters. It is then converted into a vector representation and used as input to the Transformer model. The model calculates the cosine similarity between this vector representation and the sentences already stored in the model. The calculated similarity result is used as a new weight and substituted into the next layer for further calculations. After passing through a Linear Layer and Softmax, the final Softmax result of the knowledge entity query on the file sentences is obtained. The output result set is controlled by adjusting the threshold parameter of Softmax.

[0054] 6) Using the obtained CorpusID, locate the specific file step by step from the classification results in S1 according to the hierarchical order.<Corpus,URL> Returns the corresponding URL value. Based on the obtained SegmentID, defines the specific segment number of the file where the resulting Sentence is located.

[0055] In this invention, the relevant technical terms are explained as follows: 1. Corpus: In this paper, it refers to text files, other files or photos that can be converted into text files, and corpora with industry characteristics. 2. Query: In this paper, it refers to sentences or phrases related to the knowledge entity to be queried. 3. Sentence: In this paper, Sentence is used to represent sentences in the original corpus to distinguish queries related to knowledge entities. 4. Segment: In this paper, Segment represents the location information of the Corpus where the Sentence is located. SegmentID is the encoded representation of the location information. 5. MLP: Multilayer Perceptron. 6. T5: It is an abbreviation for Transfer Text-to-Text Transformer. It is a classic model in the field of NLP for handling text-to-text tasks. 7. Knowledge Graph: A branch of AI technology that describes the connections between entities in the objective world in the form of a graph through a structured semantic knowledge base.

[0056] In this invention, the knowledge data source for the knowledge graph can be either structured or unstructured data. This invention describes a method for retrieving power-related document sets (Corpus) containing knowledge entities from a knowledge graph. It consists of two main parts, structurally similar to a twin-tower model, but unlike a twin-tower model, they are not parallel. The first part primarily uses the HCA algorithm to classify and encode Corpus files (CorpusID); and uses the Corpus2Sentence algorithm to convert Corpus into specific Sentences, while appending its SegmentPosition information from the Corpus to the Sentence, generating training data for the next step. The second part constructs a Transformer model that matches the similarity between the Query and the Sentence. The data prepared in the first part is fed into the model for training, allowing the model to memorize all CorpusIDs and Sentence information. Similarity is determined by cosine similarity, and after passing through an MLP, a Softmax algorithm is used to obtain the similarity score, outputting the corresponding CorpusID. Then, by parsing the CorpusID, the target Corpus is located, completing the task of retrieving the knowledge entity.

[0057] This invention uses a clustering model to classify all document collections and then uses multi-level clustering to label the collections hierarchically from top to bottom. This is similar to the Inverted Index technique used in traditional word segmentation search. The desired document collections are preprocessed using various techniques; the DocT5Sentence model is used to decompose the collections, generating mapping data from sentences to documents; the data obtained from training the T5 model using a base Transformer model is then subjected to Softmax on the final MLP-processed results to obtain the document labels with the highest similarity; finally, reverse labeling is used to trace the location of the original power document collection, completing the task of retrieving knowledge entities.

Claims

1. A method for retrieving a text set in which a knowledge entity of a knowledge graph is located, characterized in that, Comprising the following steps: S1, prepare the documents of various fields of the power industry, including contract documents, tender documents, technical specifications, repair work orders, financial reports and power industry corpus, to form a corpus; Map all corpus to <Corpus, URL>, and then label all <Corpus, URL>; S2, preprocess the document, according to the type of document, respectively using OCR, pdf2txt, word2txt tool to parse into txt file, and carry out slimming, delete all document blank, newline symbol mark as Sentence end symbol, remove all non-Unicode characters; S3, using Corpus2Sentence algorithm, the file is produced into corresponding Sentence experimental data, and Sentence in the original file SegmentPosition information is added, <Sentence, <CorpusID, SegmentID>> form data is obtained; In step S3, Corpus2Sentence algorithm is as follows: Set the length of each term to 32, and take comma, semicolon, period, question mark and exclamation mark as end symbol judgment, 1) first, the first term of the file is taken as a Sentence; 2) randomly select 20 groups of terms with a length of 32 as Sentence in the file; 3) build DocT5Sentence T5 model, migrate the model parameters trained in the large data set to T5 model for use, and use random sampling method to produce Sentence; 4) save all Sentence information and Sentence corresponding CorpusID information, record the present Sentence and its Segment position in the original document, that is, <Sentence, <CorpusID, SegmentID>> form data; And record the model parameters of the current T5 model; S4, build HuggingFaceTransformer model, use the pre-training parameters of T5 model in S3 to initialize the encoder and decoder of Transformer, set the learning rate of model encoder to 2X10-4, and set the learning rate of decoder to 10-4, train the model to remember all Corpus and Sentence information; S5, text processing is carried out on the knowledge entity Query of the knowledge graph to be searched, so that it only contains Unicode characters, which is used as model input, and the Cosine Similarity between it and the Sentence already remembered in the model is calculated through model calculation; The similarity result obtained is used as a new weight and substituted into the next layer calculation; Finally, the Softmax result of knowledge entity Query to file Sentence is obtained; S6、Through the CorpusID obtained, from the classification result in S1, according to the hierarchical order, step by step to locate the specific file <Corpus, URL>, return the corresponding URL value; According to the obtained SegmentID, locate the specific paragraph number of the file where the result Sentence is located.

2. The knowledge graph knowledge entity search method according to claim 1, characterized in that: In step S1, the labeling method adopts the HCA algorithm, which is as follows: The hyperparameter of the K-Means algorithm is set to 10, that is, K=10, and the threshold value of the number of files contained in each class is 20, that is, C=20; Step 1): K-Means is performed on all documents, and the classification result is labeled, with the range being [1, 10]; Step 2): When the number of files contained in a certain class in the classification result is greater than the threshold value, the K-Means algorithm is used for clustering again, and the classification result is newly labeled using the label [1, 10]; Step 3): Repeat steps 1 and 2 until the number of files in all classes is less than the threshold value or the classification result no longer changes, at which time the labels of each classification are connected to form a label string containing the classification results of each layer, that is, the CorpusID corresponding to the file.

Images