Method for extracting professional domain entity pair information based on same-category unstructured documents

Abstract

The invention provides a method for extracting professional domain entity pair information based on same-category unstructured documents. Comprising the following steps: counting expression characteristics of each professional domain entity pair to be extracted in a document, determining extraction rules of texts and information, and summarizing the rules in a template; dividing lines, watermarks, directories or invalid pages, page headers and page footers in front of a preamble page in the document are removed, and text content is extracted according to a text range and comprises text content and table content; a filtering type three-layer combination extraction model is constructed to achieve entity information extraction, the three-layer combination extraction model relates to rule-based extraction, table-based extraction, large model question and answer-based extraction schemes and sequential combination thereof, result accuracy is judged through accuracy verification, and the accuracy is fed back to a scheme layer for strategy optimization. According to the method, the problems that the generalization ability of the existing rule-based information extraction is insufficient, and machine learning and deep learning methods are lack of labeling data in professional fields can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to parameter text and information extraction in a professional field. Background Art

[0002] With the increasing development of natural language processing technology, enterprises increasingly need to use various information extraction technologies to analyze daily accumulated professional text data to mine the important value. However, various unstructured document data cannot be directly analyzed and therefore cannot be fully utilized. Therefore, text structuring technology is very important for text mining, and automated information extraction is one of the important means of text structuring technology. With the vigorous development of artificial intelligence technology, rule-based extraction, machine learning and deep learning methods are widely used in automated information extraction in professional fields.

[0003] When applying rule-based extraction, machine learning methods and deep learning methods to extract information from professional domain entities, the inventors found that the methods have at least the following disadvantages: 1. Traditional machine learning methods, such as Hidden Markov Model (HMM) and Conditional Random Field (CRF), often require a large amount of labeled data, while deep learning methods, such as LSTM networks, require even more massive amounts of labeled data. In the field of information extraction, especially automated information extraction in specific professional fields, there is almost no high-quality public data. Manually annotating entity data can only find terms with a high probability of appearing in professional fields, and the workload is large and the conversion efficiency is low. At the same time, the amount of training data required is often difficult to evaluate. Different entities may require different amounts of training samples due to different contexts.

[0004] 2. The current rule-based extraction has insufficient generalization capabilities. Although regular expressions can use wildcards and other forms to match text patterns with a certain degree of freedom, rule-based extraction can only match a few patterns summarized by experts. Pattern rules with similar semantics but different expression characters cannot be matched. Summary of the invention

[0005] The purpose of the present invention is to provide a method for extracting professional domain entity pair information in response to the above-mentioned problems in extracting professional domain entity pair information, so as to solve the problems that the existing professional domain entity extraction methods have low accuracy and completeness in extracting professional knowledge and require a large amount of artificial training corpus to train the model, as well as the problem that the existing rule-based professional domain entity information extraction has low generalization ability.

[0006] The technical solution adopted by the present invention is: A method for extracting professional domain entity pair information based on unstructured documents of the same category, characterized by comprising the following steps: S1, summarize the rules of text information extraction; S2, document text preprocessing; S3, entity pair information extraction; S4. Verification of accuracy by professional entities; S5. Information feedback and strategy tuning.

[0007] Furthermore, in step S1, rules for extracting text information are constructed for text objects in professional fields, and the rules are summarized in a template. Specifically: (1) Count the expression features of each professional domain entity concept to be extracted in the document Based on the experience of business experts, the parameter description features and value range features of the entity pairs to be extracted are summarized. Parameter description features refer to the different descriptions of the same term in different documents. Value range features refer to the representation form or concrete instance of professional term entities. For the same parameter, the value range features will have different concrete features in different documents.

[0008] (2) Determine the extraction scheme of the text to be extracted based on statistical features Count the concept expression mode and value expression mode of each professional field entity in the specific document, count the frequency and regularity of the concept expression keywords in different documents, and count the location of the professional field entity value, whether it is located in the table or in the text paragraph.

[0009] (3) Determine the rules for extracting text and information and summarize the rules in a template Extract core keywords from the concept expression pattern, and give priority matching rights to high-frequency core keywords according to the frequency; classify and construct value matching patterns from the value expression pattern. Clarify the extraction position of each professional field entity pair, which may be in the text paragraph, in the table, or may prioritize the text paragraph and then the table, or prioritize the table and then the text paragraph. Associate the professional field entity pair keyword pattern and professional field entity value pattern with the corresponding position type respectively, and assign the matching order according to the frequency order.

[0010] Furthermore, step S2 includes: (1) Eliminate invalid information Most of the information of professional entity pairs exists in the body of the document. Therefore, the extraction of entity pairs in the body of the document must first eliminate invalid information. Invalid information generally includes watermarks, headers and footers, prefaces or directory pages and previous content, dividing lines, etc. At the same time, in order to avoid the influence of unnecessary graphics or lines, cutting the document page into the text area is also conducive to the extraction of professional entity information.

[0011] (2) Organize page text and table content by paging Use document processing tools to extract page text content and table content, and clarify the correspondence between page text content, table content and page numbers.

[0012] Furthermore, step S3 includes: Combined with the positional regularity of entity pairs in professional fields, an extraction strategy is constructed from the extraction schemes for each entity pair in professional fields. The extraction schemes mainly involve rule-based extraction schemes, table-based extraction schemes, large model question-answering extraction schemes, and orderly filtering combination schemes among the three schemes. The extraction strategy mainly determines the trade-offs and priority determination of the three schemes.

[0013] (1) Rule-based extraction scheme It mainly relies on the pattern matching function of regular expressions to extract the target value according to the template information such as the expression structure, keywords, keyword order, etc. specified in step S1.

[0014] (2) Table-based extraction solution The table extraction-based solution must first traverse all tables in each page content, confirm the row coordinates of the professional field entity concept in the table based on the professional field entity expression keywords, and then extract the cell content containing the entity value in the row based on the professional field entity value expression mode.

[0015] (3) Extraction solution based on large model question answering Based on the large model question and answer extraction, we need to first prepare the question template for each professional entity, and improve the accuracy of the large model answer by adding original text extraction instructions in the prompt words, giving examples, etc. The text content needs to extract all possible relevant information from the document based on the keywords of the professional field entity description and merge them, so as to reduce noise interference and improve the answer speed.

[0016] Further, step S4 includes: The accuracy verification of entities mainly verifies the accuracy of entity values ​​in professional fields. The accuracy of entity values ​​mainly involves text length verification, numerical text value range verification, specific character verification, and expression mode verification.

[0017] The text length check mainly checks whether the character length of the extracted value exceeds the maximum allowed character length threshold; the text value domain check mainly checks whether all numbers in the text fall within the maximum and minimum value range; the specific character check mainly checks whether specific characters appear; the expression mode check mainly checks whether the professional field entity value complies with the specific expression mode; the original text check mainly checks whether the professional field entity value exists in the original text; the customized check mainly checks the special expression mode of the presentation of the professional field entity value.

[0018] Specifically, each entity in each professional field does not need to conduct accuracy verification of the entire amount mentioned above. It is necessary to select part of it for verification based on its own characteristics. Once the verification is passed, the extraction is deemed successful.

[0019] Furthermore, step S5 includes: (1) Expand or limit keywords and adjust the order of keywords The keywords with high frequency and high hit rate need to be moved up in order, and new keywords that were not previously mentioned but appeared as the document sample range increased need to be added. If the original keywords failed to match because the pattern matching rules were too broad, the rule patterns need to be limited. Keywords with too low hit rates need to be eliminated.

[0020] (2) Optimization of table extraction method By optimizing the selection of document processing tools, adjusting table extraction configuration parameters, eliminating invalid information, and taking into account existing table extraction results, the accuracy of table-based extraction of professional field entity pairs can be improved.

[0021] (3) Optimization of large-model question-answering solutions Questions and answers based on big models can be improved through big model comparison, adjustment of model parameters such as TopP and Temperature, and optimization of prompt words to improve the accuracy of answers.

[0022] The beneficial effects of the present invention are: 1. Provide more universal standards for the writing of industry specifications.

[0023] A database can be built to view the common description methods and value ranges of entity concepts in professional fields in numerous documents, thus forming professional knowledge in a statistical sense.

[0024] 2. Provide a faster and more efficient implementation path for entities in professional fields to extract information.

[0025] The accuracy rate can reach over 95%, and the response time is within 1 minute, which greatly improves the quality and efficiency of entity pair extraction in professional fields.

[0026] 3. It solves the problem of low generalization ability of rule-based extraction methods and the problem of low accuracy of deep learning-based extraction methods. It can achieve higher accuracy and a certain degree of automation, reducing the participation of experts. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of the steps of a method for identifying professional domain entity pairs in one embodiment of the present application; Figure 2 A rule-based extraction flow chart in one embodiment of the present application; Figure 3 A table-based extraction flow chart in one embodiment of the present application; Figure 4 This is a flowchart of extraction based on large model question answering in one embodiment of the present application; Figure 5 A schematic diagram of a filtering hierarchical extraction model in one embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] In order to facilitate understanding of the event extraction method provided by the embodiment of the present invention, the following first describes the application scenario of the event extraction method provided by the embodiment of the present invention, and the event extraction method is applied to extract events from a corpus text. The following is a detailed description of the extraction method in conjunction with the accompanying drawings.

[0030] refer to Figure 1 , the event extraction method provided by the embodiment of the present invention includes: Step S1, construct rules for text information extraction for text objects in professional fields, and summarize the rules in a template. According to the experience of business experts, summarize the parameter description features and value range features of the entity pairs to be extracted. Parameter description features refer to the fact that different documents may have different descriptions of the same term. For example, in the wind turbine manual in the offshore wind power field, the parameter "hub height" may be expressed as "hub height", "hub center height", etc. Value range features refer to the expression form or concrete instance of professional term entities. For the same parameter, the value range feature will have different concrete features in different documents, but generally cannot be separated from the expression form of text or numerical value, or any combination of the two. For example, the value of "hub height" in the manual document of a certain wind turbine model may be "110 meters", and in other documents it may be "123 meters", or it may be expressed as ">=90 meters", or it may be expressed as "115m / 120m / can be customized according to the project".

[0031] Therefore, for parameter information with obvious structural features, relevant patterns should be constructed according to its expression in the text, and corresponding entity pair matching rules should be configured, so that subsequent information extraction can be further performed through pattern matching.

[0032] Step S2: document text preprocessing.

[0033] Professional domain entity pair information generally exists in the body of the document. Therefore, a large amount of invalid information can be excluded by removing the dividing lines, watermarks, invalid pages before the table of contents or preface page, headers and footers, and extracting the body content according to the text range. Among them, watermarks and dividing lines can be identified by document parsing tools as page elements such as straight lines, curves, rectangles, and squares on each page, and can be removed by judging the attribute information of the page elements. At the same time, the text range can be obtained by filtering out the rectangular elements containing text content in the page and merging all rectangular ranges containing text content.

[0034] Then, with the help of document parsing tools, the text content of each page can be extracted page by page, and the text content of each page can be mapped with the page number. The text content of each page is traversed to locate the page number of the preface or directory page, and the page content before the preface or directory page is discarded.

[0035] If the document contains a header, the header generally appears after the front page or the table of contents. Compare the text content of two consecutive pages after the front page or the table of contents, and use the longest common substring algorithm to find the longest common substring of the text content of the two consecutive pages. If the length of the longest common substring exceeds a specific length, it is considered that the document contains a header. Continue to use the document parsing tool to identify non-empty words that appear in the longest common substring on the page, merge the maximum outer frame range of all words, and get the four-dimensional range of the header.

[0036] If the document contains a footer, similarly, by inverting the text content string, continue to use the longest common substring algorithm to find the longest common substring of the text content of two consecutive pages. If the length of the longest common substring exceeds a certain length, it is considered that the document contains a footer. Then use a similar method to get the four-dimensional range of the footer.

[0037] Based on the above page text range, continue to deduct the header and footer range to get the text content of the body, and do a good job of page number mapping for the next step of rule-based extraction. At the same time, using the table parsing tool and reasonably adjusting the table extraction configuration parameters, you can extract the table content represented by a two-dimensional array from the document, and also do a good job of page number mapping for the next step of table-based extraction.

[0038] Step S3: entity pair information extraction.

[0039] In order to maximize the hit rate of extracting professional domain entity pairs from the original text, the present invention constructs a filtering three-layer extraction model, that is, according to the expression characteristics of each entity pair, orderly combined extraction is performed from the three schemes of rule-based extraction, table-based extraction, and large model question and answer extraction. After any scheme is successfully extracted, the extraction ends and the result is returned. For fixed character text entity pairs, only rule-based extraction is required to complete the extraction. For some entity pair information recorded in the table, the table-based extraction method is applicable. Due to the instability of large model questions and answers, it is likely to answer non-original content or give wrong answers. Therefore, the large model-based question and answer scheme is generally performed after the above-mentioned rule-based extraction and table-based extraction fail. On the other hand, non-general domain entities and complex logical entities such as "fan model" and "project name" are given priority based on large model question and answer extraction.

[0040] The entity pairs in real documents are expressed in various forms. In similar documents, the same professional entity pair concept may appear in text paragraphs as well as in tables. By counting the frequency of professional entity pairs appearing in text paragraphs and in tables, we can find that different professional entity pairs have different frequency characteristics. Some professional entity pairs tend to appear in tables and occasionally have descriptions in text paragraphs, while some professional entity pairs tend to appear in text paragraphs and occasionally exist in tables. Therefore, for entity pair information with high table frequency, table-based extraction is prioritized. If table-based extraction fails, rule-based extraction is performed. For entity pair information with high text paragraph frequency, rule-based extraction is prioritized. If rule-based extraction fails, table-based extraction is performed. After both table-based extraction and rule-based extraction fail, the large model-based question and answer extraction module is entered.

[0041] The solution based on rule extraction mainly relies on the pattern matching function of regular expressions to extract the target value according to the template information such as the expression structure, keywords, keyword order, etc. specified in step 1.

[0042] Based on the table extraction solution, the professional domain entity values ​​are first divided into text type, numeric type, and text-numeric composite type. For professional domain entity values ​​containing text, it is necessary to determine whether the professional domain entity values ​​to be extracted are semantically distinguishable. Semantically distinguishable entities do not need to be limited by entity descriptions and can be directly extracted according to value extraction rules. For numeric types and partial text-numeric composite types, due to the lack of obvious semantic distinction, the prefix of the professional domain entity description must be added for extraction. The extraction process that requires semantic qualification is: traverse all tables in each page content, confirm the row coordinates of the professional domain entity concept in the table based on the professional domain entity expression keywords, and finally extract the cell content containing the entity value in the row according to the professional domain entity value expression mode. Professional domain entity value extraction that does not require semantic qualification only needs to traverse the content of each cell in the table and search for cell content that meets a specific pattern.

[0043] Based on the large model question and answer extraction, we need to first prepare the question template for each professional entity, and improve the accuracy of the large model answer by adding original text extraction instructions in the prompt words, giving examples, etc. The text content needs to extract all possible relevant information from the document based on the keywords of the professional field entity description and merge them, so as to reduce noise interference and improve the answer speed.

[0044] Step S4: The entity verifies the accuracy.

[0045] Since the entity concept part of the professional field entity pair is clear, the professional field entity value varies due to different forms of expression and concrete instances. Therefore, the accuracy verification of the entity pair mainly verifies the accuracy of the professional field entity value. The accuracy of the entity value mainly involves text length verification, numerical text value domain verification, specific character verification, expression mode verification, original text verification, and customized verification.

[0046] Text length check: Generally, the text length of professional entity values ​​of the same category will not exceed a certain threshold. For example, the text length of the value of the gearbox type generally does not exceed 20 characters. If it exceeds this threshold, the match is considered to have failed.

[0047] Numeric text value range check: For numeric or text numeric professional field entity pairs, professional entity values ​​generally have corresponding maximum and minimum value ranges. Therefore, the value range check of numeric text can extract continuous numbers in the entity value through regular expressions, convert continuous numeric characters into numbers, and determine whether the value range falls within the threshold range to implement value range check.

[0048] Specific character check: Specific characters must appear, otherwise the extracted information is considered invalid. For example, the blade material must contain any one of "glass", "carbon fiber", "fiber", "carbon glass", "GFRP", "CFRP".

[0049] Expression mode verification: The display form of the entity value in the professional field must conform to a certain expression mode, otherwise it is considered as extraction failure. For example, the value of "Generator Protection Level" must conform to the expression mode of [specific character IP (0 or 1 times) + a digit from 0 to 6 + a digit from 0 to 8]. Therefore, IP54 can be considered valid, while LP54 is considered invalid.

[0050] Original text verification: Original text verification is mainly used to verify the results of the large model's answer. Since the large model has problems such as non-compliance with instructions and large model hallucinations, the answer of the large model needs to be verified. It is also necessary to further extract the professional field entity value content based on the judgment words or separating punctuation marks, and verify whether the professional field entity value content appears in the original text. If it does not appear, it is considered that the large model extraction failed.

[0051] Customized verification: In addition to the above verification methods, if it is found that the entity value of a certain professional field contains some special rules, such as the value of "unit survival temperature" may be a combination of Chinese and numbers, and Chinese must appear after the numbers and exist between Chinese punctuation marks such as parentheses, then customized verification can be added in time according to the actual situation to improve the extraction accuracy.

[0052] Step S5: information feedback and model tuning.

[0053] In the process of continuously expanding the scope of document samples, since the statistical frequency of keywords expressing entities in professional fields may change, the keyword order of the rule-based extraction scheme needs to be adjusted according to the extraction effect. Keywords with high frequency and high hit rate need to be advanced in order. Conversely, keywords with low frequency and low hit rate need to be matched later or even eliminated.

[0054] Table-based extraction may fail to extract a table even though the document actually contains a table, or the document processing tool may make mistakes in identifying the original text, which may result in a lower extraction hit rate. Therefore, it is necessary to optimize the selection of document processing tools, adjust table extraction configuration parameters, remove invalid information, and take into account existing table extraction results, thereby increasing the table extraction success rate of document processing tools and improving the accuracy of table-based extraction of professional entity pairs.

[0055] Questions and answers based on large models can be optimized through large model comparison, adjustment of model parameters such as TopP and Temperature, and prompt words to improve the accuracy of answers. Temperature is a hyperparameter used to control the diversity of output. When the temperature is high, more candidate results may be included, while when the temperature is low, only very likely results will be selected. TopP can control the diversity of generated text. A smaller P value allows more words to be selected, thereby increasing the diversity of the output; a larger P value reduces the number of selected words, making the output more concentrated. Since the extraction of entity pairs in professional fields is a task that hopes that the extracted values ​​are as consistent as possible with the original text, it is necessary to increase TopP to concentrate the output results and lower Temperature to improve the accuracy of the output results.

[0056] The above embodiment is only a preferred technical solution of the present invention. Those skilled in the art should understand that the technical solutions or parameters in the embodiments can be modified or replaced without departing from the principle and essence of the present invention, and all should be covered within the protection scope of the present invention.

Claims

1. A method for extracting professional domain entity pair information based on unstructured documents of the same category, characterized in that: The following steps are involved: S1, summarize the rules of text information extraction; S2, document text preprocessing; S3, entity pair information extraction; S4. Verification of accuracy by professional entities; S5. Information feedback and strategy tuning.

2. The method according to claim 1, characterized in that The S1 includes: (1) Count the expression features of each entity concept in the professional field to be extracted in the document; (2) determining an extraction scheme for the text to be extracted based on statistical features; (3) Determine the rules for extracting text and information and summarize the rules in a template.

3. The method according to claim 1, characterized in that The S2 specifically includes: (1) Eliminate invalid information; Organize page text and table content into pages.

4. The method according to claim 1, characterized in that: The S3 extraction scheme specifically includes: A solution based on rule extraction, a solution based on table extraction, a solution based on large model question and answer extraction, and an ordered filtering combination of the three solutions.

5. The method according to claim 1, characterized in that The accuracy verification of S4 involves text length verification, numerical text value range verification, specific character verification, expression mode verification, original text verification, and customized verification.

6. The method according to claim 1, characterized in that The strategy tuning of S5 involves rule pattern adjustment, table extraction configuration parameter optimization, and large model prompt word optimization.

Images