Method for quickly retrieving key content of paper file

Abstract

The invention relates to a method for quickly retrieving key contents of a paper document, which comprises the following steps of: realizing electronic conversion of the paper document by utilizing an OCR (Optical Character Recognition) technology, and realizing word frequency statistics, word meaning analysis, document theme analysis and table theme analysis of the electronic document through an offline deployed pre-training model to obtain a keyword list and a theme list; according to the method, the key content and the key theme are combined, a keyword-theme-page number corresponding relation is generated, a two-dimensional table is finally generated, the key content serves as the column of the table, the key theme serves as the row of the table, the page number is recorded on the intersection point of the row and the column, and a reader can quickly position the key content of the article and the page number where the corresponding theme is located by checking the two-dimensional table. According to the method, a set of document key content lookup table uniformly generated by software is designed, the document key content lookup table formed by one page or multiple pages is newly added between the catalog and the text of the paper document, and a reader knows the key content and key theme of the document through the table, so that the document key content and the key theme can be obtained. And the interested contents and the corresponding themes are quickly positioned in a table look-up manner, so that the document content retrieval efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of document information retrieval technology, and in particular to a method for rapid retrieval of key content in paper documents. Background Technology

[0002] Currently, when writing articles, authors usually present the outline and main content of each paragraph to readers in the form of a table of contents, and inform readers of the location of charts in the form of a chart table of contents, and intuitively show readers the theme of the chart through the chart name.

[0003] However, in the current document content retrieval and location system, which primarily relies on document and figure directories, the directory entries only provide the main points of the article paragraphs, offering a strong generalization. This makes it difficult for readers, except for the document author, to intuitively understand the main content of the document from the directory, especially the location of certain key information. When conducting content retrieval, readers need to rely on their own experience to infer the content of the article paragraphs and the location of key information of interest based on the directory entries, and then manually flip through the pages to find the relevant paragraphs. Due to the differences in the outline structure of various types of articles, readers unfamiliar with a particular type of article often need to spend considerable time inferring and flipping through pages to find the key information of interest, gradually becoming familiar with the article structure and eventually finding one or more locations of information of interest, resulting in low efficiency in content retrieval and location. Summary of the Invention

[0004] The technical problem to be solved by this invention is to address the shortcomings of existing technologies by providing a method for rapid retrieval of key content in paper documents. A set of document key content lookup tables, uniformly generated by software, is designed. A document key content lookup table consisting of one or more pages is added between the table of contents and the main text of the paper document. Readers can use this table to understand the key content and key themes of the document, and quickly locate the content of interest and its corresponding themes by looking up the table, thereby improving the efficiency of document content retrieval.

[0005] The technical problem to be solved by this invention is achieved through the following technical solution. This invention is a method for rapid retrieval of key content in paper documents. This method utilizes OCR technology to digitize paper documents, and through an offline pre-trained model, performs word frequency statistics, semantic analysis, document topic parsing, and table topic parsing of electronic documents to obtain a list of keywords and a list of topics. It generates a "keyword-topic-page number" correspondence and finally generates a two-dimensional table, with key content as the columns and key topics as the rows. Page numbers are recorded at the intersections of rows and columns. Readers can quickly locate the key content of the article and the page number of its corresponding topic by viewing this two-dimensional table.

[0006] The technical problem to be solved by this invention can also be further achieved through the following technical solution: The specific steps of the method for quickly retrieving key content from paper documents described above are as follows: (1) Use OCR technology to recognize paper documents as electronic documents; (2) Use deep neural networks to perform word segmentation and word frequency statistics on electronic documents; (3) Perform semantic analysis to generate a keyword list; (4) Select the most suitable pre-trained model for subsequent topic analysis; (5) Use large model technology to analyze the themes of text paragraphs and table content, generate a list of text themes and record the page range in which each theme appears; (6) Generate a document key content lookup table and output a two-dimensional table with a list of keywords as columns and a list of topics as rows. The page number of the topic corresponding to the keyword is marked at the intersection of the row and column.

[0007] The technical problem to be solved by the present invention can also be further realized by the following technical solutions. For the above-mentioned method for rapid retrieval of key content of paper documents, in step (2), a deep neural network is used to perform word segmentation and word frequency statistics on electronic documents, splitting the content in the text into meaningful words or phrases, and counting the frequency of occurrence of various words, sorting them in order from high to low, to obtain a list of candidate keywords.

[0008] The technical problem to be solved by the present invention can also be further realized through the following technical solutions. For the above-mentioned method for rapid retrieval of key content of paper documents, in step (4), the pre-trained model is a model that has been pre-trained with a large amount of data and can be deployed offline for electronic document processing, including a model for word segmentation and word frequency statistics and a model for topic analysis.

[0009] The technical problem to be solved by the present invention can also be further realized through the following technical solutions. For the above-mentioned method for rapid retrieval of key content of paper documents, in step (5), the theme analysis of text paragraphs and table content is carried out using large model technology, the theme of each text or table is determined, a list of text themes is generated, and the page number range of each theme is recorded.

[0010] The technical problem to be solved by the present invention can also be further realized through the following technical solutions. For the above-mentioned method for rapid retrieval of key content of paper documents, the theme of each text or table determined in step (5) includes content summary, concept definition, quotation, terms and conditions, and technical requirements.

[0011] The technical problem to be solved by the present invention can also be further realized through the following technical solutions. For the above-mentioned method for rapid retrieval of key content of paper documents, in step (6), the inclusion of each word in the keyword list in the topic list is determined by using the large language model technology, the page number position of the keyword is recorded, and finally a document key content lookup table composed of "keyword-topic-page number" is generated.

[0012] The technical problem to be solved by the present invention can also be further realized through the following technical solutions. For the above-mentioned method for quick retrieval of key content of paper documents, the table finally generated in step (6) is a two-dimensional table with the keyword list as columns and the topic list as rows. The page number of the topic where the corresponding keyword is located is marked at the intersection of the row and column.

[0013] Compared with existing technologies, this invention utilizes an offline-deployed pre-trained model to achieve automated parsing of document content, which is more efficient than traditional manual reading and parsing. Furthermore, the offline deployment environment is suitable for document parsing tasks containing sensitive information, making it widely applicable. In addition, the accompanying page number table for key content and topics in paper documents provides readers with a method for quickly retrieving and locating the content of paper documents after they are generated and distributed, without the aid of external information processing equipment, thus improving the information retrieval efficiency of paper documents. Attached Figure Description

[0014] Figure 1 This is a diagram illustrating the process of generating a document key content lookup table for this invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Reference Figure 1 This invention presents a method for rapid retrieval of key content in paper documents. It provides readers with a way to quickly retrieve and locate the content of paper documents after they are generated and distributed, without the aid of external information processing equipment, thereby improving the efficiency of information retrieval for paper documents.

[0017] This method utilizes OCR technology to digitize paper documents. Through an offline pre-trained model, it performs word frequency statistics, semantic analysis, document topic parsing, and table topic parsing to obtain a list of keywords and topics. It generates a "keyword-topic-page number" correspondence and finally produces a two-dimensional table with key content as columns and key topics as rows. Page numbers are recorded at the intersections of rows and columns. Readers can quickly locate the key content of the article and the page number of its corresponding topic by viewing this two-dimensional table.

[0018] The specific steps of this method are as follows: Step 1: Use OCR technology to recognize the paper document into an electronic document and adjust its format, including font adjustment, deletion of redundant line breaks, and paragraph division proofreading, to obtain the electronic version of the paper document. For files that are already in electronic format, you can proceed directly to the next processing step. The second step is to use a deep neural network to perform word segmentation and word frequency statistics on the electronic document. This involves splitting the text into meaningful words or phrases, counting the frequency of each type of word, and sorting them from highest to lowest to obtain a list of candidate keywords. Step 3: Conduct semantic analysis. Taking advantage of the fact that most of the key content are nouns such as concept names, definitions, professional terms, and product names, filter out words that are nouns and have a low frequency of occurrence from the keyword candidate list obtained in the previous step, and use them as the keyword list. Step 4: Select the most suitable pre-trained model for subsequent topic analysis by setting the document type (such as technical solutions, tender documents, contract texts, etc.); Among them, pre-trained models are models that have been pre-trained with a large amount of data and can be deployed offline for electronic document processing, including: 1. Models used for word segmentation and word frequency statistics: BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained language model based on Transformers that can capture rich semantic information in text. In word segmentation, although it is not specifically designed for word segmentation, it can use its powerful language understanding capabilities to assist in word segmentation. It can also be used in semantic understanding scenarios related to word frequency statistics to help better determine the importance of words in text. BiLSTM-CRF (Bidirectional Long Short-Term Memory - Conditional Random Field) is a deep learning model that combines a bidirectional long short-term memory network (BiLSTM) and a conditional random field (CRF). It is widely used in sequence labeling tasks, such as named entity recognition (NER) and part-of-speech tagging (POS tagging).

[0019] 2. Models for topic analysis Non-negative matrix factorization (NMF) is a matrix factorization technique based on non-negativity constraints. It requires that both the input matrix and the elements of the factorized matrix be non-negative. This characteristic makes NMF perform exceptionally well when processing data with non-negative physical meaning, such as images and text. NMF represents data as a linear combination of basis matrices through additive combination, tending to generate sparse factorization results, thereby highlighting key features in the data.

[0020] Latent Semantic Analysis (LSA) is a method that uses matrix factorization to discover hidden semantic structures in text. It reduces the dimensionality of the original word-document matrix through singular value decomposition, transforming it into a latent semantic space. This space captures the potential relationships between words, effectively solving the problems of polysemy and polysemy. The latent semantic space constructed by LSA is usually orthogonal, which helps reduce redundant information in the data and improves computational efficiency.

[0021] Step 5: Use large model technology to analyze the themes of text paragraphs and tables, determine the theme of each text or table, such as content introduction, concept definition, quotation, terms and conditions, technical requirements, etc., generate a list of text themes and record the page range in which each theme appears; Step 6: Generate a key content lookup table for the document. Use large language model technology to determine the inclusion of each word in the keyword list in the topic list, record the page number position of the keyword, and finally generate a key content lookup table for the document consisting of "keyword-topic-page number". This table is a two-dimensional table with a list of keywords as columns and a list of topics as rows. The page number of the topic corresponding to the keyword is marked at the intersection of the row and column. Readers can quickly locate the page number of a topic containing a specific keyword by looking up a table, thus improving the efficiency of information retrieval from paper documents without the need for external information processing equipment.

[0022] This invention provides a method for rapidly retrieving key content from paper documents, which can be applied to various fields of paper documents. Specific application examples are as follows: I. Legal Document Retrieval 1. Scenario: When dealing with a large number of complex paper documents such as contracts and compilations of legal provisions in a law firm, lawyers need to quickly locate key clauses, definitions, liability divisions and other important content.

[0023] 2. Application process: First, OCR technology is used to recognize paper contracts or legal provisions into electronic documents and adjust the format to ensure the accuracy and readability of the text; Next, deep neural networks are used to perform word segmentation and word frequency statistics on electronic documents, breaking the text down into meaningful words or phrases, counting the frequency of word occurrences, and generating a list of candidate keywords. Then, semantic analysis is performed to select words that are nouns and whose frequency is within a certain range as a keyword list, such as "liability for breach of contract", "amount of compensation", and "effective date". Select a suitable pre-trained model based on the type of legal document for topic analysis, use large model technology to analyze the topic of text paragraphs and table content, determine the topic of each text or table, such as "contract terms", "definition and interpretation", "dispute resolution", etc., and record the page range in which each topic appears; Finally, a key content lookup table for the document is generated, with the keyword list as columns and the topic list as rows, and the page number of the topic corresponding to the keyword is marked at the intersection of the row and column.

[0024] By viewing this two-dimensional table, lawyers can quickly locate the page numbers of key clauses, improving their review efficiency.

[0025] 3. Effects: It greatly shortens the time lawyers spend searching for key information in legal documents, improves work efficiency, and reduces omissions and errors that may occur due to manual searching.

[0026] II. Research on Academic Literature 1. Scenario: When researchers are reviewing a large number of academic papers, they need to quickly understand the core viewpoints, research methods, experimental results and other key content of the papers.

[0027] 2. Application process: Using OCR technology to recognize paper academic documents into electronic documents and optimize their format; Deep neural networks are used for word segmentation and word frequency statistics to generate a candidate list of keywords. Then, semantic analysis is used to filter out the final list of keywords, such as "research methods", "experimental results", and "conclusions". Select an appropriate pre-trained model based on the subject area of ​​the academic literature, use large model technology to conduct topic analysis, determine the topic of each text, such as "Introduction", "Literature Review", "Experimental Section", "Conclusion and Discussion", etc., and record the page range; Generate a lookup table of key document content to help researchers quickly locate content of interest.

[0028] Researchers who want to understand the experimental methods in a particular paper can quickly find the page number containing the experimental section by consulting a lookup table and read the relevant content directly.

[0029] 3. Effects: It helps researchers quickly grasp the core content of academic literature, improves the efficiency of literature reading and research, and promotes academic exchange and innovation.

[0030] III. Enterprise Technical Document Management 1. Scenario: During product research and development and production, companies generate a large number of technical documents, such as product manuals, operation manuals, and design drawings. Technical personnel need to quickly find key technical parameters, operating procedures, troubleshooting methods, and other information.

[0031] 2. Application process: Use OCR technology to convert paper technical documents into electronic documents and adjust their formatting; Deep neural networks are used for word segmentation and word frequency statistics to filter out a list of keywords, such as "technical parameters", "operation steps", "fault codes", and "solutions". Select a pre-trained model based on the type of technical document, use large model technology to perform topic analysis, determine the topic of each text or table, such as "product overview", "technical specifications", "operation guide", "troubleshooting", etc., and record the page range; Generate a lookup table for key document content, allowing technical personnel to quickly locate the page number containing the information they need by referring to the table.

[0032] When equipment malfunctions, technicians can quickly find the corresponding troubleshooting method on the page number in the lookup table based on the fault code, and resolve the problem promptly.

[0033] 3. Results: It improved the efficiency of technicians in finding technical information, reduced equipment downtime, and ensured the company's production and operation.

[0034] In summary, this method utilizes OCR and an offline pre-trained model to digitize and automatically parse paper documents, generating a two-dimensional lookup table containing a "keyword-topic-page number" correspondence. This allows for rapid location of key content, improves retrieval efficiency, and is applicable to documents containing sensitive information, making it widely applicable.

Claims

1. A method for rapid retrieval of key content in paper documents, characterized in that: This method utilizes OCR technology to digitize paper documents. Through an offline pre-trained model, it performs word frequency statistics, semantic analysis, document topic parsing, and table topic parsing to obtain a list of keywords and topics. It generates a "keyword-topic-page number" correspondence and finally produces a two-dimensional table with key content as columns and key topics as rows. Page numbers are recorded at the intersections of rows and columns. Readers can quickly locate the key content of the article and the page number of its corresponding topic by viewing this two-dimensional table.

2. The method for rapid retrieval of key content in paper documents according to claim 1, characterized in that: The specific steps of this method are as follows: (1) Use OCR technology to recognize paper documents as electronic documents; (2) Use deep neural networks to perform word segmentation and word frequency statistics on electronic documents; (3) Perform semantic analysis to generate a keyword list; (4) Select the most suitable pre-trained model for subsequent topic analysis; (5) Use large model technology to analyze the themes of text paragraphs and table content, generate a list of text themes and record the page range in which each theme appears; (6) Generate a document key content lookup table and output a two-dimensional table with a list of keywords as columns and a list of topics as rows. The page number of the topic corresponding to the keyword is marked at the intersection of the row and column.

3. The method for rapid retrieval of key content in paper documents according to claim 1, characterized in that: In step (2), a deep neural network is used to perform word segmentation and word frequency statistics on the electronic document. The content in the text is split into meaningful words or phrases, and the frequency of occurrence of each type of word is counted. The words are sorted in order from high to low to obtain a list of keyword candidates.

4. The method for rapid retrieval of key content in paper documents according to claim 1, characterized in that: In step (4), the pre-trained model is a model that has been pre-trained with a large amount of data and can be deployed offline for electronic document processing, including a model for word segmentation and word frequency statistics and a model for topic analysis.

5. The method for rapid retrieval of key content in paper documents according to claim 1, characterized in that: In step (5), the topic analysis of text paragraphs and table content is carried out using large model technology to determine the topic of each text or table, generate a list of text topics and record the page range in which each topic appears.

6. The method for rapid retrieval of key content in paper documents according to claim 5, characterized in that: The subject of each text or table determined in step (5) includes a brief description, definition of concepts, quotation, terms and conditions, and technical requirements.

7. The method for rapid retrieval of key content in paper documents according to claim 1, characterized in that: In step (6), the inclusion of each word in the keyword list in the topic list is determined using large language model technology, the page number position of the keyword is recorded, and finally a document key content lookup table consisting of "keyword-topic-page number" is generated.

8. The method for rapid retrieval of key content in paper documents according to claim 7, characterized in that: Step (6) The final generated table is a two-dimensional table with the keyword list as columns and the topic list as rows. The page number of the topic corresponding to the keyword is marked at the intersection of the row and column.

Images