Large model assisted PDF text extraction and rearrangement optimization method and system

Abstract

The invention relates to the technical field of document analysis, and discloses a large model assisted PDF text extraction and rearrangement optimization method and system.The method comprises the steps that the content of a PDF document is analyzed and extracted, each page of PDF is read and traversed, then resource pre-analysis operation and block object type judgment are executed on each page of the document, and the PDF text is extracted and rearranged. Firstly, CMAP of a text is obtained through resource pre-analysis, then a corresponding text extraction mode is matched according to the type of a block object, finally, the extracted text is subjected to basic segmentation, then segmentation rearrangement is conducted through an LLM model, and the extracted content of each page is output. According to the method, the PDF file is directly read, the blocks in the PDF file are directly operated, and the corresponding text extraction mode is matched through the types of the block objects, so that the text reading and recognition speed is remarkably improved compared with the mode that the PDF is directly analyzed by directly using a deep learning model.

Description

Technical Field

[0001] The present invention relates to the field of document parsing technology, and in particular to a large model-assisted PDF text extraction and rearrangement optimization method and system Background Art

[0002] PDF is currently the most popular document exchange and distribution solution, because there is no need to worry about how the underlying PDF is implemented, only the application layer needs to be focused on, and because PDF is cross-platform and can be displayed correctly on any platform, it is widely used as a document exchange and distribution solution.

[0003] As the application of large language models continues to develop, from simple text summarization and translation, to predicting stock performance based on sentiment and financial report topics, to complex retrieval (such as RAG), we need to first extract text data from the real world. There are many types of documents that share this unstructured information, from web articles and blog posts to handwritten letters and charts. However, a large part of this text data is stored and transmitted in PDF format.

[0004] Although PDF is easy to use, extraction is very complex. Because PDF is an unstructured document, unlike structured documents such as HTML, XML, and JSON, which are easy to read and extract, how to get useful value from PDF for analysis and utilization, such as how to correctly identify and extract tables in PDF, and how to identify images in PDF and extract text, etc., while ensuring that the order of the extracted text is consistent with the reading order of humans, and improving the accuracy and efficiency of extraction are the problems that need to be solved at present. Summary of the invention

[0005] In order to solve the problems and shortcomings existing in the above-mentioned prior art, the present invention specifically proposes a large model-assisted PDF text extraction and rearrangement optimization method and system. After receiving and reading the PDF file, the present invention directly operates on the blocks in the PDF file, and then matches the corresponding text recognition and extraction method according to the type of the block object. Compared with directly using a deep learning model to parse PDF, the present invention significantly improves the text reading and recognition speed.

[0006] In order to achieve the above-mentioned object of the invention, the technical solution of the present invention is as follows: On the one hand, the present invention proposes a large model-assisted PDF text extraction and rearrangement optimization method, which mainly includes the following steps: Read and traverse each PDF page, then perform resource pre-parsing operations on each page to obtain the CMAP of the text; then determine the type of PDF block object on each page, and then match the corresponding text extraction method according to the block object type to identify and extract the block object text; The extracted text is segmented, then rearranged using the LLM model, and finally the extracted content of each page is output; When the type of the block object is ordinary text, the text content is directly identified and extracted; When the type of the block object is glyph text, the corresponding CMAP is searched in the global CMAP dictionary and glyph mapping is performed; it is determined whether the mapping is successful, and if the mapping is successful, the text of the block object is directly recognized and extracted; if the mapping is unsuccessful, the entire block object is rendered as an image and the VL model is used to recognize the language type in the image, and then the OCR recognition technology is used to recognize and extract the text in the image; When the type of the block object is a picture, the VL model is first used to parse the picture to determine whether there is text in the picture and the language used by the text; for the text information contained in the picture, OCR recognition technology is used to identify and extract it, and for the image information in the picture, the LLM model is used to describe the picture to obtain the text description it contains.

[0007] Preferably, the resource pre-parsing operation specifically includes the following steps: First, traverse the resource dictionary of each page, find and enter the / Font node and the secondary node / ToUnicode in turn, and then perform the first regular match according to the following regular expression to obtain the content in begin-end; begin([\\w\\W]+)end; Then perform a second regular match on the content in begin-end. The regular match expression is as follows: cid->multiple unicode iterations; CID->Specify unicode iteration; Multiple unicodes correspond one to one; When any of the above regular matching expressions is met, the match is successful and the corresponding CMAP of the file is obtained.

[0008] Preferably, when the / Font node does not exist, the resource pre-parsing operation process ends.

[0009] Preferably, when there is no secondary node / ToUnicode, the / Ordering node is searched and matched with a built-in universal character set (UCS), and the CMAP is obtained according to the universal character set.

[0010] Preferably, when the / Ordering node does not exist, the resource pre-resolution operation process is terminated.

[0011] Preferably, the specific process of glyph mapping is as follows: obtain each glyph in the object block, look up the Unicode code corresponding to the glyph in the CMAP according to the glyph, and then convert the glyph into a standard Unicode character.

[0012] Preferably, the method for judging whether the mapping is successful is as follows: calculate the proportion of the currently converted characters in Unicode encoding. If more than 20% of the converted characters do not fall within the Unicode encoding range, the mapping is judged to be unsuccessful, otherwise it is successful.

[0013] Preferably, the LLM model re-arranges the text into segments and outputs the extracted content in markdown format.

[0014] Preferably, after the CMAP is acquired, it is stored in a global CMAP dictionary.

[0015] Based on the same inventive concept, on the other hand, the present invention also discloses a large model-assisted PDF text extraction and rearrangement optimization system, which is used to implement the above-mentioned PDF text extraction and rearrangement optimization method, including: Resource pre-parsing module, reads and traverses each page of PDF, and then performs resource pre-parsing operation on each page to obtain the CMAP of the text; Block object parsing module determines the type of each page PDF block object; The text recognition and extraction module matches the corresponding text recognition and extraction method according to the block object type to recognize and extract the block object text; The text segmentation and rearrangement generation module performs basic segmentation on the extracted text, then uses the LLM model to rearrange the segments, and finally outputs the extracted content of each page.

[0016] On the other hand, the present invention also proposes a computer device, including a memory, a processor, and a computer program stored in the memory and executable in the processor, wherein when the processor executes the computer program, the above-mentioned PDF text extraction and rearrangement optimization method is implemented.

[0017] On the other hand, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed in a computer processor, the above-mentioned PDF text extraction and rearrangement optimization method is implemented.

[0018] Beneficial effects of the present invention: 1. The present invention directly reads PDF files and directly operates blocks in PDF files, which significantly improves the text reading speed compared to using a deep learning model to parse PDF.

[0019] 2. The present invention introduces the LLM large model to segment and rearrange the extracted text, which significantly enhances the semantics of the extracted text paragraphs compared to direct storage.

[0020] 3. The present invention introduces the VL large model to perform text discrimination and language recognition on images, filters out useless data, solves the problem of having to determine the language and load the corresponding model before OCR recognition, and improves the coverage and accuracy of image text extraction.

[0021] 4. The present invention optimizes CMAP to match CMAPs of more formats and has a built-in default mapping table, covering more extraction scenarios and having a wider range of applications.

[0022] 5. The present invention makes a secondary judgment on the result of text extraction, solves some special PDFs, such as the problem of vector text extraction, and significantly improves the accuracy of text recognition and extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The foregoing and following detailed description of the present invention will become more apparent when read in conjunction with the following drawings, in which: Figure 1 is a flow chart of the method of the present invention; Figure 2 This is a diagram of the system composition of the present invention.

[0024] In the attached figure: 201. Resource pre-analysis module; 202. Block object analysis module; 203. Text extraction module; 204. Text segmentation rearrangement generation module. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions for achieving the purpose of the present invention will be further described below through several specific embodiments. It should be noted that the technical solutions claimed for protection in the present invention include but are not limited to the following embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the scope of protection of the present invention.

[0026] PDF: Portable Document Format. It is a file format introduced by Adobe Systems in 1993. It is used to represent documents, independent of application software, hardware and operating system. PDF files can contain text, images, fonts, interactive elements (such as forms, buttons, links) and multimedia content (such as video, audio). PDF is one of the most widely used file formats, especially suitable for document distribution and archiving.

[0027] Although PDF is easy to use, extraction is very complex. Because PDF is an unstructured document, unlike structured documents such as HTML, XML, and JSON, which are easy to read and extract, how to get useful value from PDF for analysis and utilization, such as how to correctly identify and extract tables in PDF, and how to identify images in PDF and extract text, etc., while ensuring that the order of the extracted text is consistent with the reading order of humans, and improving the accuracy and efficiency of extraction are the problems that need to be solved at present.

[0028] Based on this, an embodiment of the present invention proposes a large model-assisted PDF text extraction and rearrangement optimization method. After receiving and reading a PDF file, the present invention performs a resource pre-analysis operation on the PDF file to obtain its CMAP, and then determines the type of the PDF block object at the same time, and then directly operates the block object of the PDF file, matches the corresponding text recognition and extraction method based on the type of the block object, and obtains the text contained in the block object. Compared with directly using a deep learning model to parse PDF, the present invention significantly improves the speed of text reading and recognition.

[0029] As the most basic implementation method of the present invention, this embodiment first discloses a large model-assisted PDF text extraction and rearrangement optimization method. Figure 1 , the method mainly comprises the following steps: Step S1. The system receives a PDF file and starts to enter a processing mode, preparing to parse, identify and extract the content of the document.

[0030] Step S2. For the input PDF file, which usually contains multiple pages, first read and traverse each page of the PDF, and then perform a resource pre-parsing operation on each page to obtain the CMAP of the text.

[0031] It can be understood that the CMAP file is automatically generated when the PDF file is generated and can be hidden during the generation stage.

[0032] In the implementation mode described in the present invention, the specific process and steps of the resource pre-resolution operation are as follows: Step S201. First, traverse the resource dictionary of each page, find and enter the / Font node, and then go to step S202; if the file does not have a / Font node, directly end the resource pre-parsing operation process; Step S202. Find and enter the secondary node / ToUnicode, and then go to step S203; if the file does not have a secondary node / ToUnicode, the present invention has made corresponding optimization processing for this special case, and the UCS code table (universal character set) collected by the built-in channel is built into the code. When CMAP cannot be found in some PDF files, the built-in code table can be located through the description information, and the glyph can still be converted to Unicode. Therefore, when the file does not have a secondary node / ToUnicode, by finding the / Ordering node and matching the built-in universal character set (UCS), CMAP can also be obtained according to the universal character set.

[0033] Step S203. Perform the first regular match according to the following regular expression. If the match fails, directly end the resource pre-parsing operation process; if the match succeeds, obtain the content in the file begin-end and proceed to step S204; begin([\\w\\W]+)end; Step S204. Perform a second regular expression match on the content in begin-end. If the match fails, the resource pre-analysis operation process ends directly. If the match succeeds, the CMAP corresponding to the file is obtained, and then the process goes to step S3. The expression for the second regular expression match is as follows: cid->multiple unicode iterations; CID->Specify unicode iteration; Multiple unicodes correspond one to one; When the content in begin-end satisfies any of the above regular matching expressions, the match is successful and the corresponding CMAP of the file is obtained.

[0034] Step S3. Based on the input PDF file, traverse the content of each page of the file, then determine the block objects on each page, determine their types, and then match the corresponding text extraction method according to the block object type.

[0035] It is understandable that the types of PDF block objects can be divided into ordinary text, glyph text and image types. For different block objects, different text extraction methods are used to identify and extract the text content contained in them. The specific operation process is as follows: (1) When the type of the block object is determined to be ordinary text, the text content is directly identified and extracted.

[0036] It is understandable that ordinary text is usually text that exists in plain text in a PDF. Since it is plain text itself, the text content contained therein can be directly extracted without any additional processing.

[0037] (2) When the type of the block object is determined to be glyph text, the corresponding CMAP is searched in the global CMAP dictionary and glyph mapping is performed; it is determined whether the mapping is successful. If the mapping is successful, the text of the block object is directly identified and extracted; if the mapping is unsuccessful, the entire block object is rendered as an image and the VL model is used to identify the language type in the image, and then the OCR recognition technology is used to identify and extract the text in the image.

[0038] In the embodiment described in the present invention, for glyph mapping using CMAP, the specific operations are as follows: Get each glyph in the object block, then look up the Unicode code corresponding to the glyph in CMAP according to the glyph, and then convert the glyph into a standard Unicode character.

[0039] It can be understood that CMAP stores the mapping of glyphs to Unicode, and Unicode is text.

[0040] Furthermore, whether the glyph mapping is successful is specifically judged as follows: Calculate the proportion of the currently converted Unicode characters in the Unicode encoding; if more than 20% of the converted characters do not fall within the Unicode encoding range, the mapping is unsuccessful, otherwise the mapping is successful.

[0041] It is understandable that the VL model, or visual language model, is a large model that can process and understand image and text information. It can accept image and text input, understand the image content, and generate corresponding text descriptions or answer related questions based on the image.

[0042] (3) When it is determined that the type of the block object is a picture, the VL model is used to parse the picture and the prompt word template of the picture. By asking questions to the large model, it is possible to determine whether the current picture contains text. If there is text, the language information, description information and other contents in the text are further obtained; then the text in the picture is identified and extracted using OCR recognition technology; and for the image information in the picture, the present invention introduces the LLM model, and uses the LLM model for specific picture description, language recognition and classification to obtain the text description content it contains.

[0043] It is understandable that the LLM model, or large language model, is a model based on artificial intelligence and natural language processing (NLP) technology that can understand and generate natural language text. They learn the structure, semantics, and context of language by training on large amounts of text data in order to generate human-like text output in various tasks.

[0044] It is worth mentioning that after obtaining the file CMAP through the resource pre-parsing operation, it is usually stored in the global CMAP dictionary. During the subsequent block parsing, the corresponding CMAP will be matched and searched in the global CMAP dictionary based on the metadata. Each CMAP has a corresponding name, and its corresponding CMAP name can usually be seen under the / Font node, so the name can be used to match the corresponding CMAP in the global CMAP dictionary.

[0045] Therefore, it can be understood that for the resource pre-parsing operation of the subsequent pages of the PDF file, when the current CMAP name is known through the / Font node, it will check whether the CMAP already exists in the global CMAP dictionary. If it exists, the resource pre-parsing operation will be stopped immediately, otherwise the resource pre-parsing operation process will continue to be executed until the CMAP corresponding to the file is obtained. Therefore, based on the above operation process, the parsing and extraction speed of PDF files can be significantly improved to avoid repeated operations.

[0046] Step S4. After the text content is extracted, basic segmentation is first performed to keep the context semantics intact within the limited token range through the algorithm of paragraph text overlap after segmentation. Then the text blocks after basic segmentation are spliced ​​with the prompt word template, and the content is rearranged and segmented using the LLM model through the optimized limited output template in the prompt word, and finally the target formatted text is obtained and output, making it highly readable and easy for machine and human comprehension and reference reading.

[0047] It is understandable that LLM models usually output the extracted content in markdown format.

[0048] In the embodiment described in the present invention, the text generated by the LLM model re-segmentation has the original text extraction missing punctuation segmentation and semantic segmentation. Usually we output the limited template in the following format: Title 1 Title 2 text.

[0049] Based on the same inventive concept, the embodiments of the present invention also disclose a large model-assisted PDF text extraction and rearrangement optimization system. Since the principle of solving the problem by the device is similar to that of the PDF text extraction and rearrangement optimization method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable. Figure 2A schematic diagram of the structure of a large model-assisted PDF text extraction and rearrangement optimization system provided by an embodiment of the present invention is shown in FIG. Figure 2 As shown, the system may include: a resource pre-analysis module 201, a block object analysis module 202, a text recognition and extraction module 203, and a text segmentation rearrangement generation module 204. The structure of the device is described in detail below, wherein: The resource pre-parsing module 201 reads and traverses each page of the PDF, and then performs a resource pre-parsing operation on each page to obtain the CMAP of the text; The block object parsing module 202 determines the type of each page PDF block object; The text recognition and extraction module 203 matches the corresponding text recognition and extraction method according to the block object type to recognize and extract the block object text; The text segmentation and rearrangement generation module 204 performs basic segmentation on the extracted text, and then uses the LLM model to rearrange the segments, and finally outputs the extracted content of each page.

[0050] It should be noted that the systems, devices, models or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, in this specification, the above devices are described in various units according to their functions. Of course, when implementing the present invention, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0051] Furthermore, in the present specification, adjectives such as first and second may be used merely to distinguish one element or action from another, without necessarily or implying any actual such relationship or order.

[0052] Furthermore, on the other hand, this embodiment also provides a computer device, which includes a processor, an input device, an output device and a memory, and the processor, the input device, the output device and the memory are interconnected; wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the steps in the above-mentioned embodiment.

[0053] Furthermore, another aspect of this embodiment also provides a computer-readable storage medium, characterized in that: the computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the steps in the above embodiment.

[0054] In this embodiment, the processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0055] The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs and units, such as the corresponding program units in the above method embodiments of the present invention. The processor executes various functional applications of the processor and works data processing by running the non-transitory software programs, instructions and modules stored in the memory, that is, implementing the method in the above method embodiments.

[0056] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0057] The one or more units are stored in the memory, and when executed by the processor, the method in the above embodiment is performed.

[0058] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware.

[0059] The above description is only a preferred embodiment of the present invention and does not constitute any form of hindrance to the present invention. Any simple modification or equivalent changes made to the above embodiments based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A large model-assisted PDF text extraction and rearrangement optimization method, characterized in that: The method comprises the following steps: Read and traverse each page of PDF, then perform resource pre-parsing operations on each page to obtain the CMAP of the text; Determine the type of each PDF block object, then match the corresponding text extraction method according to the block object type to identify and extract the block object text; The extracted text is segmented, then the LLM model is used to rearrange the segments, and finally the extracted content is output; When the type of the block object is ordinary text, the text content is directly identified and extracted; When the type of the block object is glyph text, the corresponding CMAP is searched in the global CMAP dictionary and glyph mapping is performed; it is determined whether the mapping is successful, and if the mapping is successful, the text of the block object is directly recognized and extracted; if the mapping is unsuccessful, the entire block object is rendered as an image and the VL model is used to recognize the language type in the image, and then the OCR recognition technology is used to recognize and extract the text in the image; When the type of the block object is a picture, the VL model is used to parse the picture to determine whether there is text in the picture and the language used by the text; for the text information contained in the picture, OCR recognition technology is used to identify and extract it; for the image information in the picture, the LLM model is used to describe the picture to obtain the text description it contains.

2. A large model-assisted PDF text extraction and rearrangement optimization method according to claim 1, characterized in that: The resource pre-parsing operation specifically includes the following steps: Traverse the resource dictionary of each page, find and enter the / Font node and the secondary node / ToUnicode in turn, perform the first regular match according to the following regular expression, and obtain the content in begin-end; begin([\\w\\W]+)end; Perform a second regular match on the content in begin-end. The regular match expression is as follows: cid->multiple unicode iterations; CID->Specify unicode iteration; Multiple unicodes correspond one to one; When the content in begin-end satisfies any of the above regular matching expressions, the match is successful and the corresponding CMAP of the file is obtained.

3. The large model-assisted PDF text extraction and rearrangement optimization method according to claim 2 is characterized in that: When the / Font node does not exist, the resource pre-parsing operation process ends.

4. The large model-assisted PDF text extraction and rearrangement optimization method according to claim 2 is characterized in that: When the secondary node / ToUnicode does not exist, the / Ordering node is searched and matched with the built-in universal character set, and the CMAP is obtained according to the universal character set.

5. The large model-assisted PDF text extraction and rearrangement optimization method according to claim 4 is characterized in that: When the / Ordering node does not exist, the resource pre-resolution operation process ends.

6. The large model-assisted PDF text extraction and rearrangement optimization method according to claim 1 is characterized in that: The specific process of glyph mapping is as follows: obtain each glyph in the object block, look up the Unicode code corresponding to the glyph in the CMAP according to the glyph, and then convert the glyph into a standard Unicode character.

7. The large model-assisted PDF text extraction and rearrangement optimization method according to claim 1 is characterized in that: The method for judging whether the mapping is successful is as follows: Calculate the proportion of the currently converted characters in the Unicode encoding. If more than 20% of the converted characters do not fall within the Unicode encoding range, the mapping is judged to be unsuccessful, otherwise it is successful.

8. The large model-assisted PDF text extraction and rearrangement optimization method according to claim 1 is characterized in that: After obtaining the CMAP, store it in the global CMAP dictionary.

9. The large model-assisted PDF text extraction and rearrangement optimization method according to claim 1 is characterized in that: After the LLM model segments and rearranges the text, it outputs the extracted content in markdown format.

10. A large model-assisted PDF text extraction and rearrangement optimization system, the system is used to implement the PDF text extraction and rearrangement optimization method according to any one of claims 1 to 9, characterized in that: include: Resource pre-parsing module, reads and traverses each page of PDF, and then performs resource pre-parsing operation on each page to obtain the CMAP of the text; Block object parsing module determines the type of each page PDF block object; The text recognition and extraction module matches the corresponding text recognition and extraction method according to the block object type to recognize and extract the block object text; The text segmentation and rearrangement generation module performs basic segmentation on the extracted text, then uses the LLM model to rearrange the segments, and finally outputs the extracted content of each page.

Images