PDF file parsing method and apparatus, electronic device, and storage medium

Abstract

The present application provides a PDF file parsing method and apparatus, an electronic device, and a storage medium. The method comprises: parsing a PDF file by means of a PDF parsing tool to obtain an initial text; inputting the initial text and first task instruction information into a large language model to obtain an initial recognition result, wherein the first task instruction information comprises a chain-of-thought reasoning example, i.e., a first chain-of-thought reasoning example, of text and table recognition, a first task instruction, and a standard output format, and instructs the large language model to perform text and table recognition on the initial text according to the first chain-of-thought reasoning example, and to output an initial recognition result according to the standard output format; and generating a target text on the basis of the initial recognition result.

Description

PDF file parsing methods, devices, electronic equipment, and storage media Technical Field

[0001] This application relates to the field of document recognition technology, specifically to a PDF file parsing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Portable document format (PDF) is a cross-platform file format. PDF files maintain consistent format and layout across different operating systems and devices, making it widely used. Since many downstream tasks in the logistics industry are NLP / Language Model (LLM) tasks, requiring data input into LLM / Language Models for processing, PDF files cannot be directly recognized by these models. Therefore, PDF parsing algorithms are needed to convert the content of PDF files into recognizable TXT text. Thus, PDF parsing algorithms have become one of the key data processing technologies for solving downstream tasks in the logistics industry. The core task of PDF parsing algorithms is to establish a mapping from the original PDF file to the logical document structure, transforming binary data into a human-readable structure. Different PDF parsing algorithms have different functions, including recognizing and extracting text, images, and tables from PDFs, and converting them into TXT text through layout restoration.

[0003] Many open-source libraries and commercial products, such as the open-source libraries PDFBox and PyPDF2, as well as commercial products like Adobe Acrobat, provide PDF parsing services that can parse PDF files.

[0004] However, for more complex PDF files, existing PDF parsing services may perform poorly, failing to extract data accurately and efficiently. Summary of the Invention

[0005] In view of the defects and shortcomings of the existing technology, this application proposes a PDF file parsing method, electronic device and storage medium, which can further identify the parsed txt text based on the first chain of thought (COT) model to obtain an initial identification result, and determine and output the target identification result based on the initial identification result, thereby solving the problem of not being able to accurately and efficiently extract data from PDF files.

[0006] According to a first aspect of the embodiments of this application, a PDF file parsing method is provided, including:

[0007] The PDF file is parsed using a PDF parsing tool to obtain the initial text;

[0008] The initial text and the first task instruction information are input into the large language model to obtain the initial recognition result. The initial recognition result includes the text and table corresponding to the content in the initial text. The first task instruction information includes the first task instruction, the first thought chain reasoning example, and the standard output format. The first thought chain reasoning example is a thought chain reasoning example for text and table recognition. The first task instruction information is used to instruct the large language model to perform text and table recognition on the initial text according to the first thought chain reasoning example and output the initial recognition result according to the standard output format.

[0009] Based on the initial recognition results, the target text is generated.

[0010] According to a second aspect of the embodiments of this application, a PDF file parsing apparatus is provided, comprising:

[0011] The parsing module is used to parse PDF files using PDF parsing tools to obtain the initial text;

[0012] The recognition module is used to input the initial text and the first task instruction information into the large language model to obtain the initial recognition result. The initial recognition result includes the text and table corresponding to the content in the initial text. The first task instruction information includes the first task instruction, the first thought chain reasoning example, and the standard output format. The first thought chain reasoning example is a thought chain reasoning example for text and table recognition. The first task instruction information is used to instruct the large language model to perform text and table recognition on the initial text according to the first thought chain reasoning example and output the initial recognition result according to the standard output format.

[0013] The generation module is used to generate target text based on the initial recognition results.

[0014] According to a third aspect of the embodiments of this application, an electronic device is provided, including a memory and a processor;

[0015] The memory is connected to the processor and is used to store programs;

[0016] The processor is used to implement the PDF file parsing method as described in the first aspect by running a program in memory.

[0017] According to a fourth aspect of the embodiments of this application, a storage medium is provided, on which a computer program is stored, and when the computer program is run by a processor, it implements the PDF file parsing method as described in the first aspect.

[0018] In the aforementioned PDF file parsing method, electronic device, and storage medium, a PDF parsing tool can parse the PDF file to obtain initial text. This initial text, along with first task instruction information, is then input into a large language model to obtain initial recognition results. These initial recognition results include text and tables corresponding to the content in the initial text. The first task instruction information includes a thought chain reasoning example for text and table recognition (i.e., the first thought chain reasoning example), the first task instruction, and a standard output format. The first task instruction information instructs the large language model to perform text and table recognition on the initial text according to the first thought chain reasoning example and output the initial recognition results according to the standard output format. Then, based on the initial recognition results, the target text is generated. Thus, by further recognizing the parsed txt text using the first COT model on top of the PDF parsing tool's work, the content of the PDF file can be recognized more accurately and stably. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] Figure 1 is a flowchart illustrating a PDF file parsing method according to an embodiment of this application;

[0021] Figure 2 is a schematic diagram of a PDF file parsing process according to an embodiment of this application;

[0022] Figure 3 is a schematic diagram of a PDF file parsing device according to an embodiment of this application;

[0023] Figure 4 is a schematic diagram of the structure of an electronic device proposed in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] Overview

[0026] As described in the background section, many open-source libraries and commercial products, such as PDFBox and PyPDF2, as well as Adobe Acrobat, provide PDF parsing services capable of parsing PDF files. However, for more complex PDF files, the parsing performance of existing PDF parsing services may be poor, resulting in messy text content that cannot accurately and efficiently extract data from the PDF file.

[0027] Building upon this foundation, the inventors further discovered that by parsing PDF files using a PDF parsing tool to obtain initial text, and then inputting this initial text along with first task instruction information into a large language model, an initial recognition result is obtained. This initial recognition result includes text and tables corresponding to the content in the initial text. The first task instruction information includes a thought chain reasoning example for text and table recognition (i.e., the first thought chain reasoning example), the first task instruction, and a standard output format. The first task instruction information instructs the large language model to perform text and table recognition on the initial text according to the first thought chain reasoning example and output the initial recognition result according to the standard output format. Subsequently, based on the initial recognition result, the target text is generated. Thus, by further recognizing the parsed txt text using the first COT model on top of the PDF parsing tool's work, the content of the PDF file can be recognized more accurately and stably.

[0028] Based on the above concept, this specification provides a PDF file parsing method, which will be described exemplarily below with reference to the accompanying drawings.

[0029] Exemplary methods

[0030] Referring to Figure 1, in an exemplary embodiment, a PDF file parsing method is provided, applicable to any electronic device. As shown in Figure 1, the PDF file parsing method includes steps S101-S103:

[0031] S101: Parse the PDF file using a PDF parsing tool to obtain the initial text.

[0032] The PDF parsing tool can be an open-source library such as PDFBox or PyPDF2, or a commercial product such as Adobe Acrobat.

[0033] Depending on the parsing tool used, the format of the initial text obtained from parsing a PDF file may vary.

[0034] Generally, the initial text is in txt format.

[0035] Of course, depending on the actual working conditions, the initial text can be in other formats.

[0036] S102: Input the initial text and the first task instruction information into the large language model to obtain the initial recognition result.

[0037] The first task instruction information includes a first task instruction, a first thought chain reasoning example, and a standard output format. The first task instruction is an instruction to perform the task of text and table recognition, which is used to instruct the large language model to recognize the text and tables in the input text. The first thought chain reasoning example is a thought chain reasoning example for text and table recognition. The standard output format is the format of the large language model output data, i.e., the initial recognition result.

[0038] The first task instruction information is used to instruct the large language model to perform text and table recognition on the initial text according to the first thought chain reasoning example, and to output the initial recognition results in accordance with the standard output format.

[0039] The initial recognition results include text and tables corresponding to the content in the initial text.

[0040] Specifically, the first example of thought chain reasoning includes preset text and thought chain reasoning results. The thought chain reasoning results are the text and table obtained by recognizing the preset text.

[0041] Optionally, the first task instruction information also includes a PDF parsing tool, which is the PDF parsing tool used in step S101 to parse the PDF file.

[0042] At this time, the first task instruction information contains three parts: (1) PDF parsing tool and first task instruction, which is used to inform the large language model of the PDF parsing tool that reads the initial text and the specific task content; (2) First thought chain reasoning example, that is, an input text and its corresponding thought chain reasoning result, which is used to inform the large language model of the thought chain logic for text and table recognition of the initial text; (3) Output data format, which is used to inform the large language model of the standard output format.

[0043] For example, the prompt information for the first task is: (1) You (the large language model) are a PDF analyst. The user uses PyPDF2 to read the messy original text in the PDF file, that is, the initial text. You are only interested in the text and tables (in the messy original text); (2) The input text is {The student information table of Class 3, Grade 1 is as follows: Name Age Gender Score\nXiaoming 6 Male 89\nXiaohong 7 Female 98}, and the output is {"text": The student information table of Class 3, Grade 1 is as follows: {"table": Name|Age|Gender|Score\nXiaoming|6|Male|89\nXiaohong|7|Female|98}, where " / n" means newline; (3) Output data format: The table is in markdown (abbreviated as md).

[0044] At this point, based on the first task prompt information, the first task prompt information is input into the initial text language model, and the output result, i.e. the initial recognition result, is {"text": XXX, "table": XXXX}, where the text is in the format of txt and the table is in the format of md.

[0045] S103: Generate the target text based on the initial recognition results.

[0046] The text and table from the initial recognition results are merged to generate the target text.

[0047] After generating the target text, the target text can be output and displayed.

[0048] In this embodiment, a PDF file is parsed using a PDF parsing tool to obtain initial text. This initial text, along with first task instruction information, is input into a large language model to obtain initial recognition results, including text and tables corresponding to the content in the initial text. The first task instruction information includes a thought chain reasoning example for text and table recognition (i.e., the first thought chain reasoning example), the first task instruction, and a standard output format. The first task instruction information instructs the large language model to perform text and table recognition on the initial text according to the first thought chain reasoning example and output the initial recognition results according to the standard output format. Then, based on the initial recognition results, the target text is generated. Thus, by further recognizing the parsed text using the first COT model on top of the PDF parsing tool's work, the content of the PDF file can be recognized more accurately and stably.

[0049] There may be errors in the initial recognition results. In order to ensure the accuracy of the final target text, in some embodiments, when generating target text based on the initial recognition results, it is necessary to filter the content in the initial recognition results to obtain the target recognition results, and then generate the target text based on the target recognition results.

[0050] In other words, the target recognition result is the content of the initial recognition result after filtering.

[0051] The content in the initial recognition results can be filtered based on the degree of matching between the content in the initial recognition results and its corresponding content in the initial text. Before filtering the content in the initial recognition results, the degree of matching between the content in the initial recognition results and its corresponding content in the initial text must first be determined.

[0052] Since the content in the initial recognition results is generally quite messy, when determining the matching degree, the similarity between the meaning represented by the content in the initial recognition results and the meaning represented by the corresponding content in the initial text can be determined as the matching degree between the content in the initial recognition results and the corresponding content in the initial text.

[0053] For example, the similarity value ranges from [0, 1] or [0, 100%], and correspondingly, the recognition result ranges from [0, 1].

[0054] The matching degree can also be represented by a score. The content in the initial text is scored based on similarity to obtain a score that can represent the similarity.

[0055] If any content in the initial recognition result passes the screening, the initial recognition result is determined as the target recognition result; if any content in the initial recognition result fails the screening, the initial text can be re-recognized to obtain a new initial recognition result.

[0056] In this embodiment, the matching degree between the content in the initial recognition result and its corresponding content in the initial text is determined. Then, based on the matching degree, the content in the initial recognition result is filtered to obtain the target recognition result. The target text is then generated based on the target recognition result. In this way, the content of the initial recognition result that has passed the filtering can be determined as the target recognition result. At this time, based on the target recognition result, the accurate target text can be obtained, achieving the effect of efficiently and accurately extracting text and tables from PDF files.

[0057] In some embodiments, when determining the matching degree between the content in the initial recognition result and its corresponding content in the initial text, the initial recognition result and the second task instruction information are input into the large language model to obtain the matching degree between the content in the initial recognition result and its corresponding content in the initial text.

[0058] Specifically, the text and table corresponding to each part of the initial recognition result are scored separately to obtain the text score and table score of each part; the text score and table score corresponding to each part are used as the matching degree between the part and its corresponding content in the initial text.

[0059] The second task instruction information includes a second task instruction and a second thought chain reasoning example. The second task instruction is an instruction to perform the matching degree determination task, used to instruct the large language model to determine the matching degree between the content in the initial recognition result and its corresponding content in the initial text. The second thought chain reasoning example is a thought chain reasoning example for performing the matching degree determination step. The second task instruction information is used to instruct the large language model to determine the matching degree according to the second thought chain reasoning example.

[0060] Specifically, the second thought chain reasoning example includes a preset recognition result and a thought chain reasoning result. The thought chain reasoning result is the matching degree obtained by determining the matching degree of the preset recognition result.

[0061] The second task instruction information includes two parts: (1) the second task instruction, which is used to inform the big language model of the specific task content; and (2) the second thought chain reasoning example, which is an input text and its corresponding thought chain reasoning result, which is used to inform the big language model of the thought chain logic for determining the matching degree of the initial recognition result.

[0062] Specifically, the content in the initial recognition result includes multiple parts. When determining the matching degree between the content in the initial recognition result and its corresponding content in the initial text, for each part, the matching degree between that part and its corresponding content in the initial text is determined.

[0063] Since the content in the initial recognition result corresponds to the content in the initial text, the content in the initial recognition result includes multiple parts. Correspondingly, the content in the initial text can also be divided into multiple parts, corresponding to the multiple parts in the content of the recognition result.

[0064] The initial recognition results include both text and tables.

[0065] Of course, the text and tables in the initial recognition results can also be further divided based on the actual working conditions. For example, the text in the initial recognition results can be further divided into first text and second text, and the tables in the initial recognition results can be further divided into first table and second table.

[0066] Accordingly, in the second thought chain reasoning example, the thought chain reasoning result includes multiple matching degrees determined by matching degree determination of each part of the preset recognition result.

[0067] At this point, the initial recognition results and the second task instruction information are input into the large language model to obtain the matching degree between each part of the initial recognition results and its corresponding content in the initial text.

[0068] For example, taking the initial recognition result as including two parts: text and table, the second task instruction information is: (1) You (the large language model) are a data analyst, mainly responsible for scoring the output table and text data based on the initial text (i.e., scoring / determining the matching degree of each part of the initial recognition result based on the initial text). The range of score / matching degree is 0-1. The higher the recognition accuracy, the closer the score / determined matching degree is to 1; (2) For example, input the initial text: {Grade 1, Class 3} The student information table is as follows: Name Age Gender Score\nXiaoming 6 Male 89\nXiaohong 7 Female 98}. The initial recognition result is: {"text": The student information table for Class 3, Grade 1 is as follows: "table": Name|Age|Gender|Score\nXiaoming|6|Male|89\nXiaohong|7|Female|98}. The output is: {The student information table for Class 3, Grade 1 is as follows: "0.98", Name|Age|Gender|Score\nXiaoming|6|Male|89\nXiaohong|7|Female|98: "0.99"}. That is, the output format is: {"text": score1, "table": score2}, where text is the text in the initial recognition result, table is the table in the initial recognition result, score1 is the matching degree between the text in the initial recognition result and its corresponding content in the initial text, and score2 is the matching degree between the table in the initial recognition result and its corresponding content in the initial text.

[0069] In this embodiment, the initial recognition result is divided into multiple parts. The initial recognition result and the second task instruction information are input into the large language model to obtain the matching degree of each part of the initial recognition result with its corresponding content in the initial text. The second task instruction information includes a second task instruction and a second thought chain reasoning example for determining the matching degree. This second task instruction information is used to instruct the large language model to determine the matching degree according to the second thought chain reasoning example. In this way, by determining the matching degree of each part of the content in the initial recognition result with its corresponding content in the initial text, the content of each part of the initial recognition result can be filtered based on the matching degree, ensuring the accuracy of the final target recognition result and the target text determined based on the target recognition result.

[0070] In some embodiments, the content in the initial recognition result is filtered based on the matching degree. When the target recognition result is obtained, for each part of the content in the initial recognition result, it is determined whether to retain the part based on the matching degree between the part and the corresponding content in the initial text. Based on whether to retain the part and the various parts of the content in the initial recognition result, the target recognition result is determined.

[0071] Based on whether or not to retain this part, and considering the various parts of the content in the initial recognition result, the content of all parts retained in the initial recognition result is determined as the target recognition result.

[0072] Specifically, for each part of the content in the initial recognition result, when determining whether to retain the part based on the matching degree between the part and its corresponding content in the initial text, the determination is based on whether the matching degree between the part and its corresponding content in the initial text is greater than the preset matching degree corresponding to the part.

[0073] If the degree of matching between this part and its corresponding content in the initial text is greater than the preset degree of matching between this part, then this part is retained; if the degree of matching between this part and its corresponding content in the initial text is not greater than the preset degree of matching between this part, then this part is discarded.

[0074] Specifically, when the text score of a portion of the target recognition result is greater than the first threshold and the table score is greater than the second threshold, the portion is retained and the format of the text and table in the portion is converted to the format required by the business; when the text score of a portion of the target recognition result is less than or equal to the first threshold, the text in the portion is discarded; when the table score of a portion of the target recognition result is less than or equal to the second threshold, the table in the portion is discarded.

[0075] In the initial recognition result, the preset matching degree (i.e., the first threshold and the second threshold) corresponding to each part of the content can be the same or different.

[0076] In this embodiment, for each part of the initial recognition result, based on the matching degree of each part with its corresponding content in the initial text, the part whose matching degree with the content in the initial text is not greater than the preset matching degree of the corresponding part is discarded, and the part whose matching degree with the content in the initial text is greater than the preset matching degree of the corresponding part is retained. This achieves the deletion of the low-accuracy part in the initial recognition result and the retention of the high-accuracy part in the initial recognition text as the target recognition result, thereby ensuring the accuracy of the target text generated based on the target recognition result.

[0077] In some embodiments, when generating target text based on target recognition results, the target recognition results are converted into a business requirement format, and the content in the target recognition results is merged to generate target text.

[0078] Business requirements include target table format and target text format.

[0079] After obtaining the target recognition results, the text format in the target recognition results is converted to the target text format, and the table format in the target recognition results is converted to the target table format.

[0080] Subsequently, when merging the content in the target recognition results, the target recognition results in the business requirement format can be merged according to the target data layout to obtain the target text.

[0081] In this context, target layout refers to the layout of the target recognition results in the PDF file, or in other words, the positional relationship between the text and tables in the target recognition results and their corresponding content in the PDF file.

[0082] In this embodiment, after obtaining the target recognition result, the text and table formats in the target recognition result are converted into the business requirement format. Then, the target recognition result in the business requirement format is merged according to the positional relationship between the corresponding content of the text and table in the PDF file, i.e., the target layout, to obtain the target text. In this way, a standardized target text can be obtained to ensure accurate recognition of the content in the PDF file.

[0083] In some embodiments, the large language model can be pre-trained using the abundant and massive PDF file data in the logistics field, enabling the large language model to have a better understanding of PDF knowledge in the logistics field and to more accurately and stably identify the content of PDF files.

[0084] In some embodiments, by changing the model structure and increasing the model size, the model can be made to handle more complex problems, thereby more accurately, stably and efficiently recognizing the content in PDFs.

[0085] In some embodiments, the aforementioned large language model is a locally deployed private model. Based on the large model, text and tables are recognized, and the text and tables in the initial recognition results are filtered to obtain the target recognition text, thereby generating the target text. This can ensure the security of data processing and effectively prevent the leakage of important internal data and information.

[0086] For example, the PDF file parsing process can be shown in Figure 2. A PDF parsing tool, such as the open-source library PyPDF2, is used as the PDF Loader to load / read the content of the PDF file as txt text data (i.e., the initial text). The tables and text content in the txt text data are relatively messy. Then, the first stage of LLM (Large Language Model) prompt COT (i.e., the first task instruction information) optimization is performed on the txt text data, with the aim of recognizing the text and tables (i.e., the initial recognition result). Next, the text and tables from the initial recognition result are input into the Large Language Model for the second stage of LLM prompt COT optimization, with the aim of scoring the recognized text and tables to obtain text scores (txt scores) and table scores (table scores), i.e., determining the matching degree. If the text score is greater than a first threshold and the table score is greater than a second threshold, the format of the recognized text and table is converted to the format required by the business, such as md format; if the text score is less than or equal to the first threshold, the recognized text is discarded; if the table score is less than or equal to the second threshold, the recognized table is discarded. Finally, the .md format text is merged with the recognized table to obtain the target text.

[0087] Exemplary device

[0088] As shown in Figure 3, this application embodiment also provides a PDF file parsing device, including a parsing module 301, a recognition module 302, and a generation module 303.

[0089] in,

[0090] Parsing module 301 is used to parse PDF files using a PDF parsing tool to obtain the initial text;

[0091] The recognition module 302 is used to input the initial text and the first task instruction information into the large language model to obtain the initial recognition result. The initial recognition result includes the text and table corresponding to the content in the initial text. The first task instruction information includes the first thought chain reasoning example and the standard output format. The first thought chain reasoning example is a thought chain reasoning example for text and table recognition. The first task instruction information is used to instruct the large language model to perform text and table recognition on the initial text according to the first thought chain reasoning example and output the initial recognition result according to the standard output format.

[0092] The generation module 303 is used to generate target text based on the initial recognition results.

[0093] The PDF file parsing device provided in this embodiment belongs to the same concept as the PDF file parsing method provided in the above embodiments of this application. It can execute the method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in this embodiment can be found in the specific processing content of the PDF file parsing method provided in the above embodiments of this application, and will not be repeated here.

[0094] The functions implemented by the parsing module 301, the identification module 302 and the generation module 303 can be implemented by the same or different processors calling software, and this application embodiment does not limit this.

[0095] Exemplary electronic devices

[0096] Another embodiment of this application also proposes an electronic device, as shown in FIG4, which includes a memory 400 and a processor 410.

[0097] The memory 400 is connected to the processor 410 and is used to store programs;

[0098] The processor 410 is configured to implement the PDF file parsing method disclosed in any of the above embodiments by running a program stored in the memory 400.

[0099] Specifically, the electronic device may also include: a bus, a communication interface 420, an input device 430, and an output device 440.

[0100] The processor 410, memory 400, communication interface 420, input device 430, and output device 440 are interconnected via a bus. Among them:

[0101] A bus can include a pathway for transmitting information between various components of a computer system.

[0102] The processor 410 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or at least one integrated circuit used to control the execution of the program of the present application. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0103] Processor 410 may include a main processor, as well as a baseband chip, modem, etc.

[0104] The memory 400 stores a program for executing the technical solution of this application, and may also store an operating system and other key business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 400 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0105] Input device 430 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0106] Output device 440 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0107] The communication interface 420 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0108] The processor 410 executes the program stored in the memory 400 and calls other devices, which can be used to implement each step of any of the PDF file parsing methods provided in the above embodiments of this application.

[0109] Those skilled in the art will understand that the structure shown in Figure 4 is only a block diagram of a part of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0110] This application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in the memory through the data interface to execute the PDF file parsing method described in any of the above embodiments. For the specific processing procedure and its beneficial effects, please refer to the embodiments of the PDF file parsing method described above.

[0111] In addition to the methods and devices described above, embodiments of this application propose a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the PDF file parsing methods according to various embodiments of this application described in the "Exemplary Methods" section of this specification.

[0112] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0113] Furthermore, embodiments of this application also propose a storage medium storing a computer program thereon, the computer program being executed by a processor of the steps in the PDF file parsing methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0114] The basic principles of the present invention have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in the present invention are merely examples and not limitations, and should not be considered as essential features of each embodiment of the present invention. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the present invention to the necessity of employing the aforementioned specific details.

[0115] The block diagrams of devices, apparatuses, devices, and systems involved in this invention are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” and “having” are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0116] It should also be noted that in the apparatus, device, and method of the present invention, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of the present invention.

[0117] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the invention. Therefore, the invention is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0118] It should be understood that the qualifying terms "first", "second", "third", "fourth", "fifth" and "sixth" used in the description of the embodiments of the present invention are only used to more clearly illustrate the technical solutions and are not intended to limit the scope of protection of the present invention.

[0119] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the invention to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for parsing PDF files, characterized in that, The method includes: The PDF file is parsed using a PDF parsing tool to obtain the initial text; The initial text and the first task instruction information are input into the large language model to obtain the initial recognition result. The initial recognition result includes text and tables corresponding to the content in the initial text. The first task instruction information includes a first task instruction, a first thought chain reasoning example, and a standard output format. The first thought chain reasoning example is a thought chain reasoning example for text and table recognition. The first task instruction information is used to instruct the large language model to perform text and table recognition on the initial text according to the first thought chain reasoning example and output the initial recognition result according to the standard output format. Based on the initial recognition results, the target text is generated.

2. The PDF file parsing method according to claim 1, characterized in that, The first task instruction information also includes the PDF parsing tool.

3. The PDF file parsing method according to claim 1, characterized in that, The step of generating target text based on the initial recognition result includes: Determine the degree of matching between the content in the initial recognition result and its corresponding content in the initial text; Based on the matching degree, the content in the initial recognition result is filtered to obtain the target recognition result; Based on the target recognition results, the target text is generated.

4. The PDF file parsing method according to claim 3, characterized in that, The content of the initial recognition result includes multiple parts, and determining the matching degree between the content of the initial recognition result and its corresponding content in the initial text includes: The initial recognition result and the second task instruction information are input into the large language model to obtain the matching degree of each of the multiple parts with its corresponding content in the initial text. The second task instruction information includes a second task instruction and a second thought chain reasoning example. The second thought chain reasoning example is a thought chain reasoning example for performing the matching degree determination step. The second task instruction information is used to instruct the large language model to perform the matching degree determination according to the second thought chain reasoning example.

5. The PDF file parsing method according to claim 4, characterized in that, The step of inputting the initial recognition result and the second task instruction information into the large language model to obtain the matching degree of each of the plurality of parts with its corresponding content in the initial text includes: Each part of the initial recognition result is scored, including both the text and the table, to obtain a text score and a table score for each part. The text score and table score corresponding to each part are used as the matching degree between the part and its corresponding content in the initial text.

6. The PDF file parsing method according to claim 5, characterized in that, The step of filtering the content in the initial recognition result based on the matching degree to obtain the target recognition result includes: For each part of the content in the initial recognition result, it is determined whether to retain the part based on the degree of matching between the part and the content corresponding to it in the initial text; All the portions that are determined to be retained are identified as the target recognition result.

7. The PDF file parsing method according to claim 6, characterized in that, The step of determining whether to retain a portion based on its matching degree with the corresponding content in the initial text includes: If the degree of matching between the part and its corresponding content in the initial text is greater than the preset degree of matching between the part, then the part is retained; If the degree of matching between the part and its corresponding content in the initial text is not greater than the preset degree of matching between the part, then the part is discarded.

8. The PDF file parsing method according to claim 7, characterized in that, If the degree of matching between the portion and its corresponding content in the initial text is greater than the preset degree of matching between the portion and the initial text, then the portion is retained, including: When the text score of the target recognition result is greater than a first threshold and the table score is greater than a second threshold, the portion is retained and the text and table format of the portion are converted into the business requirement format.

9. The PDF file parsing method according to claim 7, characterized in that, If the matching degree between the part and its corresponding content in the initial text is not greater than the preset matching degree corresponding to the part, then the part is discarded, including: When the text score of a portion of the target recognition result is less than or equal to the first threshold, the text in that portion is discarded. When the table score of a portion of the target recognition result is less than or equal to the second threshold, the table in that portion is discarded.

10. The PDF file parsing method according to claim 3, characterized in that, The step of generating the target text based on the target recognition result includes: The text and table formats of the target recognition results are converted into business requirement formats, wherein the business requirement formats include target table formats and target text formats; The target recognition results, converted to the format required by the business requirements, are merged according to the target layout to obtain the target text, wherein the target layout is the layout of the target recognition results in the PDF file.

11. The PDF file parsing method according to any one of claims 1 to 10, characterized in that, Before inputting the initial text and the first task instruction information into the large language model to obtain the initial recognition result, the method further includes: The large language model is pre-trained using PDF file data from the logistics field.

12. The PDF file parsing method according to any one of claims 1 to 10, characterized in that, The PDF parsing tool is the open-source library PyPDF2, and the initial text is txt text data.

13. A PDF file parsing device, characterized in that, The device includes: The parsing module is used to parse PDF files using PDF parsing tools to obtain the initial text; The recognition module is used to input the initial text and the first task instruction information into the large language model to obtain an initial recognition result. The initial recognition result includes text and tables corresponding to the content in the initial text. The first task instruction information includes a first task instruction, a first thought chain reasoning example, and a standard output format. The first thought chain reasoning example is a thought chain reasoning example for text and table recognition. The first task instruction information is used to instruct the large language model to perform text and table recognition on the initial text according to the first thought chain reasoning example and output the initial recognition result according to the standard output format. The generation module is used to generate target text based on the initial recognition results.

14. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory is connected to the processor and is used to store programs; The processor is used to implement the PDF file parsing method as described in any one of claims 1 to 12 by running a program in the memory.

15. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the PDF file parsing method as described in any one of claims 1 to 12.

Images