PDF (Portable Document Format) file analysis method and device, equipment and storage medium

Abstract

The invention discloses a PDF (Portable Document Format) file analysis method, device and equipment and a storage medium, relates to the technical field of data processing, and discloses a PDF file analysis method which comprises the following steps: reading a file stream of a to-be-analyzed PDF file; preprocessing a file stream of the PDF file to generate structured data; and based on a PDF extraction model obtained by pre-training, performing configuration parameter analysis on the structured data to obtain complete PDF analysis data. Based on the scheme, key information extraction and table content formatting extraction of the PDF file are completed in an intelligent analysis mode of the PDF extraction model, so that normalized extraction of PDF file contents in different formats is realized, the dependency that the PDF file is not matched with a predefined template is avoided, and the problem of incomplete data extraction caused by a conventional scheme is reduced.

Description

Technical Field

[0001] The present application relates to the technical field of data processing, and in particular to a PDF file parsing method, device, equipment and storage medium. Background Art

[0002] In today's information society, PDF files are widely used document formats with rich content and complex structures, especially in business, scientific research and government documents. File parsing technology is crucial for data extraction and information management. However, parsing PDF files is extremely challenging, especially parsing tables. Tables in PDF documents often carry key structured data, such as financial reports, statistical data and scientific research data. The accuracy and efficiency of their parsing directly affect the accuracy and efficiency of data analysis.

[0003] Although traditional AI parsing technology has made some progress in processing text and simple-structured PDF files, it often performs unsatisfactorily in the face of the heterogeneity of PDF files, including inconsistent content layout, various format changes, and a large number of file pages. These characteristics make automatic parsing complicated, especially when encountering tables with unstable headers. The current solution relies on SDK to read PDF content, extract coordinate information of text and tables, and then match this information with predefined templates. However, this method is too dependent on templates and lacks flexibility. It cannot adapt to the extraction of PDF content in different formats, resulting in incomplete PDF data extraction.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a PDF file parsing method, device, equipment and storage medium, aiming to solve the normalized extraction of the content of PDF files of different formats and reduce the technical problem of incomplete data extraction caused by conventional solutions.

[0006] To achieve the above purpose, the present application proposes a PDF file parsing method, the method comprising:

[0007] Read the file stream of the PDF file to be parsed;

[0008] Preprocessing the file stream of the PDF file to be parsed to generate structured data;

[0009] Based on the pre-trained PDF extraction model, the structured data is parsed according to configuration parameters to obtain complete PDF parsed data.

[0010] In one embodiment, the step of reading the file stream of the PDF file to be parsed includes:

[0011] Use JAVA's input and output streams to open the PDF file to be parsed and obtain the PDF data stream;

[0012] The data of the PDF data stream is read based on the reading method of the input and output streams.

[0013] In one embodiment, the step of preprocessing the file stream of the PDF file to be parsed includes:

[0014] Perform a check on the file stream to verify whether the basic attributes of the file meet expectations;

[0015] Perform a secondary check based on the result of the primary check to verify the security and correctness of the file, and determine whether to process it based on the result of the secondary check;

[0016] If the result of the secondary verification is passed, processing is performed, otherwise an error prompt is returned.

[0017] In one embodiment, the step of preprocessing the file stream of the PDF file to be parsed to generate structured data includes:

[0018] Obtaining a result of the secondary verification of the preprocessing;

[0019] Based on the result of the secondary verification, the file stream of the PDF file is processed by table recognition and structured data is extracted therefrom.

[0020] In one embodiment, the training step of the pre-trained PDF extraction model includes:

[0021] Get the content of the PDF file;

[0022] Use PDF parsing tools to read the PDF file content for preprocessing to obtain a PDF file dataset;

[0023] Performing data annotation on the PDF file dataset;

[0024] Based on the labeled PDF file dataset, a deep learning framework is used to train the model and obtain a trained PDF extraction model.

[0025] In one embodiment, before the step of performing configuration parameter parsing on the structured data to obtain complete PDF parsed data based on the pre-trained PDF extraction model, the step further includes:

[0026] The extraction prompt word template of the PDF extraction model is loaded, and the PDF extraction prompt word template is used to assist the PDF extraction model in extracting PDF key information from the structured data.

[0027] In one embodiment, the step of performing configuration parameter parsing on the structured data to obtain complete PDF parsed data based on the pre-trained PDF extraction model further includes:

[0028] Load the pre-trained PDF extraction model;

[0029] Use the PDF extraction model to identify the table of structured data, and parse the header and basic information of the table through the large model

[0030] Obtaining a normalized header of the table based on the header and basic information of the table;

[0031] Based on the normalized header, construct a mapping relationship between the header and the cell content;

[0032] Complete PDF parsing data is obtained based on the mapping relationship and PDF key information.

[0033] In addition, to achieve the above purpose, the present application also proposes a PDF file parsing device, the PDF file parsing device comprising:

[0034] The file stream reading module is used to read the file stream of the PDF file to be parsed;

[0035] A preprocessing module, used for preprocessing the file stream of the PDF file to be parsed to generate structured data;

[0036] The model parsing module performs configuration parameter parsing on the structured data based on the pre-trained PDF extraction model to obtain complete PDF parsing data.

[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes a PDF file parsing device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the PDF file parsing method described above.

[0038] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the PDF file parsing method described above are implemented.

[0039] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the PDF file parsing method described above are implemented.

[0040] One or more technical solutions proposed in this application have at least the following technical effects:

[0041] By reading the file stream of the PDF file to be parsed, further preprocessing the PDF file stream to be parsed, generating structured data after two checks, and parsing the configuration parameters of the structured data based on the pre-trained PDF extraction model to obtain complete PDF parsing data. Based on the present application scheme, the key information extraction and table content formatting extraction of the PDF file are completed by intelligent parsing of the PDF extraction model, thereby realizing the normalized extraction of the content of PDF files in different formats. Compared with the prior art, it avoids the dependency of the PDF file not matching the predefined template and reduces the problem of incomplete data extraction caused by conventional schemes. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1 A flowchart of the first embodiment of the PDF file parsing method of the present application is provided;

[0045] Figure 2 A flowchart of the second embodiment of the PDF file parsing method of this application is provided;

[0046] Figure 3 A flowchart of the third embodiment of the PDF file parsing method of this application is provided;

[0047] Figure 4 A flowchart of the fourth embodiment of the PDF file parsing method of the present application is provided;

[0048] Figure 5 A flowchart of the fifth embodiment of the PDF file parsing method of the present application is provided;

[0049] Figure 6A flowchart of the sixth embodiment of the PDF file parsing method of this application is provided;

[0050] Figure 7 A flowchart of the seventh embodiment of the PDF file parsing method of the present application is provided;

[0051] Figure 8 A flowchart of the eighth embodiment of the PDF file parsing method of the present application is provided;

[0052] Fig. 9 This is a schematic diagram of the module structure of the PDF file parsing device according to an embodiment of the present application;

[0053] Fig.10 Schematic diagram of the device structure of the hardware operating environment involved in the PDF file parsing method in the embodiment of the present application. DETAILED DESCRIPTION

[0054] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0055] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0056] The main solution of the embodiment of the present application is: by reading the file stream of the PDF file to be parsed; preprocessing the file stream of the PDF file to generate structured data; based on the pre-trained PDF extraction model, the configuration parameters of the structured data are parsed to obtain complete PDF parsing data. Based on the solution of the present application, the key information extraction and table content formatting extraction of the PDF file are completed by intelligent parsing of the PDF extraction model, thereby realizing the normalized extraction of the content of PDF files in different formats and reducing the problem of incomplete data extraction caused by conventional solutions.

[0057] Based on this, the present application embodiment provides a PDF file parsing method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the PDF file parsing method of the present application.

[0058] In this embodiment, the PDF file parsing method includes steps S10 to S30:

[0059] Step S10: Read the file stream of the PDF file to be parsed.

[0060] It should be noted that there is an Aspose library in JAVA that is used to process various file formats such as PDF, Word, Excel, PowerPoint, images, and emails. It provides a set of powerful APIs that allow developers to create, read, modify, and convert these files in JAVA applications without using software such as Microsoft Office or Adobe Acrobat. The Aspose library is widely used in enterprise applications to implement automated document processing and file conversion functions.

[0061] Specifically, the PDF file to be parsed is opened through the JAVA input and output stream, such as FileInputStream, and data is read from the opened PDF file stream. After reading the file stream, the file stream is closed to release resources.

[0062] Step S20: pre-processing the file stream of the PDF file to be parsed to generate structured data.

[0063] It should be noted that the preprocessing includes two verifications, one of which includes at least verifying the file type and file size to verify whether the basic attributes of the file meet expectations, and the second verification includes at least password verification and format verification to verify the security and correctness of the file.

[0064] Specifically, the file stream of the PDF file to be parsed is checked twice to ensure the accuracy and completeness of the data. After obtaining the result of the secondary check, the file stream of the PDF file is processed by table recognition based on the result of the secondary check and structured data is extracted therefrom.

[0065] Step S30, based on the pre-trained PDF extraction model, the configuration parameters of the structured data are parsed to obtain complete PDF parsed data.

[0066] It should be noted that, by defining an extraction hint word template, it is used to assist PDF in extracting key content from a PDF file.

[0067] Specifically, by loading the extraction prompt word template of the PDF extraction model and the pre-trained PDF extraction model, PDF key information and table headers can be extracted from the structured data, and the normalized header of the table can be obtained based on the header of the table, and the mapping relationship between the header and the cell content can be further obtained. Finally, based on the mapping of the table and cell content and the PDF key information, complete PDF parsing data can be obtained.

[0068] In this embodiment, the file stream of the PDF file to be parsed is read; the file stream of the PDF file is preprocessed to generate structured data; based on the pre-trained PDF extraction model, the structured data is configured with parameter parsing to obtain complete PDF parsing data; the key information extraction and table content formatting extraction of the PDF file are completed by intelligent parsing of the PDF extraction model, thereby achieving normalized extraction of the content of PDF files in different formats and reducing the problem of incomplete data extraction caused by mismatch of predefined templates.

[0069] Further, refer to Figure 2 The second embodiment of the PDF file parsing method of the present application provides a flow chart based on the above Figure 2 In the embodiment shown, the step S10 of "reading the file stream of the PDF file to be parsed" is further refined, including steps A201 to A202:

[0070] Step A201: Use JAVA's input and output streams to open the PDF file to be parsed and obtain a PDF data stream;

[0071] Step A202: Read the data of the PDF data stream based on the input / output stream reading method.

[0072] Specifically, you can use JAVA to create a FileInputStream object, pass in the path of the PDF file, open the PDF file to be parsed, and then use the read method of FileInputStream to read the data in the file stream. After reading the file stream of the PDF file to be parsed, use the close method of FileInputStream to close the file stream.

[0073] In this embodiment, the file stream of the PDF file to be parsed is read through the JAVA input and output stream mode, and is normally closed after reading the file stream, ensuring the correctness of the file path, which is conducive to improving the accuracy and efficiency of the file stream reading of the PDF file, and also has certain redundancy, such as exception management mechanism and resource release.

[0074] Further, refer to Figure 3 The third embodiment of the PDF file parsing method of the present application provides a flow chart based on the above Figure 3 In the embodiment shown, the step S20 of "preprocessing the file stream of the PDF file to be parsed" is further refined to include steps A301 to A303:

[0075] Step A301: perform a check on the file stream to verify whether the basic attributes of the file meet expectations.

[0076] Step A302: Perform a secondary check based on the result of the primary check to verify the security and correctness of the file, and determine whether to process it based on the result of the secondary check.

[0077] Step A303: If the result of the secondary verification is passed, processing is performed, otherwise an error prompt is returned.

[0078] It should be noted that the first verification includes at least type verification and size verification, which are used to verify whether the basic attributes of the file are in compliance, and the second verification includes at least password verification and format verification.

[0079] Specifically, by checking the MIME type or extension of the PDF file, ensure that the file type is PDF. If the file type meets expectations, proceed to the next step, otherwise an error message is returned; at the same time, by obtaining the length of the file stream, compare it with the preset maximum file size. Ensure that the file size is within a reasonable range. If the file size meets expectations, proceed to the next step, otherwise an error message is returned; when verifying whether the password is correct, you can use a PDF parsing library, such as Apache PDFBox, to try to open the PDF file and enter the preset password. If the password is correct, proceed to the next step, otherwise an error message is returned; when verifying whether the internal structure and format of the PDF file meet expectations, use the PDF parsing library to parse the PDF file, check whether the number of pages, content, metadata, etc. of the file meet expectations, if the format is correct, proceed to the next step; otherwise an error message is returned; process the PDF file that has passed the secondary verification, convert the file stream of the file to be parsed into structured data, and if the secondary verification fails, an error message is generated to prompt the user that the file verification failed.

[0080] In this embodiment, the correctness of the file path can be ensured based on a primary verification, a secondary verification and a processing process. Among them, an exception handling mechanism and resource release are included when the file stream is opened and closed to avoid resource waste, thereby improving the accuracy and efficiency of file stream preprocessing.

[0081] Further, refer to Figure 4 The fourth embodiment of the PDF file parsing method of the present application provides a flow chart based on the above Figure 4 In the embodiment shown, the step S20 of "preprocessing the file stream of the PDF file to be parsed to generate structured data" is further refined to include steps A401 to A402:

[0082] Step S401: Obtain the result of the secondary verification of the preprocessing.

[0083] Step S402: Based on the result of the secondary verification, the file stream of the PDF file is processed by table recognition to extract structured data therefrom.

[0084] It should be noted that the table recognition refers to extracting table data from a PDF file using a table recognition tool or library; the structured data at least includes a character string to be extracted, formatted PDF basic information, and formatted table information.

[0085] Specifically, the result of the secondary verification of the file stream of the PDF file to be parsed is checked to determine whether the file passes the verification. If the file stream passes the verification, a PDF parsing library such as Apache PDFBox, Tabula, etc. is further used to identify the table in the PDF file, extract the table header and cell content of the table, and convert it into formatted data. Among them, the table header and cell content can be converted into a structured data format such as JSON and XML, and the structured data at least includes the character string to be extracted, formatted PDF basic information and formatted table information.

[0086] In this embodiment, by obtaining the result of the secondary verification and processing the file stream of the PDF file using table recognition, unified structured data is obtained, which effectively improves the data extraction efficiency of the PDF extraction model and avoids the problems of incomplete data extraction and low flexibility of PDF files in various formats.

[0087] In one possible embodiment, referring to Figure 5 , the step S20 further includes:

[0088] By reading the file stream of the PDF file, the basic attributes of the file and the correctness of the file are verified. The table recognition method is used for the verified file stream to obtain the character string to be extracted, the formatted PDF basic information and the formatted table information. Otherwise, an error message is returned.

[0089] The above are only two feasible implementations of step S20 provided in this embodiment, and this embodiment does not specifically limit the specific implementation of step S20.

[0090] Further, refer to Figure 6 The fifth embodiment of the PDF file parsing method of the present application provides a flow chart based on the above Figure 6 In the embodiment shown, the "training steps of the pre-trained PDF extraction model" are further refined to include steps A601 to A604:

[0091] Step A601: Obtain the content of the PDF file.

[0092] Step A602: Use a PDF parsing tool to read the PDF file content and perform preprocessing to obtain a PDF file dataset.

[0093] Step A603: perform data annotation on the PDF file dataset.

[0094] Step A604: Perform model training using a deep learning framework based on the annotated PDF file dataset to obtain a trained PDF extraction model.

[0095] It should be noted that the preprocessing at least includes data cleaning, standardization, format conversion and other operations on the PDF file; the data annotation is to annotate the preprocessed PDF file data and add label information; the deep learning framework includes at least TensorFlow, PyTorch, etc., and optimizes the model performance by adjusting hyperparameters such as learning rate, batch size, etc. In addition, the training of the PDF extraction model is applied to NLP, which is the full name of Natural Language Processing. It is a branch of computer science that involves artificial intelligence, linguistics and computer science, and mainly studies how to process and understand human natural language. The main goal of NLP is to enable computers to understand, generate, parse and operate the language used by humans in daily life to achieve functions such as human-computer interaction, information retrieval, sentiment analysis, machine translation, and question-answering systems.

[0096] Specifically, PDF files are collected from various sources, and a PDF file data set for training a PDF extraction model is collected, wherein a PDF parsing tool is used to read the PDF file content, perform data cleaning, remove invalid data and redundant information, and store the preprocessed PDF file content as structured data, such as JSON or XML format. A data annotation tool or algorithm is used to annotate the key content and table headers of the PDF file, and the annotated data set is used to train the PDF extraction model. The validation set is used to evaluate the model, and the model's accuracy, recall rate and other indicators are calculated. According to the evaluation results, the model parameters are adjusted to improve the performance.

[0097] This embodiment effectively conducts the training process of the PDF extraction model to obtain a PDF extraction model with reasonable design, high data quality, appropriate parameter adjustment, accurate model evaluation and stable model deployment, which is conducive to more efficient, accurate and stable PDF file extraction process, providing a good user experience and higher application value.

[0098] In a feasible implementation, before "based on the pre-trained PDF extraction model, performing configuration parameter parsing on the structured data to obtain complete PDF parsed data" in step S30, it also includes: loading an extraction prompt word template of the PDF extraction model, and using the PDF extraction prompt word template to assist the PDF extraction model in extracting PDF key information from the structured data.

[0099] It should be noted that the PDF extraction prompt word template is used to assist the PDF extraction model in extracting key information from structured data. Among them, the data serialization format includes at least XML, JSON and YAML, and the YAML (YAML AintMarkup Language) is a data serialization format, which is commonly used for configuration files, data exchange and storage. Compared with XML and JSON, YAML is generally considered to be more concise, easier to read and write, and its design goal is to make it easy for both humans and machines to read and write.

[0100] Specifically, an extraction prompt word template can be defined, the defined extraction prompt word template can be saved as a JSON file, the extraction prompt word template can be loaded from the JSON file into the PDF extraction model, and the loaded extraction prompt word template can be used to guide the PDF extraction model to extract key information from structured data.

[0101] In this embodiment, by loading the PDF extraction prompt word template in the PDF extraction model, the accuracy and efficiency of PDF file parsing are effectively improved, the maintainability and scalability of the PDF extraction model are enhanced, the redundant operations of the PDF extraction model when extracting key information are reduced, the model parsing process is simplified, unnecessary calculation and processing time are reduced, the user experience is improved, and the waiting time and error information are reduced.

[0102] Further, refer to Figure 7 The sixth embodiment of the PDF file parsing method of the present application provides a flow chart based on the above Figure 7 The embodiment shown further refines the step S30 of "based on the PDF extraction model obtained by pre-training, performing configuration parameter parsing on the structured data to obtain complete PDF parsing data", including steps A701 to A705:

[0103] Step A701: Load a pre-trained PDF extraction model.

[0104] Specifically, load the pre-trained PDF extraction model from the model storage path, verify whether the model loading is successful, and ensure that the model file is complete and not damaged.

[0105] Step A702: Use the PDF extraction model to identify the table of the structured data, and parse the header and basic information of the table through the large model.

[0106] Specifically, the table part in the PDF file is identified by using the PDF extraction model, and the header information of the table is identified and extracted by the large language model to ensure the accuracy and completeness of the header information.

[0107] Step A703: Obtain a normalized header of the table based on the header and basic information of the table.

[0108] It should be noted that the normalization of the extracted table header and basic information is to ensure the consistency of the header format, eliminate redundant information and inconsistency in the header, and facilitate the subsequent construction of mapping relationships.

[0109] Specifically, the extracted header information is cleaned and standardized through the PDF extraction model, and the header information is converted into a unified format, such as removing spaces, unifying upper and lower case, etc.

[0110] Step A704: Based on the normalized header, a mapping relationship between the header and the cell content is constructed.

[0111] It should be noted that the mapping relationship is used to establish an association between the table header and the cell content, which facilitates subsequent data extraction and analysis.

[0112] Specifically, based on the normalized header information, the cell contents in the table are identified, and a mapping relationship between the header and the cell contents is constructed to ensure that each header corresponds to the correct cell content.

[0113] Step A705: Obtain complete PDF parsing data based on the mapping relationship and PDF key information.

[0114] Specifically, the mapping relationship between the table header and the cell content and the PDF key information are integrated to generate structured parsed data.

[0115] In this embodiment, based on the pre-trained PDF extraction model, the structured data is configured with parameter parsing to obtain complete PDF parsing data, and the header information is normalized, which effectively eliminates the inconsistency of the PDF file format, ensures the accuracy, efficiency and completeness of the PDF file parsing, and makes the PDF extraction model more efficient, reliable and user-friendly in practical applications.

[0116] In one possible embodiment, referring to Figure 8 , the step S30 further includes:

[0117] Obtain the character string to be extracted, formatted PDF basic information and formatted table information, load the pre-trained PDF extraction model and load the PDF extraction prompt word template, the character string to be extracted obtains structured key data based on the PDF extraction model and the PDF extraction prompt word template, the formatted PDF basic information identifies the table based on the PDF extraction model, and the table header and basic information of the table are parsed by a large model for normalization to obtain the structured header number, the table Map data is obtained based on the mapping relationship between the constructed header and the cell content, and the structured key data and the table Map data are constructed to obtain complete PDF parsed data.

[0118] The above are only two feasible implementations of step S30 provided in this embodiment, and this embodiment does not specifically limit the specific implementation of step S30.

[0119] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the PDF file parsing method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0120] In addition, this application also provides a PDF file parsing device, please refer to Fig. 9 , the PDF file parsing device comprises:

[0121] A file stream reading module, used for reading the file stream of the PDF file to be parsed;

[0122] A preprocessing module, used for preprocessing the file stream of the PDF file and generating structured data according to the preprocessing result;

[0123] The model parsing module is based on the PDF extraction model obtained by pre-training, and the structured data obtains complete PDF parsing data through the model parsing configuration parameters of the PDF extraction model.

[0124] The PDF file parsing device provided by the present application adopts the PDF file parsing method in the above embodiment, which can solve the technical problem of incomplete data extraction caused by the dependency of the PDF file not matching the predefined template and the conventional solution. Compared with the prior art, the beneficial effects of the PDF file parsing device provided by the present application are the same as the beneficial effects of the PDF file parsing method provided by the above embodiment, and the other technical features in the PDF file parsing device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0125] In addition, the present application also provides a PDF file parsing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the PDF file parsing method in the above-mentioned embodiment 1.

[0126] Reference below Fig.10 , which shows a schematic diagram of the structure of a PDF file parsing device suitable for implementing the embodiment of the present application. The PDF file parsing device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.10 The PDF file parsing device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0127] like Fig.10As shown, the PDF file parsing device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the xxx device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the PDF file parsing device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a PDF file parsing device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0128] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0129] The PDF file parsing device provided by the present application adopts the PDF file parsing method in the above embodiment, which can solve the technical problem of incomplete data extraction caused by the dependency of the PDF file not matching the predefined template and the conventional solution. Compared with the prior art, the beneficial effects of the PDF file parsing device provided by the present application are the same as the beneficial effects of the PDF file parsing method provided by the above embodiment, and the other technical features in the PDF file parsing device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0130] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0131] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0132] In addition, the present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the PDF file parsing method in the above embodiment.

[0133] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0134] The computer-readable storage medium may be included in the PDF file parsing device; or may exist independently without being installed in the PDF file parsing device.

[0135] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a PDF file parsing device, the PDF file parsing device: based on a pre-trained PDF extraction model, the structured data obtains complete PDF parsing data through the model parsing configuration parameters of the PDF extraction model.

[0136] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0137] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0138] The modules involved in the embodiments described in the present application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0139] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned PDF file parsing method, and can solve the technical problems of incomplete data extraction caused by the dependency of PDF files not matching predefined templates and conventional solutions. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the PDF file parsing method provided by the above-mentioned embodiment, and will not be repeated here.

[0140] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A PDF file parsing method, characterized in that: The method comprises: Read the file stream of the PDF file to be parsed; Preprocessing the file stream of the PDF file to be parsed to generate structured data; Based on the pre-trained PDF extraction model, the structured data is parsed according to configuration parameters to obtain complete PDF parsed data.

2. The method according to claim 1, characterized in that The step of reading the file stream of the PDF file to be parsed comprises: Use JAVA's input and output streams to open the PDF file to be parsed and obtain the PDF data stream; The data of the PDF data stream is read based on the reading method of the input and output streams.

3. The method according to claim 2, characterized in that The step of preprocessing the file stream of the PDF file to be parsed includes: Perform a check on the file stream to verify whether the basic attributes of the file meet expectations; Perform a secondary check based on the result of the primary check to verify the security and correctness of the file, and determine whether to process it based on the result of the secondary check; If the result of the secondary verification is passed, processing is performed, otherwise an error prompt is returned.

4. The method according to claim 3, characterized in that The step of preprocessing the file stream of the PDF file to be parsed to generate structured data includes: Obtaining a result of the secondary verification of the preprocessing; Based on the result of the secondary verification, the file stream of the PDF file is processed by table recognition and structured data is extracted therefrom.

5. The method according to claim 4, characterized in that The training steps of the pre-trained PDF extraction model include: Get the content of the PDF file; Use PDF parsing tools to read the PDF file content for preprocessing to obtain a PDF file dataset; Performing data annotation on the PDF file dataset; Based on the labeled PDF file dataset, a deep learning framework is used to train the model and obtain a trained PDF extraction model.

6. The method according to claim 5, characterized in that Before the step of performing configuration parameter parsing on the structured data to obtain complete PDF parsing data based on the pre-trained PDF extraction model, the step further includes: The extraction prompt word template of the PDF extraction model is loaded, and the PDF extraction prompt word template is used to assist the PDF extraction model in extracting PDF key information from the structured data.

7. The method according to claim 6, characterized in that The step of performing configuration parameter parsing on the structured data to obtain complete PDF parsed data based on the pre-trained PDF extraction model includes: Load the pre-trained PDF extraction model; Using the PDF extraction model to identify the table of the structured data, and parsing the header and basic information of the table through the large model; Obtaining a normalized header of the table based on the header and basic information of the table; Based on the normalized header, construct a mapping relationship between the header and the cell content; Complete PDF parsing data is obtained based on the mapping relationship and PDF key information.

8. A PDF file parsing device, characterized in that: The device comprises: The file stream reading module is used to read the file stream of the PDF file to be parsed; A preprocessing module, used for preprocessing the file stream of the PDF file to be parsed to generate structured data; The model parsing module performs configuration parameter parsing on the structured data based on the pre-trained PDF extraction model to obtain complete PDF parsing data.

9. A PDF file parsing device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the PDF file parsing method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the PDF file parsing method according to any one of claims 1 to 7 are implemented.

Images