Multi-dimensional PDF literature analysis and metadata extraction method

Abstract

The invention discloses a multi-dimensional PDF literature analysis and metadata extraction method, and relates to the technical field of data processing. According to the method, a layered, collaborative and self-adaptive hybrid analysis framework is constructed. The method comprises the following steps: firstly, preprocessing and analyzing a PDF document, and judging language and type attributes of the PDF document; then, shunting processing is carried out according to a judgment result; Grobid is preferentially adopted for analyzing the English text type PDF; and an analysis scheme based on an OCR (Optical Character Recognition) engine is adopted for the Chinese or picture type PDF. For an uncovered area or low-confidence metadata of the method, a refined extraction strategy based on a large language model and a multi-OCR voting mechanism is innovatively introduced to improve the accuracy. In addition, the method further comprises the steps of text recombination based on coordinate mapping, reference structured analysis based on a three-layer progressive rule and the like. According to the method, on the premise that the cost is controlled, high-precision and automatic extraction of PDF document metadata, text and reference document information is achieved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a multi-dimensional PDF document parsing and metadata extraction method, which is applicable to the intelligent parsing and metadata extraction of multilingual and multi-type PDF documents. Background Technology

[0002] With the rapid development of artificial intelligence, especially large language model technology, massive amounts of high-quality text data have become crucial for model training and optimization. Academic papers and patent documents, as highly condensed forms of human knowledge, are extremely valuable training corpora and data sources. However, PDFs, as the standard format for disseminating academic documents, are designed for reading and printing and typically lack machine-readable structured semantic information. Automating and accurately extracting metadata (such as title, author, abstract, keywords, publication date, etc.), text content, and references from unstructured PDF documents has been a long-standing and complex technical challenge.

[0003] While existing technologies offer various approaches to PDF parsing, each has significant limitations when applied individually, making it difficult to meet the current demands for large-scale, high-precision, and multi-scenario applications. For example, dedicated parsing tools (such as Grobid) are efficient at processing English text-based PDFs with similar training corpora, but suffer from language incompatibility and incomplete metadata extraction when dealing with Chinese documents or PDFs with complex layouts. Large language models, while possessing powerful understanding capabilities and able to overcome language and format barriers, incur high computational costs and time overhead when directly used for long document parsing, making them unsuitable for handling massive amounts of data. Traditional OCR technology, while capable of recognizing multilingual text, lacks an understanding of the document's logical structure, relying solely on fixed rules for information extraction, resulting in low accuracy, poor generalization ability, and an inability to reliably extract rich metadata and complex structures such as references. These shortcomings collectively constitute the technological bottleneck for automatically and cost-effectively acquiring high-quality structured data from diverse PDF documents. Summary of the Invention

[0004] 1. The technical problem that the invention aims to solve In view of the shortcomings of the existing technology, the present invention provides a multi-dimensional PDF document parsing and metadata extraction method. The present invention designs a hierarchical, collaborative, and adaptive hybrid parsing framework. This hybrid parsing framework intelligently selects and combines the optimal technology stack according to the inherent characteristics of the document, such as language and file type. Through strategies such as coarse-grained positioning + fine-grained extraction, rule priority + model fallback, and multi-source cross-validation, it strives to maximize the accuracy and completeness of extracted information while ensuring processing efficiency and controlling the total cost.

[0005] 2. Technical Solution To achieve the above objectives, the technical solution provided by this invention is as follows: The present invention provides a method for multi-dimensional PDF document parsing and metadata extraction, comprising the following steps: Step 1: Preprocess and stream the PDF document, converting each page of the PDF document independently into a high-resolution byte stream or image buffer format, denoted as an ordered list [page-1, page-2, ..., page-N], to provide a standard and unified input for subsequent parsing; Step 2: Determine the language and type attributes of the document through sampling statistics and trial-and-error analysis strategies; Step 3: Receive and verify the user-uploaded custom metadata template. If the verification is successful, inject it into the subsequent extraction process as the priority basis for metadata extraction. Step 4: For documents identified as English text in Step 2, enable the Grobid-based parsing path, use the Grobid server for efficient parsing, perform coarse extraction of metadata for page areas not covered by Grobid parsing, and determine the file type based on the rules. Step 5: For documents identified as Chinese text or image type in Step 2, enable the parsing path based on the OCR engine, use the OCR engine with layout analysis capabilities to process all pages, perform a coarse extraction of metadata for the page areas not covered by the OCR parsing, and combine the rules to determine the file type. Step 6: Extract metadata from the uncovered page areas based on the large language model and voting mechanism; Step 7: Standardize the parsing results using coordinate system mapping, and perform global sorting and splicing based on the coordinates of the text blocks to reorganize the text structure; Step 8: Identify and structure the reference blocks using a three-tiered progressive rule; Step 9: Integrate the refined metadata, structured text, and references, and synthesize the final structured document object output according to the predefined template.

[0006] Furthermore, step 2 includes: Step 2.1, Language Attribute Judgment: Select several representative pages using a sampling strategy, perform fast scanning using a lightweight OCR engine, and calculate the ratio of Chinese characters to English characters in the extracted plain text. Determine the dominant language type of the document based on a preset threshold, and use this dominant language type as the document's language attribute. Step 2.2, Type Attribute Determination: When the language attribute is determined to be English, attempt to use Grobid to perform a tentative parsing of the document. If the parsing is successful, mark the document as "text type"; if the parsing fails, determine the document as "image type".

[0007] Furthermore, step 4 includes: Step 4.1: Submit all pages to the Grobid server for parsing; Step 4.2: Using the coordinate information output by the Grobid server, mark the successfully parsed areas as "covered" and the remaining areas as the uncovered page areas (page-s). Step 4.3: Use the OCR engine to parse the uncovered page area page-s to obtain preliminary parsed text. Then, use a regular expression parser combined with the Grobid parsing results to extract metadata and determine the file type.

[0008] Furthermore, step 5 includes: Step 5.1: Use an OCR engine with layout analysis capabilities to process all pages and output recognition results with hierarchical structure and coordinate information; Step 5.2: Based on the prior knowledge definition rules of academic document structure, identify the core content blocks and mark them as "covered", and record the remaining areas as the uncovered page areas page-s; Step 5.3: Apply a targeted set of regular expressions to extract basic metadata and make a preliminary judgment on file types.

[0009] Furthermore, the document type determination outputs a confidence score based on the matched feature combination. If the score is greater than a set threshold, the output is "academic paper" or "patent document". If the score is less than or equal to the set threshold, the output is a custom type.

[0010] Furthermore, step 6 includes: Step 6.1: For the uncovered page area page-s, use multiple different lightweight OCR engines to perform text recognition and obtain different text versions of the same content; Step 6.2: Based on the initially determined file type or user-defined metadata template, generate instruction prompts for extracting metadata from the large model; Step 6.3: Submit the prompt words to the large language model for independent metadata extraction, and obtain a set of metadata extraction results; Step 6.4: For each metadata field, collect its value from all extraction results and select the final value using a frequency statistics-based decision fusion strategy. Step 6.5: Merge and resolve conflicts between the metadata of each field after fusion and determination, and the stable information extracted from the main parsing path to form a complete document metadata dictionary.

[0011] Furthermore, in step 6.4, the frequency-based decision fusion strategy includes majority voting and weighted average, selecting the value with the highest frequency as the final value of the metadata extracted by multiple OCR engines.

[0012] Furthermore, in step 6.4, for cases where the frequency is the same or all results are empty, the following alternative strategies are adopted in sequence: prioritize the selection of values ​​based on the historical confidence scores of each OCR engine on the metadata field; if the confidence scores cannot be determined, then fall back to the results extracted based on regular expressions.

[0013] Furthermore, the three-level progressive rule in step 8 includes: The first layer involves comprehensively utilizing title features, format features, and location features to identify the reference area; The second layer involves splitting continuous text into independent single references within the identified reference area based on the serial number pattern or line beginning alignment feature. The third layer involves using regular expressions and heuristic rules to extract the author, title, source, year, and DOI / ISBN subfields for each reference.

[0014] Furthermore, in step 9, if the system fails to identify any known type, the document is defined as a "general document," and the document title and complete identification text content are extracted.

[0015] 3. Beneficial effects Compared with existing known technologies, the technical solution provided by this invention has the following significant advantages: (1) The present invention provides a multi-dimensional PDF document parsing and metadata extraction method that does not rely on a single technical path. Instead, it designs a hierarchical, collaborative, and adaptive hybrid parsing framework. Based on the inherent characteristics of the document, such as language and file type, the hybrid parsing framework intelligently selects and combines the optimal technology stack. Through strategies such as coarse-grained positioning + fine-grained extraction, rule priority + model fallback, and multi-source cross-validation, it strives to maximize the accuracy and completeness of extracted information while ensuring processing efficiency and controlling total cost.

[0016] (2) The multi-dimensional PDF document parsing and metadata extraction method of the present invention first performs rapid analysis on the PDF document to determine its language attributes and file type; for English text PDFs, Grobid is used for efficient parsing; for Chinese PDFs or image PDFs, an enhanced parsing scheme based on PaddleOCR is adopted; for metadata parts that are difficult to extract completely or have low confidence in both methods, a lightweight large model voting mechanism is innovatively introduced for refinement; at the same time, the present invention designs a set of text reconstruction methods based on coordinate mapping and a hierarchical progressive reference identification and parsing rule base to solve the problems of multi-source parsing result fusion and complex structure identification. Finally, a unified and structured JSON or XML format result is output, covering document type, full metadata, structured text and reference list, aiming to achieve a balance between total processing cost and parsing accuracy. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the implementation of a multi-dimensional PDF document parsing and metadata extraction method provided by the present invention. Figure 2 This is a flowchart of the parsing process using Grobid provided by the present invention; Figure 3 This is a flowchart of text parsing using PaddleOCR provided by the present invention; Figure 4 This is a flowchart of the large model metadata extraction process provided by the present invention. Detailed Implementation

[0018] To further understand the content of this invention, a detailed description of the invention will be provided in conjunction with the accompanying drawings and embodiments.

[0019] See Figure 1 The multi-dimensional PDF document parsing and metadata extraction method of this embodiment includes the following core steps in its specific implementation process: Step 1: Preprocess and stream the PDF document to provide a standard and unified input for all subsequent parsing paths. The input PDF document undergoes unified preprocessing, converting all pages into an internal representation format. Specifically, each page of the PDF document is independently converted into a high-resolution byte stream or image buffer format (typically 200 dpi), denoted as an ordered list [page-1, page-2, ..., page-N]. This step aims to standardize the input, eliminating the impact of differences in PDF versions, embedded fonts, etc., providing a consistent page data source for all subsequent processing modules.

[0020] Step 2: Through sampling statistics and trial-and-error parsing strategies, intelligently determine the language and type attributes of the document. This step is the foundation for subsequent traffic splitting and includes two sequential sub-steps: Step 2.1, Language Attribute Determination: A sampling strategy is employed to balance speed and reliability. Several representative pages (e.g., first page, middle pages, last page) are randomly selected or selected at fixed intervals, denoted as page-n. A lightweight OCR engine without layout analysis capabilities is used to quickly scan these sample pages and extract the plain text. Subsequently, the number and proportion of Chinese and English characters in the extracted plain text are statistically analyzed. Based on a pre-defined threshold set using statistical experience based on typical academic document characteristics (e.g., a document is considered Chinese if the proportion of Chinese characters exceeds a certain percentage (e.g., 10%)), the dominant language type (Chinese or English) of the document is determined. This determination provides priority guidance for subsequent processes; the system retains the ability to handle mixed language content.

[0021] Step 2.2, Type Attribute (Text / Image) Determination: This determination is closely related to the use of Grobid. The strategy in this embodiment is: when the language is determined to be English, attempt to use Grobid to exploratoryly parse the document. If parsing is successful and non-empty content is obtained, the document is marked as "text," and the parsing results from Grobid can be used initially; if parsing fails and throws a specific exception ("PDF parsing resulted in empty content"), the document is immediately determined to be "image" (generally a PDF file generated from multiple images or screenshots), and subsequent processes will bypass Grobid and directly enter the OCR processing branch.

[0022] Step 3: Receive and verify the user-uploaded custom metadata template. After successful verification, inject it into the subsequent extraction process as the priority basis for metadata extraction. This step allows users to provide custom structured parsing rules for specific document formats, improving the flexibility of parsing.

[0023] After receiving a custom metadata template uploaded by a user in the form of structured data (such as JSON), the system will perform strict format and logic validation. If the validation fails, it will return False and indicate the reason for the error; if the validation succeeds, the parsed template will be injected into the subsequent metadata extraction process as a priority.

[0024] The format specifications for parsing templates are as follows: json { "file_type": "User-defined type name, such as 'Conference Paper'", "fields": [ {"name": "Field 1 name", "description": "Field 1 description"}, {"name": "Field 2 name", "description": "Field 2 description"}, / / ... More fields ], "few-shots": [ { "raw_text": "A typical example of raw text from this type of document", "result": {"Field1 name": "Corresponding value 1", "Field2 name": "Corresponding value 2", ...} }, / / ... More examples ]} If a custom metadata template is set, the file types include: academic papers, patent documents, and custom types.

[0025] Step 4: For English text documents, prioritize using the Grobid-based parsing path. This path first utilizes the Grobid server for efficient parsing, extracts metadata rules for page areas not covered by Grobid parsing to fill in gaps, and then uses these rules to determine the file type. Step 4.1, Grobid Main Parsing: Submit all pages to the Grobid server for parsing. The Grobid server will output semi-structured XML data including title, author, abstract, main text sections, and references. If the Grobid server throws an exception or returns an empty result in this step, it is considered a failure of this path, and the system will automatically jump to step 5.

[0026] Step 4.2, Page Filtering and Key Region Location: Using the text and reference coordinate information output by the Grobid server, successfully parsed page regions are marked as "covered" in the pages list. The remaining page regions may contain necessary metadata, including authors, doi, etc., and are extracted separately and recorded as uncovered page regions (page-s, supplementary pages).

[0027] Step 4.3, Metadata Coarse Extraction and Rule-Based Initial File Type Determination: If the user has entered a file type, this step is skipped. For documents with an undefined file type, since Grobid server parsing will lose some metadata, this embodiment uses an OCR engine to parse the uncovered page area (page-s) to obtain preliminary parsed text. Through a series of pre-set regular expression parsers, the metadata of the preliminary text is parsed out. Combined with the metadata information obtained from Grobid parsing, a confidence score is output based on the matched feature combinations (such as "Abstract", "DOI" or "claims", "application number"). If the score is greater than a set threshold, the determined file type is output, such as "academic paper" or "patent document"; if the score is less than or equal to the set threshold (less than 60% of the matched features), the output file type is a custom type.

[0028] Step 5: For Chinese text documents or image-based PDF documents, prioritize the parsing path based on the OCR engine. This path first uses an OCR engine with layout analysis capabilities to process all pages, identify core content blocks, and then applies rules to extract basic metadata and determine the file type for the remaining uncovered page areas. Step 5.1, Overall OCR Analysis: Use an OCR engine with layout analysis capabilities (such as PaddleOCR) to process all pages and output recognition results with hierarchical structure and coordinate information.

[0029] Step 5.2, Block Filtering Based on Page Layout Analysis: Based on the prior knowledge of academic document structure, identify "main text blocks" and "reference blocks," etc. After marking these core content blocks as "covered," the remaining page areas are recorded as uncovered page areas (page-s) for subsequent metadata extraction.

[0030] Step 5.3, Metadata Coarse Extraction and Rule-Based Initial File Type Determination: If the user has entered a file type, skip this step. For documents with no clearly defined file type, apply a targeted set of regular expressions to extract basic metadata and make an initial determination of the file type. If the user has defined a metadata template, a threshold judgment is also required on the confidence score; if the score is low (less than 60% of the features are matched), it is determined to be a custom type.

[0031] Step 6: Refine and extract metadata from uncovered page areas based on a large language model and voting mechanism. This step aims to leverage the understanding capabilities of a large language model to extract metadata from uncovered page regions denoted as page-s, reducing the extraction of metadata from useless page regions to improve accuracy and efficiency.

[0032] This step uses an OCR engine without page layout parsing capabilities. In addition to improving running speed, it also avoids the loss of some metadata information (such as the journal number in the header and the author biography in the footnote) by OCR engines with page layout parsing capabilities. Using a lightweight OCR engine will parse all the text, perform high-precision metadata extraction in the uncovered page areas, and ensure stability through a fusion strategy.

[0033] Step 6.1: Multiple OCR Text Alternate Generation: For each area in page-s, multiple different lightweight OCR engines are used to perform text recognition, obtaining different text versions of the same content (Text_v1, Text_v2,...) to compensate for the recognition errors of a single engine. Here, a lightweight OCR engine refers to an OCR engine that only has text recognition capabilities and lacks page layout, image, and table recognition functions, thus improving running speed. Furthermore, compared to the OCR engine with page layout parsing capabilities in Step 5, this OCR engine can recognize all text on the page, including headers (generally containing metadata such as journal information) and footnotes (generally containing metadata such as author biographies), improving the accuracy of metadata extraction.

[0034] Step 6.2: Construct a prompt word template: Based on the file type initially determined in Steps 4.3 and 5.3, select an appropriate prompt word template; The system has built-in prompt word templates for two commonly used file types: academic papers and patent documents. The prompt word template for academic papers includes fields such as doi, journal number, author, mailing address, keywords, publication time, abstract, and title metadata. For Chinese documents, bilingual author, mailing address, keywords, abstract, and title need to be extracted. The prompt word template for patent documents includes fields such as patent name, basic metadata group, abstract, claims, specification, and appendix.

[0035] If the user has uploaded a valid custom metadata template, the system will customize the metadata type based on the template and generate instruction prompts for extracting large model metadata by combining the selected template with the original OCR text extracted from the uncovered page area page-s.

[0036] Step 6.3, Parallel Information Extraction from the Large Language Model: The prompt words are submitted to the Large Language Model API for independent metadata extraction, resulting in a set of metadata extraction results (Result_v1, Result_v2, ...).

[0037] Step 6.4, Multi-Result Decision Fusion: For each metadata field that needs to be extracted, collect its metadata values ​​from all extracted results (Result_vX). For example: Author: [`OCR1`: {"value": "Wang San", "score": 0.5}, `OCR2`: {"value": "Wang San", "score": 0.4}, `OCR3`: {"value": "Li Si", "score": 0.6}], `Abstract`:......} Adopt a decision fusion strategy based on frequency statistics (such as majority voting, weighted average, etc.), and select the value with the highest occurrence frequency as the final value. The final value described here refers to the final value of the metadata values extracted by multiple OCR engines. In the above example, for the metadata of `Author`, the value of `Wang San` has the highest score and is used as the final value of `Author`.

[0038] For the situation where the occurrence frequencies are the same or all results are empty, the system will successively adopt the following alternative strategies for decision-making: First, based on the historical confidence scores of each OCR engine in this metadata field, select the value extracted by the OCR engine with the highest score; if the confidence scores cannot determine either, then finally fallback to the results extracted based on regular expressions in step 4 or step 5.

[0039] Confidence is a statistical result, such as the average accuracy rate of multiple OCR engines for each metadata (such as Author) in multiple tests. Assume that the confidence levels of multiple OCR engines for `Author` are 90%, 80%, and 70% respectively, and the extraction results are: {"Author": [ `OCR1`: {"value": "Wang San", "score": 0.5}, `OCR2`: {"value": "Zhang麻子", "score": 0.5}, `OCR3`: {"value": "Li Si", "score": 0.5} ]}, then according to the result of OCR1, it is `Wang San`. Or use the results of the metadata extracted by regular expressions in steps 4 and 5. This mechanism aims to reduce the impact of fluctuations in a single data source or single model parsing on the results.

[0040] Step 6.5, Final metadata integration and output: Integrate and resolve conflicts between the metadata of each field determined by fusion and the stable information extracted by the main parsing path, form a complete document metadata dictionary, and record the data sources and confidence levels of each field.

[0041] Step 7, Perform coordinate standardization mapping on the parsing results, and perform global sorting and splicing according to the text block coordinates to reorganize the text structure For the parsing results of the Grobid server, it already provides the coordinate information of the text.

[0042] The text blocks in the OCR engine's parsing results have their own coordinates, which are different from Grobid's coordinates, mainly due to the difference in dpi. OCR can be set programmatically, while Grobid's coordinates are fixed at 72.

[0043] Therefore, the results of OCR parsing are normalized and mapped to the coordinate system (scaled by dpi / 72) to correspond to Grobid coordinates.

[0044] After mapping the coordinates of the main text content, the system performs global sorting and splicing based on the coordinates of the text blocks (prioritizing the Y-coordinate of the top-left corner, followed by the X-coordinate) to form a preliminary reading order. For complex layouts such as multi-column layouts and mixed text and images, the system can combine block types and visual dividing lines to perform auxiliary, rule-based, or lightweight model-based order adjustments to improve the coherence of the main text.

[0045] Step 8: Intelligent identification and structured parsing of reference blocks using a three-tiered progressive rule system. First layer: Block boundary detection. This layer comprehensively utilizes title features (such as "References"), format features (hanging indentation), and positional features (end of document) to identify reference areas.

[0046] Second layer: Item splitting. Within the identified reference area, continuous text is split into independent single references based on the serial number pattern (such as [1], 1.) or the line beginning alignment feature.

[0047] The third layer: Structured field extraction. For each reference, a series of targeted regular expressions and heuristic rules are applied to extract sub-fields, specifically including the following fields: Author field: Matches author names in both Chinese and English. Chinese: Common Chinese name patterns (usually 2 to 4 characters, but not limited to). English: Patterns such as initials followed by a period or full name followed by a comma.

[0048] Title field: Captures text within book titles or quotation marks, or identifies it by specific fonts or styles.

[0049] Source field: Identifies journal, conference, or publisher information containing keywords such as "Journal of" or "Vol."

[0050] Year field: Matches four-digit years.

[0051] DOI / ISBN fields: Match standard formats using strict regular expressions.

[0052] Step 9: Integrate the refined metadata, structured text, and references, and synthesize the final structured document object output according to the predefined template. Information fusion: The refined metadata, structured text, and structured reference list are combined according to a predefined template to form the final structured document object.

[0053] Document Type Determination and Fallback: If the system fails to identify any known type, the document is defined as a "general document." Under this fallback, the system extracts the document's title and provides the complete recognized text content, ensuring basic structured output and preventing completely unparseable cases.

[0054] Output: The final result is output in a standardized data format (such as JSON-LD, XML).

[0055] The present invention and its embodiments have been described above illustratively. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention, and are not actually limited thereto. Therefore, if those skilled in the art are inspired by this description and design similar embodiments and examples without departing from the spirit of the present invention, they should all fall within the protection scope of the present invention.

Claims

1. A method for multi-dimensional PDF document parsing and metadata extraction, characterized in that, Includes the following steps: Step 1: Preprocess and stream the PDF document, converting each page of the PDF document independently into a high-resolution byte stream or image buffer format, denoted as an ordered list [page-1, page-2, ..., page-N], to provide a standard and unified input for subsequent parsing; Step 2: Determine the language and type attributes of the document through sampling statistics and trial-and-error analysis strategies; Step 3: Receive and verify the user-uploaded custom metadata template. If the verification is successful, inject it into the subsequent extraction process as the priority basis for metadata extraction. Step 4: For documents identified as English text in Step 2, enable the Grobid-based parsing path, use the Grobid server for efficient parsing, perform coarse extraction of metadata for page areas not covered by Grobid parsing, and determine the file type based on the rules. Step 5: For documents identified as Chinese text or image type in Step 2, enable the parsing path based on the OCR engine, use the OCR engine with layout analysis capabilities to process all pages, perform a coarse extraction of metadata for the page areas not covered by the OCR parsing, and combine the rules to determine the file type. Step 6: Extract metadata from the uncovered page areas based on the large language model and voting mechanism; Step 7: Standardize the parsing results using coordinate system mapping, and perform global sorting and splicing based on the coordinates of the text blocks to reorganize the text structure; Step 8: Identify and structure the reference blocks using a three-tiered progressive rule; Step 9: Integrate the refined metadata, structured text, and references, and synthesize the final structured document object output according to the predefined template.

2. The method for multi-dimensional PDF document parsing and metadata extraction according to claim 1, characterized in that, Step 2 includes: Step 2.1, Language Attribute Judgment: Select several representative pages using a sampling strategy, perform fast scanning using a lightweight OCR engine, and calculate the ratio of Chinese characters to English characters in the extracted plain text. Determine the dominant language type of the document based on a preset threshold, and use this dominant language type as the document's language attribute. Step 2.2, Type Attribute Determination: When the language attribute is determined to be English, attempt to use Grobid to perform a tentative parsing of the document. If the parsing is successful, mark the document as "text type"; if the parsing fails, determine the document as "image type".

3. The method for multi-dimensional PDF document parsing and metadata extraction according to claim 1, characterized in that, Step 4 includes: Step 4.1: Submit all pages to the Grobid server for parsing; Step 4.2: Using the coordinate information output by the Grobid server, mark the successfully parsed areas as "covered" and the remaining areas as the uncovered page areas (page-s). Step 4.3: Use the OCR engine to parse the uncovered page area page-s to obtain preliminary parsed text. Then, use a regular expression parser combined with the Grobid parsing results to extract metadata and determine the file type.

4. The method for multi-dimensional PDF document parsing and metadata extraction according to claim 1, characterized in that, Step 5 includes: Step 5.1: Use an OCR engine with layout analysis capabilities to process all pages and output recognition results with hierarchical structure and coordinate information; Step 5.2: Based on the prior knowledge definition rules of academic document structure, identify the core content blocks and mark them as "covered", and record the remaining areas as the uncovered page areas page-s; Step 5.3: Apply a targeted set of regular expressions to extract basic metadata and make a preliminary judgment on file types.

5. A method for multi-dimensional PDF document parsing and metadata extraction according to claim 3 or 4, characterized in that, The document type determination outputs a confidence score based on the matched feature combination. If the score is greater than a set threshold, the output is "academic paper" or "patent document". If the score is less than or equal to the set threshold, the output is a custom type.

6. The method for multi-dimensional PDF document parsing and metadata extraction according to claim 1, characterized in that, Step 6 includes: Step 6.1: For the uncovered page area page-s, use multiple different lightweight OCR engines to perform text recognition and obtain different text versions of the same content; Step 6.2: Based on the initially determined file type or user-defined metadata template, generate instruction prompts for extracting metadata from the large model; Step 6.3: Submit the prompt words to the large language model for independent metadata extraction, and obtain a set of metadata extraction results; Step 6.4: For each metadata field, collect its value from all extraction results and select the final value using a frequency statistics-based decision fusion strategy. Step 6.5: Merge and resolve conflicts between the metadata of each field after fusion and determination, and the stable information extracted from the main parsing path to form a complete document metadata dictionary.

7. The method for multi-dimensional PDF document parsing and metadata extraction according to claim 6, characterized in that, In step 6.4, the frequency-based decision fusion strategy includes majority voting and weighted average, and the value with the highest frequency is selected as the final value of the metadata extracted by multiple OCR engines.

8. The method for multi-dimensional PDF document parsing and metadata extraction according to claim 7, characterized in that, In step 6.4, for cases where the frequency is the same or all results are empty, the following alternative strategies are adopted in sequence: prioritize the selection of values ​​based on the historical confidence scores of each OCR engine on the metadata field; if the confidence scores cannot be determined, then fall back to the results extracted based on regular expressions.

9. The method for multi-dimensional PDF document parsing and metadata extraction according to claim 1, characterized in that, The three-level progressive rule in step 8 includes: The first layer involves comprehensively utilizing title features, format features, and location features to identify the reference area; The second layer involves splitting continuous text into independent single references within the identified reference area based on the serial number pattern or line beginning alignment feature. The third layer involves using regular expressions and heuristic rules to extract the author, title, source, year, and DOI / ISBN subfields for each reference.

10. The method for multi-dimensional PDF document parsing and metadata extraction according to claim 1, characterized in that, In step 9, if the system fails to identify any known type, the document is defined as a "general document", and the document title and complete identification text content are extracted.

Images