PDF document content processing method and device, equipment, storage medium and program product

Abstract

The invention discloses a PDF (Portable Document Format) document content processing method and device, equipment, a storage medium and a program product, and relates to the technical field of document structured processing. Preprocessing the PDF document to obtain a to-be-processed data set corresponding to each page of the PDF document; determining the page type of each page of the PDF document based on all the to-be-processed data sets and the image of each page of the PDF document; and based on the to-be-processed data set corresponding to each directory page and the image of the directory page, extracting a hierarchical structure relationship of each title data in the directory page, and constructing a directory tree. And matching the title data of the directory page and the title data of the non-directory page based on the semantic similarity and the text similarity between the title data of the directory page and the title data of the non-directory page, and correspondingly filling the content data under each title node of the directory tree according to a matching result to obtain a structured representation result of the PDF document. According to the method and the device, the semantic reduction degree and the structural quality of the PDF document are improved.

Description

Technical Field

[0001] This application relates to the field of document structuring technology, and in particular to methods, apparatus, devices, storage media, and computer program products for processing PDF document content. Background Technology

[0002] With the widespread digitization of documents such as textbooks, academic papers, legal documents, and corporate reports, PDF has become one of the most commonly used electronic document formats. While PDF can fully preserve layout and style, its lack of structured information poses significant challenges to content parsing, knowledge extraction, and semantic retrieval, especially in applications requiring high-precision chapter division and semantic reconstruction.

[0003] The structural processing of PDF documents mainly relies on page number ranges, fixed separators, or text length-based segmentation. Some improvement schemes attempt to introduce a table of contents for assistance, but most remain at the level of hard rule processing such as page number positioning, lacking in-depth analysis of the semantic hierarchy of the table of contents and accurate attribution of the main text content. These methods often lead to inaccurate chapter division, fragmented context, and semantic mismatches, especially in textbooks, regulations, or multi-level technical reports, where complex numbering systems and nested directories are difficult to map stably, resulting in the loss of the document's semantic hierarchy. Therefore, there is an urgent need for a high-precision structural processing method that integrates table of contents structure analysis, title alignment, semantic segmentation, and content mapping to improve the semantic restoration and structural recovery of PDF documents. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, device, storage medium, and computer product for processing PDF document content, aiming to solve the technical problem of inaccurate chapter division in related technologies for PDF document content processing.

[0005] To achieve the above objectives, this application proposes a method for processing PDF document content, the method comprising: Obtain a PDF document containing at least one page; The PDF document is preprocessed to obtain a dataset corresponding to each page of the PDF document; the dataset includes title data and / or content data. Based on all the datasets to be processed and images of each page of the PDF document, determine the page type of each page in the PDF document; the page type includes table of contents pages and / or non-table of contents pages; Based on the dataset to be processed and the image of each directory page, the hierarchical structure relationship of each title data in the directory page is extracted, and a directory tree is constructed. The node structure of the directory tree corresponds to the hierarchical structure of the title data in the directory page. Based on the semantic and textual similarity between the title data of the table of contents page and the title data of non-table of contents pages, the title data of the table of contents page and the title data of non-table of contents pages are matched. According to the matching results, the content data is filled into each title node of the table of contents tree to obtain the structured representation of the PDF document.

[0006] In one embodiment, the step of determining the page type of each page in a PDF document based on all datasets to be processed and images of PDF document pages includes: Sort the datasets to be processed according to page number. For each dataset, take the dataset to be processed and the two datasets before and after it as a group of data to be recognized. Input the data into the language page recognition model to obtain the first page type of the current page in the group of data to be recognized. The dataset to be processed is marked as the current page, the dataset before it is marked as the previous page, and the dataset after it is marked as the next page. After converting all pages of the PDF document into images, they are input into a visual page recognition model to obtain the second page type of all images to be recognized. Based on the first page type and the second page type, determine the page type of each page in the PDF document.

[0007] In one embodiment, the step of determining the page type of each page in a PDF document based on a first page type and a second page type includes: The first directory page number sequence is determined based on the first page type, and the second directory page number sequence is determined based on the second page type. Take the union of the first directory page number sequence and the second directory page number sequence, and take the longest consecutive page number sequence in the union as the first candidate page number sequence; The intersection of the first directory page number sequence and the second directory page number sequence is taken to obtain the first page number intersection sequence. Continuous interval extraction is performed on the first page number intersection sequence to obtain at least one second candidate page number sequence. For each second candidate page number sequence, take the intersection of the second candidate page number sequence and the first page number intersection sequence to obtain the second page number intersection sequence; Select the second candidate page number sequence that meets the preset requirements from all the second candidate page number sequences as the page number range of the table of contents of the PDF document; the intersection density is determined based on the ratio of the sequence length of the second page number intersection sequence to the sequence length of the second candidate page number sequence; the preset requirements include that the intersection density is greater than a preset threshold and the sequence length is the maximum value among all the second candidate page number sequences; If no second candidate page number sequence meets the preset requirements, the first candidate page number sequence will be used as the range of page numbers for the table of contents of the PDF document.

[0008] In one embodiment, the step of extracting the hierarchical structure relationship of each title data in the directory page and constructing a directory tree based on the dataset to be processed corresponding to each directory page and the image of the directory page includes: For each directory page, the dataset to be processed is input into the language-based directory structure extraction model to obtain the first directory level extraction result of the directory page. For each table of contents page of a PDF document, the table of contents page is converted into an image and input into a visual directory structure extraction model to obtain the second directory level extraction result of the table of contents page; A directory tree is constructed based on the extraction results of the first and second directory levels from the directory pages.

[0009] In one embodiment, the step of matching the title data of a directory page with the title data of a non-directory page based on the semantic and textual similarity between the title data of the directory page and the title data of a non-directory page includes: Retrieve the title data of all directory pages and the title data of all non-directory pages, and construct a directory title list and a main text title list respectively; Based on the order of the title data in the table of contents list, the table of contents list is traversed sequentially forward. For each table of contents title data traversed in the table of contents list, a strong match is performed on the subsequent text title data after the last matched text title data in the text title list, until the first text title data that meets the strong match condition is found or no text title data that meets the strong match condition is found. The strong match condition is determined based on the semantic similarity between the table of contents title data and the text title data, the text similarity between the table of contents title data and the text title data, and the strong match similarity threshold. After traversing all the table of contents title data, if there are table of contents title data that do not match the main text title data, then based on the weak matching condition, the table of contents title data that do not match the main text title data are weakly matched with the unmatched main text title data. The weak matching condition is determined based on the semantic similarity between the table of contents title data and the main text title data, the text similarity between the table of contents title data and the main text title data, and the weak matching similarity threshold. Obtain the title data matching table; each table of contents title data value corresponds to at most one body title data value, and each body title data value appears only once.

[0010] In one embodiment, the step of filling the content data into each title node of the directory tree according to the matching results to obtain the structured representation of the PDF document includes: Determine the location information of each text title in the title data matching table within the dataset to be processed; Based on location information and the matching relationship between the main text title data and the table of contents title data, the content data is populated into each title node of the table of contents tree to obtain a structured representation of the PDF document.

[0011] Secondly, to achieve the above objectives, this application further provides a PDF document content processing apparatus, the apparatus comprising: The acquisition module is used to acquire PDF documents, which contain at least one page. The preprocessing module is used to preprocess the PDF document to obtain the dataset to be processed corresponding to each page of the PDF document; the dataset to be processed includes title data and / or content data; The classification module is used to determine the page type of each page in a PDF document based on all the datasets to be processed and images of each page of the PDF document; the page type includes table of contents pages and / or non-table of contents pages; The directory tree construction module is used to extract the hierarchical structure relationship of each title data in the directory page based on the dataset to be processed and the image of the directory page, and to construct a directory tree. The node structure of the directory tree corresponds to the hierarchical structure of the title data in the directory page. The matching module is used to match the title data of the table of contents page with the title data of the non-table of contents page based on the semantic and textual similarity. According to the matching results, the content data is filled into the corresponding title nodes of the table of contents tree to obtain the structured representation of the PDF document.

[0012] Thirdly, to achieve the above objectives, this application further provides a PDF document content processing device, the device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the above-described PDF document content processing method.

[0013] Fourthly, to achieve the above objectives, this application further provides a storage medium, which is a computer-readable storage medium, and stores a computer program thereon. When the computer program is executed by a processor, it implements the steps of the above-described PDF document content processing method.

[0014] Fifthly, to achieve the above objectives, this application further provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the above-described PDF document content processing method.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: This application combines text recognition and image recognition to determine page type, extract directory structure, and match titles in PDF documents. This achieves accurate correspondence between directory page titles and non-directory page titles, improving the accuracy of PDF document recognition and reducing the possibility of problems such as inaccurate chapter division, contextual fragmentation, and semantic mismatch. The multi-layered matching method based on semantic and text similarity ensures accurate matching between directory titles and main text titles, accurately mapping content data to directory tree nodes to obtain a structured document representation. This effectively improves the semantic restoration and structure quality of PDF documents, facilitating subsequent operations such as retrieval, display, and knowledge extraction. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the PDF document content processing method in the embodiments of this application.

[0019] Figure 2 This is a schematic diagram of the module connections for a PDF document content processing device.

[0020] Figure 3 This is a schematic diagram of a PDF document content processing device.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] The main solution of this application embodiment is: by preprocessing the PDF document, combining language and visual models to perform dual recognition and fusion of page type, directory structure and title content, first constructing a directory tree corresponding to the directory page level, then achieving accurate matching between the directory title and the main text title based on semantic similarity and text similarity, and filling the main text content into the corresponding directory node, thereby obtaining a semantically complete and hierarchically clear structured PDF document representation.

[0025] Specifically, this application provides a method for processing PDF document content, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the PDF document content processing method of this application.

[0026] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or PDF document content processing device capable of performing the above functions. The following description uses a PDF document content processing device as an example to illustrate this embodiment and the subsequent embodiments.

[0027] In this embodiment, the PDF document content processing method includes steps S10 to S50: Step S10: Obtain the PDF document.

[0028] Step S20: Preprocess the PDF document to obtain a dataset to be processed corresponding to each page of the PDF document. The dataset to be processed includes title data and / or content data.

[0029] It should be noted that a PDF document contains at least one page. Step S20 involves preprocessing each page of the PDF document to obtain a corresponding dataset to be processed; that is, one page corresponds to one dataset to be processed. The title data may be a title item in the table of contents or a title item outside the table of contents. Content data represents data that is not a title item. For example, flag bits can be used in the dataset to be processed to mark data belonging to title items, thus distinguishing title data from body text data.

[0030] In one feasible implementation, step S20 may include steps A10 to A20: Step A10: Use a PDF parsing tool to parse the PDF document and obtain a set of analyzable objects to be processed; each object element in the set of objects to be processed contains text information, type tags, and page number information.

[0031] Step A20: Merge object elements with the same page number information into a single dataset to be processed.

[0032] In one example, an open-source PDF parsing tool (such as the open-source framework MinerU) can be used to parse and process the PDF document, resulting in a set of objects to be processed in JSON format, represented as: In this context, each curly brace {} represents an object element. Within each object element, `text` represents the text information, indicating the text content of that object element. "For type markers, ":1 indicates that the element belongs to the heading level and does not contain..." This indicates that the element in question is not a heading level element. This is page number information, indicating the page number of the PDF document where the object element is located.

[0033] Furthermore, they will have the same The object elements are merged to form a dataset to be processed.

[0034] Step S30: Based on all the datasets to be processed and the images of each page of the PDF document, determine the page type of each page of the PDF document; the page type includes table of contents pages and / or non-table of contents pages.

[0035] It should be noted that this step involves identifying the page type of each page in the processed PDF document. This is because the main steps in this embodiment require matching the title data of the table of contents page and non-table of contents pages. Therefore, this step is most effective when processing PDF documents that contain a table of contents page. Of course, it is understood that the processed PDF document may not contain a table of contents page, and this will not affect the implementation of this step.

[0036] Specifically, this step combines the dataset to be processed corresponding to each page of the PDF document with the images of each page to identify the page type of each page in the PDF. In a feasible implementation, step S30 may include steps B10 to B30: Step B10: Sort the datasets to be processed according to page number. For each dataset to be processed, take the dataset to be processed and the two datasets before and after it as a group of data to be recognized, and input them into the language page recognition model to obtain the first page type of the current page in the group of data to be recognized.

[0037] The dataset to be processed is marked as the current page, the previous dataset to be processed is marked as the previous page, and the next dataset to be processed is marked as the next page.

[0038] Step B20: After converting all pages of the PDF document into images, input them into the visual page recognition model to obtain the second page type of all images to be recognized.

[0039] Step B30: Determine the page type of each page in the PDF document based on the first page type and the second page type.

[0040] Step B30 includes steps B31 to B36: Step B31: Determine the first directory page number sequence based on the first page type, and determine the second directory page number sequence based on the second page type.

[0041] Step B32: Take the union of the first directory page number sequence and the second directory page number sequence, and take the longest consecutive page number sequence in the union as the first candidate page number sequence.

[0042] Step B33: Take the intersection of the first directory page number sequence and the second directory page number sequence to obtain the first page number intersection sequence, and extract continuous intervals from the first page number intersection sequence to obtain at least one second candidate page number sequence.

[0043] Step B34: For each second candidate page number sequence, take the intersection of the second candidate page number sequence and the first page number intersection sequence to obtain the second page number intersection sequence.

[0044] Step B35: Select a second candidate page number sequence that meets the preset requirements from all second candidate page number sequences as the table of contents page number range of the PDF document; the intersection density is determined based on the ratio of the sequence length of the second page number intersection sequence to the sequence length of the second candidate page number sequence; the preset requirements include that the intersection density is greater than a preset threshold and the sequence length is the maximum value among all second candidate page number sequences.

[0045] Step B36: If no second candidate page number sequence meets the preset requirements, then the first candidate page number sequence is used as the range of page numbers for the table of contents of the PDF document.

[0046] Specifically, to achieve accurate identification of PDF document page types, this embodiment uses a language-based page recognition model to identify page types based on the text data of the dataset to be processed, followed by a visual-based page recognition model. Finally, a fusion algorithm is used to determine the page type of each page in the PDF document based on the recognition results of the visual and language-based page recognition models. It should be noted that the input to the language-based page recognition model is the dataset to be processed corresponding to the current page, previous page, and next page, and the output is the page type identification result (first page type) and confidence score for the current page. The input to the visual-based page recognition model is an image from any dataset to be processed, and the output is the page type identification result (second page type) and confidence score for that dataset. The language-based page recognition model can be a general Large Language Model (LLM), and the visual-based page recognition model can be a general Vision Language Model (VLM). The confidence score output helps to avoid model illusions.

[0047] For example, all datasets to be processed in the dataset to be processed are arranged according to page number information. The datasets are sorted, and for each dataset to be processed, it, along with its preceding and following datasets, are treated as a group of data to be identified. This group, combined with prompt words, is then input into a language-based page recognition model to obtain the first page type. For ease of understanding, this can be represented as: in, This represents the dataset to be processed on the current page. This represents the dataset to be processed from the previous page of the current page. This indicates the next page of the dataset to be processed, where type represents the page type of the current page, and score represents the confidence score (first confidence information) of the language page recognition model.

[0048] Subsequently, each page of the PDF document is converted into an image. Combined with prompts, these images are sequentially input into a visual page recognition model to identify the page type of each page, thus obtaining the second page type. For ease of understanding, this can be represented as: in, This indicates the image on the current page, "type" indicates the page type, and "score" represents the confidence score (second confidence score) of the visual page recognition model.

[0049] Subsequently, based on the first page type and the second page type, the page type of each page is determined. Specifically, based on the first page type, the page number of the table of contents in the PDF document is determined as the first table of contents page number sequence LLM_pages. Then, based on the second page type, the page number of the table of contents in the PDF document is determined as the second table of contents page number sequence VLM_pages. The first table of contents page number sequence and the second table of contents page number sequence are combined into a union, and the longest consecutive page number sequence in the union is selected as the first candidate page number sequence r_longest.

[0050] Subsequently, the intersection of the first directory page number sequence LLM_pages and the second directory page number sequence VLM_pages is taken to obtain the first intersection sequence. and the first intersection sequence Perform continuous interval extraction (in this example, if the page number interval between two adjacent page numbers in the page number sequence is no more than 1, then the two page numbers are considered to be consecutive), to obtain at least one second candidate page number sequence r.

[0051] For each second candidate page number sequence r, for the second candidate page number sequence Intersection sequence with the first page number Then, the intersection is calculated to obtain the second page number intersection sequence. Based on the sequence length of the second page number intersection sequence and the sequence length of the second candidate page number sequence, the intersection density ratio of the second candidate page number sequence is determined. For ease of understanding, the intersection density ratio is calculated as follows: in, This indicates the sequence length of the intersection sequence of the second page numbers. This represents the sequence length of the second candidate page number sequence r.

[0052] Based on the intersection density and sequence length of each second candidate page number sequence r, the second candidate page number sequence that meets the following conditions is determined as the page number range of the table of contents of the PDF document. The judgment condition is that the intersection density of the second candidate page number sequence r is greater than a preset threshold and the sequence length is the longest among all second candidate page number sequences r.

[0053] If none of the second candidate page number sequences r meet the requirements, then the first candidate page number sequence will be used as the range of page numbers for the table of contents of the PDF document.

[0054] Understandably, by simultaneously using language-based models (based on text sequence features) and visual models (based on page image features) to initially identify PDF page types, and combining the results of the two through intersection density, the misjudgment of a single model can be avoided, thus improving the accuracy of table of contents page identification. By extracting and filtering continuous page number sequences, it is further ensured that the identified table of contents page number range is continuous and reasonable, conforming to the actual document structure, so as to achieve the correct division between table of contents pages and non-table of contents pages.

[0055] Step S40: Based on the dataset to be processed and the image of each directory page, extract the hierarchical structure relationship of each title data in the directory page and construct a directory tree.

[0056] It should be noted that the node structure of the directory tree corresponds to the hierarchical structure of the title data in the directory page.

[0057] For example, the directory tree adopts a hierarchical tree structure, with the entire PDF document as the root node, and each directory title as a child node attached level by level. The parent-child relationship between nodes corresponds to the hierarchical relationship of the directory titles. Title nodes at the same level exist side by side as sibling nodes, and title nodes at different levels expand level by level according to the hierarchical relationship, thus forming a tree structure that progresses from the main title to chapters, then to subsections, entries, etc., which can fully reflect the hierarchy and organizational logic of the directory in the PDF document.

[0058] In related technologies, directory tree construction is usually carried out according to different levels of title nodes based on features such as indentation and numbering. This embodiment, for more accurate construction, provides a method based on the text and image information of each directory page. In a feasible implementation, step S40 includes steps C10 to C30: Step C10: For each directory page, input the dataset to be processed into the language category directory structure extraction model to obtain the first directory level extraction result of the directory page.

[0059] Step C20: For each table of contents page of a PDF document, convert the table of contents page of the PDF document into an image and input it into the visual directory structure extraction model to obtain the second directory level extraction result of the table of contents page.

[0060] Step C30: Based on the first directory level extraction result and the second directory level extraction result of the directory page, construct the directory tree.

[0061] It should be noted that the language category directory structure extraction model can be a general large language model (LLM), and the visual category directory structure extraction model can be a general visual language model (VLM).

[0062] For example, firstly, it is determined whether the title data in the dataset to be processed has a certain directory rule, such as "Chapter 1", "Section 1", "Module 1", etc. If it exists, the dataset to be processed is segmented according to the directory rule, and then, with the help of prompt words, it is input into the language-based directory structure extraction model to extract the hierarchical structure relationship of each segment. If it does not exist, the dataset to be processed is directly input into the language-based directory structure extraction model with the help of prompt words to extract the first directory level extraction result.

[0063] The table of contents of the PDF document is converted into image pages and fed into a visual directory structure extraction model. With the help of prompt words, the hierarchical structure relationship of each title in the directory is extracted to obtain the second directory level extraction result.

[0064] Finally, based on the first and second directory level extraction results of the directory page, a directory tree is constructed. In this example, the directory tree can be constructed by combining the first and second directory level extraction results with prompt words and inputting them again into the language class large model to achieve the fusion processing of the first and second directory level extraction results, prevent information loss, obtain the final directory hierarchy structure, and construct the directory tree.

[0065] Understandably, the extraction of the table of contents hierarchy in this example relies on both text and visual models, considering not only the hierarchical information of the table of contents page titles but also the visual layout features of the PDF document. By using a dual-modal fusion approach, the error rate of hierarchical extraction in a single mode is reduced, and the resulting directory tree structure is closer to the actual document structure, thus enhancing the accuracy of subsequent content organization.

[0066] Step S50: Based on the semantic and textual similarity between the title data of the table of contents page and the title data of the non-table of contents page, match the title data of the table of contents page and the title data of the non-table of contents page. According to the matching results, fill the corresponding content data under each title node of the table of contents tree to obtain the structured representation result of the PDF document.

[0067] It should be noted that this step matches the table of contents titles with the titles (body titles) in the non-table of contents pages based on the textual and semantic similarity between the non-table of contents page title data and the table of contents page title data, in order to obtain a complete PDF structured representation result. Here, textual similarity refers to the similarity of the title data at the character level, and semantic similarity refers to the similarity of the title data at the semantic level.

[0068] In one feasible implementation, steps D10 to D60 in step S50 are: Step D10: Obtain the title data of all directory pages and the title data of all non-directory pages, and construct the directory title list and the main text title list respectively.

[0069] Step D20: Based on the order of the title data in the table of contents list, traverse the table of contents list forward sequentially. For each table of contents title data traversed in the table of contents list, perform a strong match with the text title data following the last matched text title data in the text title list, until the first text title data that meets the strong match condition is found or no text title data that meets the strong match condition is found. The strong match condition is determined based on the semantic similarity between the table of contents title data and the text title data, the text similarity between the table of contents title data and the text title data, and the strong match similarity threshold.

[0070] Step D30: After traversing all the directory title data, if there is any directory title data that does not match the main text title data, then based on the weak matching condition, perform a weak match between the directory title data that does not match the main text title data and the unmatched main text title data. The weak matching condition is determined based on the semantic similarity between the directory title data and the main text title data, the text similarity between the directory title data and the main text title data, and the weak matching similarity threshold.

[0071] Step D30 also includes step D31: If there are table of contents titles that do not match the main text titles, then the upper bound is the main text title data matched by the closest matching table of contents title data before the table of contents title data that does not match the main text titles, and the lower bound is the main text title data matched by the closest matching table of contents title data after the table of contents title data that does not match the main text titles. Weak matching is then performed between the table of contents titles that do not match the main text titles and the main text title data between the upper and lower bounds.

[0072] Step D40: Obtain the title data matching table. Each table of contents title data value corresponds to at most one body text title data value, and each body text title data value appears only once.

[0073] Specifically, to achieve accurate alignment between table of contents titles and main text titles, this implementation combines semantic similarity and textual similarity, employing a dual scoring mechanism supplemented by a strong-weak matching hierarchical strategy. Furthermore, it introduces forward-only constraints on the matching path to ensure the stability and semantic consistency of the matching results. This algorithm is suitable for parsing multi-level, structurally rigorous document structures, and is particularly well-suited for table of contents-driven main text structure reconstruction scenarios.

[0074] For ease of understanding, for example, we first construct a list of catalog titles (catalog_titles) and a list of content titles (content_titles).

[0075] Based on the order of the table of contents headings in the table of contents list, all headings are traversed sequentially in a forward-only manner. For each heading, a strong match is performed against each subsequent heading in the main text, until the first matching heading is found or no match is found. It's understood that to ensure consistency, the order of the headings matches the order of the headings within the table of contents, and the matching order with the main text headings should match the order of the headings within the main text.

[0076] In this example, the following steps can be used to determine whether a strong match is satisfied between the table of contents heading data and the body heading data: Determine the textual and semantic similarity between the currently traversed directory title data and the text title data to be matched. Based on the textual and semantic similarity, determine a fused similarity. If the fused similarity is not less than a preset similarity threshold, then the currently traversed directory title data and the text title data to be matched satisfy the strong matching condition.

[0077] Semantic similarity was determined using the cosine similarity algorithm, while textual similarity was determined using the text ranking and comparison algorithm.

[0078] Specifically, an embedding model can be used to represent each title text in the list, obtaining the semantic information of the title text. In this example, the embedding model uses the gte-qwen model: Semantic similarity can be calculated in the following ways: in, This represents the textual semantic information of the i-th directory title data that has been traversed. This represents the textual semantic information of the j-th title in the list of title data.

[0079] Text similarity can be calculated in the following ways: For the traversed table of contents title data and the body title data that is matching the table of contents title data, remove the whitespace characters in the table of contents title data and the body title data.

[0080] The edit score between the table of contents title data and the body title data is determined based on the minimum number of edit operations required.

[0081] The edit scores are normalized to obtain the text similarity between the table of contents title data and the main text title data.

[0082] For ease of understanding, this calculation method can be expressed as: in, This represents the table of contents headings after spaces have been removed. This represents the text title data after removing spaces. () represents the function for calculating the minimum number of edit operations.

[0083] Understandably, removing leading and trailing spaces from the table of contents and body headings can prevent a decrease in matching accuracy due to formatting issues. The minimum number of edit operations represents the minimum number of edit operations (insertion, deletion, replacement) required to convert all characters in the table of contents headings into all characters in the body headings. Normalizing this minimum number of operations to the 0–1 range yields the text similarity score. Text similarity and semantic similarity are combined to form a weighted fusion score, fusion_score. The fusion score is calculated as follows: The weighting factor alpha represents the weighting factor for semantic similarity, and beta represents the weighting factor for text similarity.

[0084] If the fusion score (fusion_score) is not less than the set similarity threshold (similarity_threshold), then the two titles (e.g., the i-th table of contents title data and the j-th body title data) are considered to meet the strong matching condition, and they are added to the title data matching table (alignment).

[0085] i The index of the i-th directory title data being traversed is the index of the directory title data, and j represents the matching index of the text title data that matches the i-th directory title data.

[0086] Furthermore, to facilitate finding table of contents title data that does not match the main text title data, if the current table of contents title data does not match the main text title data, then... The matching index for the table's directory title data is -1: .

[0087] If the j-th title in the title list is the most recently matched title, then the search for a title that matches the next title in the table of contents list begins from the (j+1)-th title in the title list. In this example, this can be achieved by setting a pointer variable `content_idx` that points to the first title allowed to match the current title. If the current title is matched, the pointer is set to the next title matching the current title, ensuring that the next match is only performed with subsequent titles.

[0088] After the traversal is complete, search for the table of contents title data that did not match the main text title data (i.e., The table contains table of contents headings with index -1. If such headings exist, then based on weak matching conditions, the table of contents headings that did not match the main text headings are weakly matched with the unmatched main text headings. In this example, the weak matching method is as follows: For weak matching, based on the fact that the strong matching has already traversed the list of directory titles, the indexes of unmatched directory titles are determined from the preliminary title data matching table alignment.

[0089] Based on the table of contents index, unmatched table of contents titles are identified. For each unmatched title, its matching range is determined. This range is calculated from the text titles matched by the previous matched title (as the upper bound `prev_match`) and the text titles matched by the next matched title (as the lower bound `next_match`). These upper and lower bounds limit the range of text titles that can be matched during the weak matching phase (i.e., only text titles that satisfy the weak matching condition can be found between the text titles matched by the previous and next matched titles).

[0090] Within the established upper and lower bounds, the semantic and textual similarities between unmatched table of contents titles and body titles are recalculated. A more lenient matching attempt is made by adjusting the similarity threshold (reducing the similarity_threshold). This strategy allows for attempts to find suitable matches with lower similarity, ensuring that already paired body titles are not repeatedly matched during weak matching, thus avoiding false matches. Based on strong and weak matching, each table of contents title data value in the final alignment table corresponds to at most one body title data value, and each body title data value appears only once.

[0091] Step D50: Determine the location information of each text title data in the dataset to be processed in the title data matching table.

[0092] Step D60: Based on the location information and the matching relationship between the main text title data and the table of contents title data, the content data is filled into each title node of the table of contents tree to obtain the structured representation result of the PDF document.

[0093] This implementation achieves the final structured representation of the PDF document through steps D50 to D60, forming a nested structure with the PDF document as the root node and each chapter as a child node. Specifically, the alignment table reflects the matching relationship between the table of contents titles and the main text titles. Based on each matching relationship, the main text titles and their corresponding content data can be backfilled into the table of contents title nodes in the table of contents tree structure.

[0094] For example, the content data corresponding to the body title can be determined in the following way: For each dataset to be processed, based on type label "Determine the position of the header items in the dataset to be processed. Classify the content data (excluding type markers) between two adjacent header items under the first header item. For two adjacent datasets to be processed, if there is content data before the first header item in the latter dataset, classify that content data under the last header item of the former dataset. For ease of understanding, the dataset classified according to header items can be represented as:" The `title` field represents the title item; `datalist` represents the content data under that title; and `page_idx` represents the page number range of the title item and its corresponding content data in the PDF document.

[0095] In this example, according to The structured representation of a PDF document can be determined in the following ways: from The code reads the main text title items and removes spaces and whitespace characters to ensure consistency and comparability. It then performs a title section range mapping generation step, using the start and end page numbers of each main text title item as the key (start, end) and the corresponding main text title text as the value, and writes this into the section range map (section_map). Extract the correspondence between the main text title items and page ranges (start page, end page) from the section_map, and establish an inverted index from the title to multiple page ranges to support the situation where the same title appears multiple times in the document.

[0096] Set a heading counter to record the number of times the same heading has been assigned, and match it with the table of contents headings in ascending order of page number to avoid duplicate matching of the same body heading.

[0097] The output is a nested JSON structure with the PDF document as the root node and each chapter as a child node, preserving the original semantic hierarchy of the document, which facilitates subsequent retrieval, display and knowledge extraction.

[0098] Understandably, this embodiment, by introducing a dual recognition mechanism of language and visual models, can not only accurately extract the table of contents and title information from the text feature dimension of PDF documents, but also combine visual features such as layout, font style, and positional relationships to achieve precise differentiation between the table of contents and the main text pages. This avoids table of contents recognition errors caused by complex document layout or OCR recognition errors, thus improving recognition accuracy. Secondly, in the matching stage between the table of contents and the main text, a dual calculation strategy of semantic similarity and text similarity is adopted, combined with sequential forward traversal and strong and weak matching methods, so that the table of contents titles can achieve a high-precision correspondence with the main text titles. This effectively overcomes the problems of mismatch and omission that are prone to occur in traditional page number-based or rule-based matching methods, ensuring the integrity and correctness of the document's hierarchical relationship. Thirdly, the main text content is filled into the constructed table of contents tree nodes according to the matching relationship, so that the document can achieve a semantically complete and hierarchically clear structured representation, making the chapter division more accurate and improving the efficiency and reliability of subsequent content retrieval, knowledge extraction, and semantic analysis. In summary, this embodiment provides a structured processing solution that balances accuracy and practicality when dealing with diverse and complex PDF documents, fundamentally solving the shortcomings of existing technologies in document semantic restoration, chapter organization, and knowledge utilization.

[0099] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the PDF document content processing method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0100] This application also provides a PDF document content processing device; please refer to... Figure 2 The PDF document content processing device includes: Module 10 is used to acquire a PDF document, which contains at least one page; The preprocessing module 20 is used to preprocess the PDF document to obtain a dataset to be processed corresponding to each page of the PDF document; the dataset to be processed includes title data and / or content data. The classification module 30 is used to determine the page type of each page in the PDF document based on all the datasets to be processed and the images of each page of the PDF document; the page type includes table of contents pages and / or non-table of contents pages; The directory tree construction module 40 is used to extract the hierarchical structure relationship of each title data in the directory page based on the dataset to be processed and the image of the directory page, and to construct a directory tree. The node structure of the directory tree corresponds to the hierarchical structure of the title data in the directory page. The matching module 50 is used to match the title data of the table of contents page and the title data of the non-table of contents page based on the semantic similarity and textual similarity between the title data of the table of contents page and the title data of the non-table of contents page. According to the matching results, the content data is filled into each title node of the table of contents tree to obtain the structured representation result of the PDF document.

[0101] The PDF document content processing apparatus provided in this application, employing the PDF document content processing method in the above embodiments, can solve the technical problem of inaccurate chapter division in related technologies regarding PDF document content processing. Compared with related technologies, the beneficial effects of the PDF document content processing apparatus provided in this application are the same as those of the PDF document content processing method provided in the above embodiments, and other technical features in the PDF document content processing apparatus are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0102] This application provides a PDF document content processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the PDF document content processing method in the above embodiments.

[0103] The following is for reference. Figure 3This document illustrates a structural diagram of a PDF document content processing device suitable for implementing embodiments of this application. The PDF document content processing device in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 3 The PDF document content processing device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0104] like Figure 3 As shown, the PDF document content processing device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the PDF document content processing device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the PDF document content processing device to communicate wirelessly or wiredly with other devices to exchange data. Although various systems are shown in the figure, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0105] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0106] The PDF document content processing device provided in this application, employing the PDF document content processing method in the above embodiments, can solve the technical problem of inaccurate chapter division in PDF document content processing in related technologies. Compared with related technologies, the beneficial effects of the PDF document content processing device provided in this application are the same as those of the PDF document content processing method provided in the above embodiments, and other technical features in this PDF document content processing device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0107] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0108] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0109] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the PDF document content processing method in the above embodiments.

[0110] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0111] The aforementioned computer-readable storage medium may be included in a PDF document content processing device; or it may exist independently and not be assembled into a PDF document content processing device.

[0112] The aforementioned computer-readable storage medium carries one or more programs that, when executed by a PDF document content processing device, cause the PDF document content processing device to: acquire a PDF document containing at least one page; preprocess the PDF document to obtain a dataset to be processed corresponding to each page of the PDF document; the dataset to be processed includes title data and / or content data; determine the page type of each page of the PDF document based on all datasets to be processed and images of each page of the PDF document; the page type includes a table of contents page and / or a non-table of contents page; extract the hierarchical structure relationship of each title data in the table of contents page based on the dataset to be processed corresponding to each table of contents page and the image of the table of contents page, construct a table of contents tree, the node structure of the table of contents tree corresponding to the hierarchical structure of the title data in the table of contents page; match the title data of the table of contents page and the title data of the non-table of contents page based on the semantic similarity and textual similarity between the title data of the table of contents page and the title data of the non-table of contents page, and, according to the matching results, fill the corresponding content data under each title node of the table of contents tree to obtain a structured representation of the PDF document.

[0113] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0115] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0116] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described PDF document content processing method. This solves the technical problem of inaccurate chapter division in related technologies regarding PDF document content processing. Compared with related technologies, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the PDF document content processing method provided in the above embodiments, and will not be elaborated upon here.

[0117] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the PDF document content processing method described above.

[0118] The computer program product provided in this application can solve the technical problem of inaccurate chapter division in PDF document content processing of related technologies. Compared with related technologies, the beneficial effects of the computer program product provided in this application are the same as those of the PDF document content processing method provided in the above embodiments, and will not be repeated here.

[0119] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for processing PDF document content, characterized in that, The method includes: Obtain a PDF document containing at least one page; The PDF document is preprocessed to obtain a dataset to be processed corresponding to each page of the PDF document; the dataset to be processed includes title data and / or content data. Based on all the datasets to be processed and the images of each page of the PDF document, determine the page type of each page of the PDF document; the page type includes table of contents pages and / or non-table of contents pages; Based on the dataset to be processed and the image of each directory page, the hierarchical structure relationship of each title data in the directory page is extracted, and a directory tree is constructed. The node structure of the directory tree corresponds to the hierarchical structure of the title data in the directory page. Based on the semantic and textual similarity between the title data of the directory pages and the title data of the non-directory pages, the title data of the directory pages and the title data of the non-directory pages are matched. According to the matching results, the content data is filled into each title node of the directory tree to obtain the structured representation result of the PDF document.

2. The method as described in claim 1, characterized in that, The step of determining the page type of each page of the PDF document based on all the datasets to be processed and the images of the PDF document pages includes: The datasets to be processed are sorted by page number. For each dataset, the dataset to be processed and the two datasets before and after it are taken as a group of data to be identified and input into a language page recognition model to obtain the first page type of the current page in the group of data to be identified. The dataset to be processed is marked as the current page, the dataset before it is marked as the previous page, and the dataset after it is marked as the next page. After converting all pages of the PDF document into images, they are input into a visual page recognition model to obtain the second page type of all the images to be recognized. Based on the first page type and the second page type, the page type of each page in the PDF document is determined.

3. The method as described in claim 2, characterized in that, The step of determining the page type of each page in the PDF document based on the first page type and the second page type includes: The first directory page number sequence is determined based on the first page type, and the second directory page number sequence is determined based on the second page type. Take the union of the first directory page number sequence and the second directory page number sequence, and take the longest consecutive page number sequence in the union as the first candidate page number sequence; The intersection of the first directory page number sequence and the second directory page number sequence is taken to obtain a first page number intersection sequence. Continuous interval extraction is performed on the first page number intersection sequence to obtain at least one second candidate page number sequence. For each second candidate page number sequence, the intersection of the second candidate page number sequence and the first page number intersection sequence is taken to obtain the second page number intersection sequence; Select a second candidate page number sequence that meets the preset requirements from all the second candidate page number sequences as the table of contents page number range of the PDF document; the intersection density is determined based on the ratio of the sequence length of the second page number intersection sequence to the sequence length of the second candidate page number sequence; the preset requirements include an intersection density greater than a preset threshold and a sequence length that is the maximum value among all the second candidate page number sequences; If no second candidate page number sequence meets the preset requirements, then the first candidate page number sequence is taken as the range of page numbers for the table of contents of the PDF document.

4. The method as described in claim 1, characterized in that, The step of extracting the hierarchical structure relationship of each title data in the directory page and constructing a directory tree based on the dataset to be processed and the image of the directory page corresponding to each directory page includes: For each directory page, the dataset to be processed is input into the language-based directory structure extraction model to obtain the first directory level extraction result of the directory page. For each table of contents page of a PDF document, the table of contents page is converted into an image and input into a visual directory structure extraction model to obtain the second directory level extraction result of the table of contents page; The directory tree is constructed based on the extraction results of the first and second directory levels of the directory pages.

5. The method as described in claim 1, characterized in that, The step of matching the title data of the directory page and the title data of the non-directory page based on the semantic and textual similarity between the title data of the directory page and the title data of the non-directory page includes: Obtain the title data of all the directory pages and the title data of all the non-directory pages, and construct a directory title list and a body title list respectively; Based on the order of the title data in the directory title list, the directory title list is traversed sequentially forward. For each directory title data traversed in the directory title list, a strong match is performed on the subsequent text title data after the last matched text title data in the text title list, until the first text title data that meets the strong match condition is found or no text title data that meets the strong match condition is found. The strong match condition is determined based on the semantic similarity between the directory title data and the text title data, the text similarity between the directory title data and the text title data, and a strong match similarity threshold. After traversing all the directory title data, if there is any directory title data that does not match the main text title data, then based on the weak matching condition, the directory title data that does not match the main text title data is weakly matched with the unmatched main text title data; the weak matching condition is determined based on the semantic similarity between the directory title data and the main text title data, the text similarity between the directory title data and the main text title data, and the weak matching similarity threshold. Obtain a title data matching table; each directory title data in the title data matching table corresponds to at most one body title data and each body title data appears only once.

6. The method as described in claim 5, characterized in that, The step of filling the content data into each title node of the directory tree according to the matching results to obtain the structured representation of the PDF document includes: Determine the position information of each text title data in the title data matching table within the dataset to be processed; Based on the location information and the matching relationship between the main text title data and the table of contents title data, the content data is filled into each title node of the table of contents tree to obtain the structured representation result of the PDF document.

7. A PDF document content processing device, characterized in that, The device includes: An acquisition module is used to acquire a PDF document, wherein the PDF document contains at least one page; A preprocessing module is used to preprocess the PDF document to obtain a dataset to be processed corresponding to each page of the PDF document; the dataset to be processed includes title data and / or content data. A classification module is used to determine the page type of each page in the PDF document based on all the datasets to be processed and images of each page of the PDF document; the page type includes table of contents pages and / or non-table of contents pages; The directory tree construction module is used to extract the hierarchical structure relationship of each title data in the directory page based on the dataset to be processed and the image of the directory page, and construct a directory tree, wherein the node structure of the directory tree corresponds to the hierarchical structure of the title data in the directory page; The matching module is used to match the title data of the directory pages and the title data of the non-directory pages based on the semantic similarity and textual similarity between them. According to the matching results, the content data is filled into each title node of the directory tree to obtain the structured representation of the PDF document.

8. A PDF document content processing device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the PDF document content processing method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the PDF document content processing method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the PDF document content processing method as described in any one of claims 1 to 6.

Images