An AI-driven method for accurately locating content in a PDF document

Abstract

The application discloses an AI-driven PDF document content accurate positioning method, and relates to the technical field of intelligent document processing, which comprises the following steps: analyzing the page text, page image, boundary box coordinate and target content of a PDF document to be positioned, constructing a hierarchical document structure tree, associating the reading order and hierarchical affiliation to the corresponding boundary box coordinate, and generating document analysis data; processing the document analysis data by using image recognition branches and semantic understanding branches, screening candidate texts by combining cross-modal alignment and layout anchor point analysis, and generating candidate positioning results; determining the page code local area based on the candidate positioning results, page image and hierarchical document structure tree, and performing boundary box tokenization verification, compatible search and geometric splicing to generate target boundary boxes. Through front-end coordinate conversion and text space verification, the application reduces the highlight deviation caused by the inconsistency between the PDF coordinate and the front-end pixel coordinate.

Description

Technical Field

[0001] This invention relates to the field of intelligent document processing technology, and in particular to an AI-driven method for accurate content location in PDF documents. Background Technology

[0002] With the development of digital office work, electronic contract review, financial document archiving, technical report retrieval, and intelligent archive management, PDF documents have become an important medium for cross-platform information exchange and long-term storage. Current methods for locating PDF content typically rely on text parsing tools, keyword search tools, OCR recognition tools, and front-end PDF rendering tools. These tools can extract character content from text-based PDFs and, based on keyword matching results, redirect or highlight elements on the page. For scanned PDFs, image-embedded PDFs, and multi-column mixed-layout PDFs, related technologies are increasingly employing image text recognition, layout analysis, and semantic retrieval to assist in document content acquisition, enabling PDF document processing to evolve from simple text retrieval to hybrid text-image parsing and intelligent retrieval.

[0003] However, existing PDF content location technologies are still mainly limited to "text matching" or "image text recognition," lacking stable structural constraints between target content and page location. For complex documents such as contracts, reports, and invoices, pages often contain titles, body text, tables, image descriptions, headers and footers, and multi-column content simultaneously. Traditional keyword retrieval is easily affected by full-width and half-width characters, special spaces, punctuation encoding, line breaks, and line splits. While ordinary OCR recognition can extract text from scanned documents, the recognition results often lack consistency verification with PDF page coordinates, layout hierarchy, and front-end display area. This can easily lead to problems such as correct candidate text but misaligned highlighted areas, incomplete location areas, or locating the wrong page object, making it difficult to meet the application requirements for accurate content location in complex PDF documents. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides an AI-driven method for accurate content positioning in PDF documents, which solves the problem that target content in complex PDF documents is difficult to accurately correspond to the page layout structure and display coordinates.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides an AI-driven method for precise content location in PDF documents, comprising: The process involves parsing the page text, page images, bounding box coordinates, and target content of the PDF document to be located, constructing a hierarchical document structure tree, and associating the reading order and hierarchical relationship with the corresponding bounding box coordinates to generate document parsing data. The document parsing data is processed using image recognition and semantic understanding branches, and candidate texts are filtered by combining cross-modal alignment and page anchor point parsing to generate candidate positioning results; Based on the candidate localization results, page images, and hierarchical document structure tree, the local area of ​​the page number is determined, and bounding box lexical verification, compatibility search, and geometric stitching are performed to generate the target bounding box; Based on the target bounding box, page size, and layout structure, perform front-end coordinate transformation and text space verification to generate positioning and rendering data; By positioning and presenting data, the target pixel area is marked and centered on the PDF rendering page, generating the PDF document content positioning result.

[0007] As a preferred embodiment of the AI-driven PDF document content accurate positioning method of the present invention, the construction of the hierarchical document structure tree specifically includes: Based on the line position of the page text, the text area boundary and bounding box coordinates in the page image, and combined with the font size, font attributes, numbering format, text alignment, wireframe cells, text baseline alignment, image caption number and page edge position, the page object type is identified and the page object is divided into categories, resulting in a set of page object coordinates. The reading order of page objects is determined by their vertical and horizontal positions, multi-column division results, and bounding box adjacency relationships within the page object coordinate set, thus obtaining the reading order chain. The hierarchy is determined by identifying the subordinate relationships between headings, body text, tables, image descriptions, and headers and footers based on the reading order chain, page object type, and bounding box adjacency. Establish the same-page correspondence between the reading order chain, hierarchical subordinate relationship, and page object coordinate set, and write the page object coordinate, reading order position, hierarchical subordinate position, and candidate text box set into the corresponding tree node to obtain the hierarchical document structure tree.

[0008] As a preferred embodiment of the AI-driven PDF document content accurate positioning method of the present invention, the generation of document parsing data specifically includes: Extract page text, page image, and bounding box coordinates from the PDF document to be located. Perform character recognition on the text region boundary in the page image to obtain OCR text and OCR word box. Then, perform character normalization on the target content to obtain normalized target content. The hierarchical document structure tree is used to mark the reading order and hierarchical subordinate position of the page text and OCR text, resulting in structured page text; By establishing a correspondence between structured page text, page images, bounding box coordinates, OCR word boxes, and normalized target content according to page numbers, document parsing data is obtained.

[0009] As a preferred embodiment of the AI-driven PDF document content accurate positioning method of the present invention, the step of processing document parsing data using image recognition and semantic understanding branches specifically includes: By detecting page images in document parsing data through the image recognition branch, locating the text region boundaries in the page images, performing OCR recognition on the text within the text region boundaries, and mapping the OCR word boxes from page image coordinates to PDF page coordinates, and mapping the text region boundaries to the bounding box coordinates of structured page text, image candidate fragments are obtained. By comparing the normalized target content, structured page text, and OCR text through the semantic understanding branch, and retaining the reading order position and hierarchical subordinate position of the candidate text, semantic candidate fragments are obtained. Using page number, bounding box coordinates, reading order position, and hierarchical subordinate position as common indices uniformly generated by the hierarchical document structure tree, a two-branch correspondence is performed on image candidate fragments and semantic candidate fragments to obtain branch candidate results; Based on the branch candidate results, candidate text sources in text-based PDFs, scanned PDFs, image-embedded PDFs, and complex layout PDFs are labeled to obtain the branch candidate source set.

[0010] As a preferred embodiment of the AI-driven PDF document content accurate positioning method of the present invention, the generation of candidate positioning results specifically includes: Cross-modal alignment is performed on candidate text, page image blocks, and bounding box coordinates in the branch candidate source set. The image pixel coordinates of the page image blocks are mapped to PDF page coordinates. Coordinate alignment, text alignment, and object alignment are performed on the candidate text, page image blocks, and bounding box coordinates to obtain the text-image alignment result. Using page objects in the hierarchical document structure tree as page anchors, the text-image alignment results are verified for page position to obtain page anchor matching results. By using the anchor point matching results, candidate texts that match the normalized target content but do not match the reading order, page object position, or hierarchical subordinate position are excluded, thus obtaining candidate positioning results.

[0011] As a preferred embodiment of the AI-driven PDF document content accurate positioning method of the present invention, wherein: determining the local area of ​​page numbers specifically includes: Extract the candidate text, page number, preliminary bounding box, page anchor point matching result, reading order position, and hierarchical subordinate position from the candidate positioning results to obtain the candidate positioning index; Based on the candidate positioning index, a page visual multi-vector retrieval and late interaction matching mechanism is used to limit the range of page objects where candidate text is located in the page image to obtain the candidate page region. By verifying the reading order and hierarchical position of candidate page areas using the hierarchical document structure tree, a local area of ​​page number is obtained.

[0012] As a preferred embodiment of the AI-driven PDF document content accurate positioning method of the present invention, the bounding box word element verification specifically includes: Extract candidate text, preliminary bounding boxes, page anchor point matching results, and hierarchical subordinate positions from local areas of page numbers. Decompose the candidate text into candidate words and bind the candidate words to PDF character boxes, OCR word boxes, or word bounding boxes allocated according to character width ratio to construct candidate box word sequence. The normalized target content is split into a target word sequence, and the target word sequence is semantically mapped to the candidate box word sequence to obtain the target candidate mapping chain; Based on the textual meaning of the candidate texts compared with the target candidate corresponding chain, the page position of the preliminary bounding box, the page object position and hierarchical subordinate position of the page anchor point matching result, the semantic space consistency result is obtained; When the semantic space consistency result shows that the candidate text is consistent with the meaning of the normalized target content and the initial bounding box falls within the range of the corresponding page object, the corresponding candidate box word is retained to obtain the effective candidate box word. When the semantic space consistency result shows that the candidate text has a semantic correspondence with the normalized target content and the preliminary bounding box does not fall within the scope of the corresponding page object, the corresponding candidate box word is removed to obtain invalid candidate box word. The valid candidate box terms are rearranged according to their reading order in the hierarchical document structure tree to obtain the verification bounding box sequence.

[0013] As a preferred embodiment of the AI-driven PDF document content accurate positioning method of the present invention, the compatible search and geometric stitching specifically includes: Text compatibility search is performed using normalized target content and verification bounding box sequences. Differences in full-width and half-width characters, special spaces, punctuation encoding, and line breaks are handled to obtain compatibility hit results. When the compatible hit result covers the complete normalized target content, extract the bounding box corresponding to the compatible hit result to obtain the complete hit box; When the compatibility hit result does not cover the complete normalized target content, adjacent candidate box words are selected according to the reading order to obtain the group of boxes to be spliced. Based on the hierarchical subordinate position, reading order chain, and page object coordinate set, the groups of boxes to be spliced ​​are spliced ​​to obtain spliced ​​bounding boxes. A set of bounding boxes is formed by merging within the same line, splitting across lines, splitting across segments, and retaining cross-page objects separately. Duplicate areas, adjacent overlapping areas, and invalid areas in the complete hit box and spliced ​​bounding boxes are removed to obtain the target bounding box.

[0014] As a preferred embodiment of the AI-driven PDF document content accurate positioning method described in this invention, the generation of positioning presentation data specifically includes: Extract the page size, page orientation, and PDF coordinate position of the page number corresponding to the target bounding box, and extract the visible page boundary, page rotation angle, PDF rendering scaling ratio, device pixel ratio, page scroll container offset, page spacing, and PDF rendering viewport transformation matrix to obtain the page coordinate reference. Based on the page coordinate reference, the vertical direction and display unit of the target bounding box are transformed, and the corner points of the target bounding box are transformed from PDF page coordinates to PDF rendering viewport coordinates. Combined with the device pixel ratio, page scroll container offset and page spacing, the front pixel area is obtained. The candidate text, layout anchor point matching results, and hierarchical subordinate positions corresponding to the front pixel region and the target bounding box are verified in text space to obtain pixel verification results. Based on the pixel verification results, the front-end pixel areas that are consistent with the text meaning, page object position, and hierarchical subordinate position corresponding to the target bounding box are retained. Front-end pixel areas that deviate from the page object range, hierarchical subordinate position, or candidate text corresponding area are eliminated and re-transformed. If the re-transformation still fails the text space verification, the verification bounding box sequence is returned to redetermine the target bounding box. The position of the highlight rectangle and the page scroll position are determined by the front-end pixel areas that are consistent with the verification, and the positioning and presentation data are obtained.

[0015] As a preferred embodiment of the AI-driven PDF document content accurate positioning method of the present invention, the generation of PDF document content positioning results specifically includes: Based on the positioning data, the target pixel area is determined in the PDF rendering page to obtain the target display area; Draw highlight marks in the target display area, and adjust the display position of the PDF rendering page according to the page scroll position to obtain the target centered page; When new target content is received, the document parsing data, candidate location results, verification bounding box sequence, target bounding box, and location presentation data are called to redetermine the target display area. The highlighted range is determined according to the PDF original text bounding box corresponding to keywords, sentence fragments, field names, or field values ​​to obtain the PDF document content location result.

[0016] The beneficial effects of this invention are as follows: By uniformly incorporating page text, page images, bounding box coordinates, and target content into document parsing data, and saving the reading order and hierarchical relationship through a hierarchical document structure tree, the location of target content no longer depends on the result of a single keyword hit, but is simultaneously constrained by page objects, layout structure, and coordinate range; by processing scanned documents, image-embedded PDFs, and text-based PDFs through image recognition and semantic understanding branches respectively, the coverage of candidate texts for different types of PDF pages can be improved; by filtering candidate texts with matching text content but inconsistent reading order, layout object position, or hierarchical position through cross-modal alignment and layout anchor point parsing, the probability of mislocation under complex layouts can be reduced; by generating target bounding boxes through bounding box lexical verification, compatible search, and geometric splicing, the problem of incomplete positioning caused by full-width and half-width characters, special spaces, line breaks, and cross-text block splitting can be handled; by generating positioning presentation data through front-end coordinate transformation and text space verification, the highlight offset caused by inconsistency between PDF coordinates and front-end pixel coordinates can be reduced. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart for an AI-driven method for precise content location in PDF documents.

[0019] Figure 2 This diagram illustrates the construction of a hierarchical document structure tree and the generation of document parsing data.

[0020] Figure 3 This diagram illustrates the dual-branch candidate text filtering and layout anchor point matching.

[0021] Figure 4 A schematic diagram showing the verification, splicing, and front-end positioning of the target bounding box. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 As one embodiment of the present invention, this embodiment provides an AI-driven method for accurately locating PDF document content, including the following steps: S1. Parse the page text, page images, bounding box coordinates, and target content of the PDF document to be located, construct a hierarchical document structure tree, and associate the reading order and hierarchical relationship with the corresponding bounding box coordinates to generate document parsing data.

[0026] S1.1. Receive the PDF document to be located and the target content input by the user. The PDF document to be located comes from PDF files in office documents, contract documents, report documents, and invoice documents. The target content input by the user comes from keywords, phrases, sentence fragments, and field names entered in the front-end search box. First, parse the document attributes and page content of the PDF document to be located, and read the MediaBox (the media box of the PDF page, used to represent the complete physical page boundary of the PDF page in the file), CropBox (the cropping box of the PDF page, used to represent the actual rendering and display area of ​​the PDF page), page rotation angle, page width, page height, number of character streams, number of embedded images, and page rendered image size of each page. Then, read the image object identifier, image pixel width, image pixel height, current image transformation matrix of the image in the PDF content stream, and image placement rectangle corresponding to the embedded image. The current image transformation matrix is ​​used to represent the affine mapping relationship between the local coordinates of the embedded image and the coordinates of the PDF page. The image placement rectangle is used to represent the left and top edges of the embedded image in the PDF page. The system identifies the right and bottom boundaries of an embedded image. When the current image transformation matrix corresponding to the embedded image can be directly read, it serves as the basis for converting the local coordinates of the embedded image to the coordinates of the PDF page. When only the image placement rectangle can be read, a proportional mapping relationship is established between the image pixel width, image pixel height, and the image placement rectangle. The horizontal and vertical pixel positions in the local coordinates of the embedded image are mapped to the PDF page coordinates within the image placement rectangle. When neither the current image transformation matrix nor the image placement rectangle can establish a corresponding relationship, the corresponding OCR word box is not included in the candidate text and is only retained as the image region boundary to avoid OCR results without a coordinate basis from participating in target localization. Pages where character streams can be directly extracted are identified as text-type PDF pages. Pages where only page images can be obtained and effective character streams cannot be directly extracted are identified as scanned PDF pages. Pages containing both character streams and embedded image text are identified as image-embedded PDF pages. Pages with multiple columns, tables, titles, image descriptions, and headers / footers are identified as complex layout PDF pages. This yields PDF page type data.

[0027] Based on the PDF page type, different pages of the PDF document to be located are processed. For text-based PDF pages, the page text, line position, character position, font size, font bold attribute, character spacing, line height, paragraph indentation, text alignment, and page coordinates are extracted to obtain the page text data. For scanned PDF pages and image-embedded PDF pages, the page image is denoised, grayscaled, and contrast-enhanced. The text region boundaries in the page image are extracted, and OCR (Optical Character Recognition) is performed on the text within the text region boundaries to obtain OCR text, OCR character boxes, OCR word boxes, and OCR confidence scores. For embedded image text in image-embedded PDF pages, the OCR word boxes are first recorded as word box coordinates in the local coordinates of the embedded image. Then, based on the current image transformation matrix or image placement rectangle corresponding to the embedded image, the OCR word boxes are mapped from the local coordinates of the embedded image to the placement area of ​​the embedded image in the PDF page and converted into OCR bounding boxes in the PDF page coordinates. For scanned PDF pages or images that are rendered as full-page images, OCR is performed. For embedded PDF pages, OCR word boxes are recorded as word box coordinates in the full-page rendered image coordinates. Then, based on the full-page rendering viewport transformation matrix, CropBox, and page rotation angle, a reverse mapping relationship is established from the full-page rendered image coordinates to the PDF page coordinates, converting the OCR word boxes into OCR bounding boxes in the PDF page coordinates. The full-page rendering viewport transformation matrix is ​​the affine transformation matrix used when generating the page rendered image from the PDF page coordinates. Its reverse mapping relationship is used to restore the pixel positions in the page rendered image to the PDF page coordinates. Therefore, each OCR word box entering the candidate text has both OCR local coordinates and PDF page coordinates, which can establish a same-page correspondence with the page object coordinate set, reading order chain, and hierarchical subordination relationship. Text areas with an OCR confidence score below 0.80 are not considered as candidate text, but are only retained as image area boundaries to obtain page image data. For complex layout PDF pages, both page text data and page image data are retained, and the bounding box coordinates of titles, body text, tables, image descriptions, and headers and footers in the page are recorded to obtain complex page coordinate data.

[0028] S1.2. Perform character normalization processing on the target content input by the user, converting full-width characters to half-width characters, special spaces to ordinary spaces, punctuation codes to unified punctuation forms, and restoring line breaks and hyphens to continuous text, to obtain normalized target content; character normalization processing only changes the retrieval expression form of the target content, without changing the textual meaning of the target content, and the normalized target content is used for subsequent candidate text filtering and compatible search.

[0029] Dividing page objects based on page text data, page image data and complex page coordinate data; the page objects refer to the text area, title area, body text area, table area, image caption area, and header and footer area that have independent bounding box coordinates and layout functions in a PDF page.

[0030] When dividing page objects, the median font size, median line height, median character width of the body text in the same page, and the page size are first used as determination criteria; the title area is identified according to at least two of the following features: the font size is not less than 1.15 times the median font size of the body text, has a bold attribute, is located before the body text area, has a numbering format, or is center-aligned; the body text area is identified according to continuous text lines, the horizontal projection overlap ratio of adjacent lines is not less than 0.50, and the vertical spacing between adjacent lines is not greater than 1.5 times the median line height; the table area is identified according to closed cells formed by visible wireframes, the number of intersections of horizontal and vertical lines is not less than 4, or the text baselines in the same area form an alignment structure of no less than 2 columns and no less than 2 rows; the image caption area is identified according to that its text starts with "Fig.", "Table", "Figure", "Table" or the corresponding numbering format, and the minimum boundary distance from the adjacent image area or table area is not greater than 1.2 times the median line height; the header and footer area is identified according to that its bounding box is located within the top 8% or the bottom 8% of the page height, and is not continuous with the body text area in the reading order.

[0031] The line position of the page text is determined by the outer boundary of the character positions in the same text line; when there is an OCR word frame within the outer contour of the text, the outer rectangle of the OCR word frame is taken as the range of the image text area; when there is no valid OCR word frame within the outer contour of the text, the outer rectangle of the text outer contour is taken as the range of the image text area; the bounding box coordinates determine the position range of the page object on the PDF page, and the position range of the page object is the respective covered rectangular coordinate range of the title, body text, table, image caption, and header and footer; the text area range, image text area range and page object position range are all located within the page size range of the corresponding page number, to obtain a page object coordinate set.

[0032] The reading order of page objects is determined based on their vertical and horizontal positions and bounding box adjacency in the page object coordinate set. Vertical position determines the vertical order of page objects within the page, horizontal position determines the horizontal order within the same line, and bounding box adjacency determines whether adjacent page objects are continuous reading content. Bounding box adjacency is determined by the minimum vertical spacing, minimum horizontal spacing, and projection overlap ratio of two page objects on the same page. When the vertical spacing between two page objects is no greater than 1.5 times the median line height and the horizontal projection overlap ratio is no less than 0.30, they are considered vertically continuous. When the horizontal spacing between two page objects is no greater than 2 times the median character width and the vertical projection overlap ratio is no less than 0.40, they are considered horizontally continuous. When the ratio of the overlapping area of ​​the bounding boxes of two page objects to the area of ​​the smaller bounding box is greater than 0.10, the order is determined by the center point of the bounding boxes and the object type: titles take precedence over body text, image descriptions follow the corresponding image area, and tables are processed according to their overall position in the reading order chain.

[0033] When a page has a multi-column layout, the vertical order of page objects within the same column is first determined, followed by the reading order between adjacent columns, resulting in a reading order chain. Multi-column layouts are identified using the horizontal projection histogram of page objects. A multi-column layout is determined when there is a continuous vertical blank space within the same page with a length not less than 60% of the page height, a width not less than 4% of the page width, and at least three text page objects on both sides of the blank vertical space. The column width is the width between the minimum left boundary and the maximum right boundary of page objects within the same column, and the column spacing is the blank width between adjacent column boundaries. When sorting multiple columns, first determine the column order from left to right according to the horizontal position of the columns, and then sort them from top to bottom according to the vertical position of the page objects within each column. A cross-column title is a title area whose boundary box horizontally covers no less than 60% of the combined width of the two columns, and the cross-column title is placed before the text of each column below it. A cross-column table is placed as an independent page object after the most recent continuous text above it and before the text below it. When the footnote area is located within the bottom 8% range of the page and has a numbered correspondence with the text, it is placed after the corresponding text paragraph. When the sidebar area is located outside the text column and is not continuous with the text, it is placed after the text on the same page.

[0034] S1.3. Identify the hierarchical relationship between page objects based on the reading order chain; when the title area is located before the body text area and is continuous with the body text area in terms of page position, the title area is determined as the parent page object of the body text area.

[0035] Among them, continuous layout means that the title area and the body text area are located in the same column or the same span, the vertical distance from the lower boundary of the title area to the upper boundary of the body text area is no more than twice the midpoint of the row height, and there are no other body text areas, table areas or image description areas between them.

[0036] When a table area and the text areas before and after the table are consecutive in the reading order chain, the table area is determined as a sibling page object of the text area; where consecutive in the reading order chain means that there are no other page objects of different levels between the table area and the text area, and the minimum boundary distance between the two is no greater than twice the midpoint value of the row height.

[0037] When the image description area and the corresponding image text area are adjacent in the bounding box adjacency relationship, the image description area is determined as the subordinate page object of the image text area; where, adjacent means that the minimum boundary distance between the image description area and the image text area is not greater than 1.2 times the midpoint value of the row height, and the overlap ratio between the horizontal projection of the image description area and the horizontal projection of the image text area is not less than 0.40.

[0038] When the header and footer areas are located at the edge of the page and are not continuous with the body text area, the header and footer areas are defined as page-level subordinate page objects, resulting in a hierarchical relationship. Here, the page edge refers to the top 8% height range or the bottom 8% height range of the page, and discontinuity means that the vertical distance between the header / footer area and the body text area is greater than twice the midpoint of the line height, or that the two do not have a numbering, semantic, or spatial continuity relationship in the reading order chain.

[0039] Based on page numbers and page object location ranges, a reading order chain, hierarchical hierarchy, and same-page correspondence between page object coordinate sets are established. Page object coordinates refer to the left, top, right, and bottom boundaries of the page object's location range in PDF page coordinates. Reading order position refers to the sequential position of the page object in the reading order chain. Hierarchical hierarchy position refers to the parent, sibling, and subordinate position of the page object in the hierarchical hierarchy. Same-page correspondence means that the same page object within the same page number simultaneously corresponds to one page object coordinate, one reading order position, and one hierarchical hierarchy position. The page object coordinates in the page object coordinate set, the reading order position in the reading order chain, and the hierarchical hierarchy position in the hierarchical hierarchy are matched item by item according to the same-page correspondence to obtain a hierarchical document structure tree. Each node in the hierarchical document structure tree includes at least a page number, page object type, page object coordinates, reading order number, parent node identifier, sibling node identifier, subordinate node identifier, and a set of candidate text boxes, enabling subsequent candidate text, image regions, OCR word boxes, and front-end pixel regions to be traced back to the same page object node.

[0040] Each segment of page text in the page text data is labeled using a hierarchical document structure tree, and each OCR word box in the OCR text is labeled using the same hierarchical document structure tree. This ensures that each segment of page text or each OCR word box simultaneously has character content, page number, bounding box coordinates, reading order position, and hierarchical subordinate position, resulting in structured page text.

[0041] Establish intra-page correspondences between structured page text, page image data, bounding box coordinates, and normalized target content based on page numbers. Intra-page correspondences refer to the structured page text corresponding to the page image data of the same page number, each segment of the structured page text corresponding to a bounding box coordinate, an OCR word box corresponding to a PDF page coordinate mapped from the image coordinates, and the normalized target content corresponding to all page numbers to be retrieved and the candidate search range for each page number. When the same page number contains multiple page objects, arrange the structured page text according to the reading order and save the hierarchical relationship between page objects according to their subordinate positions to obtain document parsing data.

[0042] S2. The document parsing data is processed using image recognition and semantic understanding branches, and candidate texts are filtered by combining cross-modal alignment and page anchor point parsing to generate candidate positioning results.

[0043] S2.1. The image recognition branch uses the YOLOv8 model, and the semantic understanding branch uses the BERT model. Before processing the document parsing data, historical multi-scene PDF samples are used to complete the training of the image recognition branch and the semantic understanding branch. The historical multi-scene PDF samples include text-type PDF pages, scanned PDF pages, image-embedded PDF pages, and complex layout PDF pages. The page images, text content, document types, text region boundaries, bounding box coordinates, keywords, and precise coordinates in the historical multi-scene PDF samples are labeled to form the training dataset. The training dataset is divided into training set, validation set, and test set in a 7:2:1 ratio.

[0044] The training dataset consists of at least 12,000 PDF pages, with at least 2,000 pages of each PDF page type. Each page type contains at least 500 targets for cross-line targets, cross-paragraph targets, table targets, header / footer distracting targets, and multi-column targets. Precise coordinates refer to the true bounding box coordinates of the target content after page size normalization. Bounding box coordinates represent page objects or candidate text regions, and precise coordinates are used as the final annotation results for target content localization. Keyword annotation includes manually entered historical search terms, field names, field values, and sentence fragments in the document. When the keyword is a field name, the original bounding box of the field name is annotated; when the keyword is a field value, the bounding box of the field value that is adjacent to the field name (e.g., a colon, a table cell, or a vertically adjacent relationship) is annotated. Each annotated bounding box is verified for consistency between two annotation results. An annotation is considered valid if the intersection-union ratio (IU) of the two annotation results is not less than 0.90; otherwise, the annotation result is determined after re-verifying the page object type, text content, and boundary range.

[0045] When training the YOLOv8 model, the page images in the training set are converted to a uniform image size, and the text region boundaries are labeled as object detection bounding boxes. Document types and page object types are labeled as object categories, resulting in image recognition training samples. These training samples are then input into the YOLOv8 model, which outputs predicted text region boundaries, predicted document types, predicted page object types, and prediction confidence scores. Positional differences are calculated between the predicted text region boundaries and the labeled text region boundaries. Category differences are calculated between the predicted document types and predicted page object types and the labeled document types and page object types. The results of the positional difference calculation and the category difference calculation are then used to determine the final outcome. The detection parameters of the YOLOv8 model were adjusted. Positional differences were evaluated using the intersection-union ratio (IU / U) of the predicted text region boundaries and the labeled text region boundaries. Class differences were evaluated using the classification cross-entropy. Prediction confidence was used to filter low-confidence regions. When the mAP50 (mean accuracy of each page object detection category when the IU / U threshold is 0.50) of page object detection in the validation set is not less than 0.90, the average IU / U of text region boundaries is not less than 0.85, the document type classification accuracy is not less than 0.95, and the mAP50 change is less than 0.01 over three consecutive training rounds, the text region boundary localization result is considered stable, and an image recognition branch capable of locating text region boundaries in page images is obtained.

[0046] When training the BERT model, the normalized target content and structured page text fragments in the training set are first combined into text pairs. Whether a text fragment corresponds to the normalized target content is labeled as a semantic matching tag. Text fragment identifiers, page number identifiers, and bounding box indices are used as lookup labels. PDF coordinates are not used as the direct regression output of BERT, resulting in semantic understanding training samples. These training samples are then input into the BERT model, which outputs the semantic correspondence between text fragments and normalized target content, semantic matching scores, and candidate text fragment identifiers. Based on the candidate text fragment identifiers, the corresponding page numbers and bounding box coordinates are retrieved from the document parsing data, yielding page number lookup results and bounding box lookup results. Results: The semantic correspondence results and semantic matching labels were compared using difference calculations. The candidate text fragment identifiers and backtracking labels were also compared using difference calculations. Based on the difference calculation results, the semantic parameters of the BERT model were fine-tuned using a learning rate of 0.001 and a batch size of 32. The difference calculations used binary cross-entropy and candidate fragment ranking loss. Text fragments with a semantic matching score of not less than 0.80 were included in the candidate set. When the semantic matching F1 score in the validation set was not less than 0.88, the top 3 recall rate of the target content was not less than 0.95, and the change in the F1 score over three consecutive training rounds was less than 0.01, the semantic correspondence results were considered stable, and a semantic understanding branch capable of comparing normalized target content and structured page text was obtained.

[0047] When validating the image recognition and semantic understanding branches using a test set, the page images from the test set are input into the image recognition branch to obtain the test text region boundaries, test document type, test page object type, and test confidence. OCR recognition is then performed on the scanned text and image-embedded text within the test text region boundaries to obtain test OCR text boxes, which are then mapped to PDF page coordinates. The normalized target content, structured page text fragments, and corresponding text from the test OCR text boxes are input into the semantic understanding branch to obtain test semantic candidate fragments, test page numbers, and test bounding box coordinates. The test text region boundaries, test page numbers, and test bounding box coordinates are then compared with the labeled text region boundaries, labeled page numbers, and labeled bounding box coordinates in the test set for consistency verification. Candidate text localization results that match the verification are retained. During consistency verification, the candidate text localization results are considered consistent if the intersection-union ratio (IU) between the test text region boundaries and the labeled text region boundaries is not less than 0.85, the test page numbers match the labeled page numbers, and the IU between the test bounding box coordinates and the precise coordinates is not less than 0.80.

[0048] S2.2. The image recognition branch detects page image data in the document parsing data. Based on the text region boundaries in the page image, it locates the text outlines in the scanned PDF page and the image-embedded PDF page. The text outlines are converted into text region boundaries in the PDF page coordinates. OCR recognition is used to obtain the OCR text, OCR word boxes, and OCR confidence scores within the text outlines. The OCR word boxes are mapped proportionally according to the width and height of the page rendered image, the width and height of the PDF page, forming OCR bounding boxes consistent with the PDF page coordinates. The text region boundaries are then mapped to the bounding box coordinates in the page object coordinate set to obtain image candidate segments.

[0049] The semantic understanding branch processes the normalized target content and structured page text in the document parsing data. The semantic understanding branch first divides the normalized target content into target word sequences, and then divides the structured page text and OCR text into text fragments according to the reading order. Subsequently, it compares the contextual semantic relationship between the target word sequences and text fragments, retains the text fragments that correspond to the meaning of the normalized target content, and simultaneously retains the page number, bounding box coordinates, reading order position, and hierarchical subordinate position of the text fragments to obtain semantic candidate fragments.

[0050] Semantic candidate fragments are only used to identify original text fragments that may contain target content. The final highlighted boundary is based on the actual word boundary box in the original text fragment. When the normalized target content is a field name, the semantic candidate fragment corresponds to the original text of the field name. When the normalized target content contains both field name and field value, the field name is used as the page anchor point to locate the original text of the field value that has the same table row, the same text following the colon, or an adjacent relationship with the field name.

[0051] S2.3. Page number, bounding box coordinates, reading order position, and hierarchical subordinate position are used as common indices to perform a two-branch correspondence between image candidate segments and semantic candidate segments. The common index is not directly taken from the model output, but is uniformly generated by the hierarchical document structure tree. For scanned PDF pages, after the text region boundary output by the image recognition branch is recognized by OCR and mapped to coordinates, the OCR page object, OCR reading order position, and OCR hierarchical subordinate position are supplemented according to the methods in S1.2 and S1.3. For text-type PDF pages, when there are no valid image candidate segments, the semantic candidate segments and their nodes in the hierarchical document structure tree are used as common indices. For image-embedded PDF pages and complex layout PDF pages, the OCR bounding boxes of the image candidate segments are mapped to PDF page coordinates and then corresponded with the semantic candidate segments.

[0052] When image candidate fragments and semantic candidate fragments have the same page number, and the text region boundary of the image candidate fragment and the bounding box coordinates of the semantic candidate fragment are within the same page object coordinate range, the image candidate fragment and the semantic candidate fragment are merged into the same candidate text record. When a text-type PDF page contains only semantic candidate fragments, the semantic candidate fragments are retained and the candidate text source is marked as a text-type PDF. When a scanned PDF page contains only image candidate fragments, the OCR text in the image candidate fragments is used as candidate text and the candidate text source is marked as a scanned PDF. When both image-embedded PDF pages and complex layout PDF pages contain image candidate fragments and semantic candidate fragments, the candidate text records corresponding to the two branches that are successfully obtained are retained according to the common index to obtain the branch candidate results.

[0053] Based on the branch candidate results, the candidate text sources in text-based PDFs, scanned PDFs, image-embedded PDFs, and complex layout PDFs are labeled to obtain the branch candidate source set. Each candidate text record in the branch candidate source set includes candidate text, page number, candidate text source, page image block, bounding box coordinates, reading order position, and hierarchical subordinate position. The page image block refers to the local area of ​​the page image corresponding to the bounding box coordinates of the candidate text. The page image block preserves the visual position of the candidate text in the page image.

[0054] S2.4. A cross-modal alignment mechanism is used to process candidate text, page image blocks, and bounding box coordinates in the branch candidate source set. This mechanism places the word sequence of the candidate text, the image region of the page image block, and the bounding box coordinates under the same page number for correspondence, confirming whether the text content of the candidate text, the visual region of the page image block, and the bounding box coordinates point to the same page object. Confirmation methods include coordinate alignment, text alignment, and object alignment: Coordinate alignment means that after the page image block is mapped from image pixel coordinates to PDF page coordinates, the intersection-over-union (IoU) ratio with the candidate text bounding box is not less than 0.70; text alignment means that the semantic matching score between the OCR text or structured page text and the normalized target content is not less than 0.80, and the compatible character matching coverage is not less than 0.75; object alignment means that the page image block, candidate text bounding box, and layout anchor point all fall into the same page object node in the hierarchical document structure tree. When the word sequence of the candidate text, the image region of the page image block, and the bounding box coordinates point to the same page object, the corresponding candidate text record is retained, resulting in a text-image alignment result. When mapping image pixel coordinates to PDF page coordinates, a proportional mapping relationship is established using the page rendered image width, page rendered image height, CropBox width, CropBox height, and page rotation angle. The mapped left, top, right, and bottom boundaries are then unified to the PDF page display coordinates.

[0055] Page objects in the hierarchical document structure tree are used as layout anchors to verify the layout position of text-image alignment results. Layout anchors refer to title areas, body text areas, table areas, image caption areas, and header / footer areas in the hierarchical document structure tree that have page object coordinates, reading order positions, and hierarchical subordinate positions. For complex PDF pages, the page objects containing candidate text are first located according to the reading order chain. Then, the candidate text within the page objects is parsed using the page objects as layout anchors, establishing a correspondence between the candidate text and the layout anchors to obtain the layout anchor matching results.

[0056] The text-image alignment results are filtered using the page anchor point matching results. When a candidate text matches the normalized target content, and the candidate text's reading order position, page object position, and hierarchical subordinate position are all consistent with the page anchor point matching results, the candidate text is retained. When a candidate text matches the normalized target content, but the candidate text's reading order position, page object position, and hierarchical subordinate position are inconsistent, the candidate text is excluded. The retained candidate text records are arranged according to page number and reading order position to form candidate positioning results.

[0057] S3. Based on the candidate localization results, page image, and hierarchical document structure tree, determine the local area of ​​the page number, and perform bounding box lexical verification, compatibility search, and geometric stitching to generate the target bounding box.

[0058] S3.1. Extract the candidate text, page number, preliminary bounding box, page anchor point matching result, reading order position, and hierarchical subordinate position from the candidate positioning results as candidate positioning indexes; the candidate text in the candidate positioning index is the text fragment retained after matching with the normalized target content, the page number is the PDF page number where the candidate text is located, the preliminary bounding box is the rectangular coordinate range of the candidate text in the PDF page coordinates, the page anchor point matching result is the page object corresponding to the candidate text, the reading order position is the sequential position of the candidate text in the reading order chain, and the hierarchical subordinate position is the superior, sibling, and subordinate position of the candidate text in the hierarchical subordinate relationship.

[0059] A page visual multi-vector retrieval and late-interaction matching mechanism is employed to process candidate positioning indices and page images. This mechanism performs item-by-item matching between the normalized target content in the candidate positioning index, candidate text, and text regions, table regions, body text regions, and layout object positions in the page images to obtain page number candidate results. The page visual multi-vectors include target text vectors, candidate text vectors, image region vectors, and layout structure vectors. The target text vector is obtained by semantic understanding branch encoding of target words in the normalized target content. The candidate text vector is obtained by semantic understanding branch encoding of candidate words in the structured page text or OCR text. The image region vector consists of the number of connected components in the page image block, the horizontal projection histogram, the vertical projection histogram, the edge density, and the normalized bounding box coordinates. The layout structure vector consists of the page object type, reading order position, hierarchical subordinate position, bounding box center point coordinates, bounding box width, and bounding box height.

[0060] To further explain, late interaction matching refers to matching the normalized target content's lexical sequence with multiple local regions in the page image, and then determining the page range where the target content is located based on the position of the matched local regions. During late interaction matching, the semantic matching score of each target lexical with candidate lexicals on the same page is calculated, and the highest matching score for that target lexical within each page object is retained. The highest matching scores of all target lexicals within the same page object are averaged to obtain the text late interaction score. The intersection-union ratio (IUU) of the candidate bounding boxes and the page object bounding boxes, reading order consistency, and hierarchical subordination consistency are combined to form the layout consistency score. When the text late interaction score is not lower than 0.78 and the layout consistency score is not lower than 0.80, the corresponding page object is determined as a page number candidate result. If there are multiple page number candidate results for the same target content, the top three candidate results are retained for subsequent verification according to the weighted order of text late interaction score, layout consistency score, and candidate bounding box coverage.

[0061] S3.2. Based on the page number candidate results, limit the range of the page object containing the candidate text in the page image to obtain the candidate page region; the candidate page region is a rectangular page region that covers the candidate text, candidate page object, and candidate local coordinate range within the candidate page number; the boundary of the candidate page region must not exceed the page object position range in the page object coordinate set, and the page number corresponding to the candidate page region must not exceed the page number range in the document parsing data.

[0062] The reading order and hierarchical position of candidate page areas are verified using the hierarchical document structure tree. Candidate page areas are retained when the reading order of candidate text within a candidate page area matches the reading order in the candidate positioning index, and when the hierarchical position of page objects within a candidate page area matches the hierarchical position in the page anchor matching result. Candidate page areas are excluded when the reading order and hierarchical position are inconsistent. The retained candidate page areas are then arranged according to page number and reading order to obtain the page number local area.

[0063] Extract candidate text, preliminary bounding boxes, page anchor point matching results, and hierarchical subordinate positions from local areas of page numbers. Decompose the candidate text into candidate words, and bind each candidate word to its corresponding preliminary bounding box to construct a candidate word sequence. The bounding boxes for candidate words are preferentially obtained by merging character-level PDF bounding boxes; if no character-level PDF bounding boxes exist, OCR word boxes are used; if only text region boundaries exist without word boxes, word bounding boxes are allocated within the text region boundaries according to the proportion of candidate text character width.

[0064] S3.3. The normalized target content is split into a sequence of target lexical units, and the sequence of target lexical units is semantically mapped to the sequence of candidate box lexical units to obtain a target candidate correspondence chain. The target candidate correspondence chain records the textual meaning correspondence, page number correspondence, reading order correspondence, and hierarchical subordinate correspondence between target lexical units and candidate box lexical units. The target candidate correspondence chain is used to determine whether a candidate box lexical unit is a component of the normalized target content. When the semantic correspondence is a synonymous expression relationship rather than a character-compatible hit relationship, the target candidate correspondence chain is only used to determine the original text candidate lexical units, and does not replace the original text lexical units in the PDF with user-input lexical units. The final highlighted range always corresponds to the original text candidate lexical units that actually exist in the PDF page.

[0065] The semantic space consistency result is obtained by comparing the textual meaning of the candidate text, the page position of the initial bounding box, the page object position of the page anchor matching result, and the hierarchical subordinate position. A semantic space consistency result is recorded as consistent when the candidate text has the same meaning as the normalized target content, the initial bounding box falls within the page object range corresponding to the page anchor matching result, and the hierarchical subordinate position of the candidate text is consistent with the hierarchical subordinate position in the hierarchical document structure tree. A semantic space consistency result is recorded as inconsistent when the candidate text has a semantic correspondence with the normalized target content, but the initial bounding box does not fall within the page object range corresponding to the page anchor matching result, or the hierarchical subordinate position of the candidate text is inconsistent with the hierarchical subordinate position in the hierarchical document structure tree. Falling within the page object range means that the center point of the candidate word bounding box is located within the page object range corresponding to the page anchor, and the overlap area between the candidate word bounding box and the page object range is not less than 0.85 of the candidate word bounding box area.

[0066] S3.4. Process candidate box words based on semantic space consistency results; when the semantic space consistency results are consistent, retain the corresponding candidate box words to obtain valid candidate box words; when the semantic space consistency results are inconsistent, remove the corresponding candidate box words to obtain invalid candidate box words; rearrange the valid candidate box words according to the reading order position in the hierarchical document structure tree to obtain the verification bounding box sequence.

[0067] Text compatibility search is conducted using normalized target content and a verification bounding box sequence. The search compares the normalized target content with candidate text in the verification bounding box sequence segment by segment, handling differences in full-width and half-width characters, special spaces, punctuation encoding, and line breaks / hyphens to obtain compatibility hit results. The compatibility search uses the target word sequence as a baseline, matching candidate bounding box words one by one in the reading order. A character compatibility hit is defined as when the target word and candidate bounding box words are completely identical after character normalization, or completely identical after removing differences in spaces and punctuation encoding. When the character compatibility hit ratio of consecutive target words is not less than 0.90, the compatibility hit result is considered to cover the corresponding target content segment.

[0068] When the compatible hit result covers the complete normalized target content, the bounding box corresponding to the compatible hit result is extracted to obtain the complete hit box; the complete hit box retains the page number, left boundary, top boundary, right boundary and bottom boundary of the normalized target content in the PDF page coordinates.

[0069] When the compatibility hit result does not cover the complete normalized target content, adjacent candidate box words are selected according to the reading order to obtain a group of boxes to be concatenated. The candidate box words in the group of boxes to be concatenated are located on the same page and in a continuous reading order. The hierarchical subordinate position in the group of boxes to be concatenated is consistent with the target candidate corresponding chain. The group of boxes to be concatenated covers the target words in the normalized target content that are not covered by the compatibility hit result. For sentence fragments that span lines, paragraphs, or pages, the group of boxes to be concatenated is saved in segments according to the order of the target words in the target candidate corresponding chain, and is not forcibly merged into a single rectangle.

[0070] S3.5. Geometrically stitch the groups of boxes to be stitched together based on hierarchical position, reading order chain, and page object coordinate set to obtain stitched bounding boxes. The geometric stitching arranges the candidate box words in the group according to the reading order chain, and merges the bounding boxes of consecutive candidate box words within the same page object range into the same stitched bounding box. When the normalized target content spans adjacent text lines, multiple stitched bounding boxes are retained according to the reading order chain. When the normalized target content spans adjacent page objects, the stitched bounding boxes within the corresponding page object range are retained according to hierarchical position. When the vertical central difference of consecutive candidate box words within the same line is not greater than 0.5 times the line height median, they are merged into the same line bounding box. Candidate box words spanning multiple lines are not merged into a single large rectangle, but are output as multiple bounding boxes per line. Candidate box words spanning multiple paragraphs or page objects are output as separate bounding box sets per page object.

[0071] Remove duplicate, adjacent overlapping, and invalid regions from the complete hit box and the spliced ​​bounding box to obtain the target bounding box. Duplicate regions are bounding box regions with the same page number and completely identical boundary range. Adjacent overlapping regions are bounding box regions with the same page number and overlapping boundary range. Invalid regions are bounding box regions that do not correspond to valid candidate box words, do not correspond to page anchor point matching results, or are not within the range of the page object coordinate set.

[0072] The method for clearing adjacent overlapping areas is as follows: when the overlap area of ​​two bounding boxes is not less than 0.80 of the area of ​​the smaller bounding box, the bounding box that covers more target words is retained; when the two bounding boxes cover the same number of target words, the bounding box with a higher intersection-union ratio with the page anchor point matching result is retained.

[0073] S4. Based on the target bounding box, page size, and layout structure, perform front-end coordinate transformation and text space verification to generate positioning and rendering data.

[0074] S4.1. Extract the target bounding box, the page number corresponding to the target bounding box, the candidate text corresponding to the target bounding box, the page anchor matching result, the reading order position, and the hierarchical subordinate position; the target bounding box includes the page number where the target content is located, the left boundary, top boundary, right boundary, and bottom boundary corresponding to the target content; the page number corresponding to the target bounding box is used to determine the PDF page to be converted, the candidate text corresponding to the target bounding box is used for subsequent text space verification, and the page anchor matching result, reading order position, and hierarchical subordinate position are used to confirm the page object and hierarchical position to which the target bounding box belongs, thus obtaining the target bounding box conversion data.

[0075] Based on the target bounding box transformation data, extract the page size, page orientation, and PDF coordinate position of the page number corresponding to the target bounding box to obtain the page coordinate reference. The page size refers to the page width and page height of the page number corresponding to the target bounding box, the page orientation refers to the rotation direction of the page number corresponding to the target bounding box in the PDF rendering page, and the PDF coordinate position refers to the left, top, right, and bottom boundaries of the target bounding box in the PDF page coordinates. The page coordinate reference is used to unify the PDF page coordinates, page size, and page orientation to avoid positioning offset of the target bounding box under different page sizes and page orientations.

[0076] The page coordinate reference also includes MediaBox, CropBox, page rotation angle, PDF rendering scaling ratio, device pixel ratio, page scroll container offset, page spacing, and PDF rendering viewport transformation matrix; when CropBox exists, CropBox is used as the visible page boundary, and when CropBox does not exist, MediaBox is used as the visible page boundary.

[0077] S4.2. Convert the vertical direction and display units of the target bounding box according to the page coordinate reference to obtain the front-end pixel area; when converting the vertical direction of the target bounding box, convert the vertical position in the PDF page coordinates to the vertical display position in the PDF rendering page, so that the target bounding box adapts to the vertical direction of the front-end display coordinates; when converting the display units of the target bounding box, convert the PDF coordinate position of the target bounding box to the front-end pixel coordinates according to the page size, display scaling ratio and page orientation of the PDF rendering page; the front-end pixel area includes the left pixel position, top pixel position, right pixel position and bottom pixel position of the target content in the PDF rendering page.

[0078] During the transformation, the four corner points of the target bounding box are first transformed from PDF page coordinates to PDF rendering viewport coordinates, then multiplied by the device pixel ratio to obtain canvas pixel coordinates, and the left offset, top offset, and page spacing of the PDF rendering page in the scrolling container are superimposed to obtain the front pixel area. When the page rotation angle is 0 degrees, 90 degrees, 180 degrees, or 270 degrees, the four corner points are transformed through the corresponding viewport transformation matrix, and then the minimum x-coordinate, minimum y-coordinate, maximum x-coordinate, and maximum y-coordinate of the four corner points after transformation are taken as the boundary of the front pixel area.

[0079] The front-end pixel region is matched with the candidate text, page anchor point matching results, and hierarchical subordinate position of the target bounding box to obtain the pixel verification result. The front-end lexical region corresponding to the candidate text used in the text space verification is not calculated separately from the target bounding box, but is established as an independent reference region according to the source of the candidate text: when the candidate text comes from the character stream of a text-type PDF page, the PDF renderer text layer generates the front-end lexical region corresponding to the candidate text according to the character box, page viewport transformation matrix, PDF rendering scaling ratio, and device pixel ratio; when the candidate text comes from the O of a scanned PDF page or an image-embedded PDF page... When processing CR results, the OCR word boxes mapped to PDF page coordinates in S1.1 are transformed using page coordinate references to generate the front-end word regions corresponding to the candidate text. When the candidate text is located in a table area, image description area, or header / footer area, the coordinates of the corresponding page object in the hierarchical document structure tree are simultaneously transformed using page coordinate references to generate the front-end pixel range of the page object, which serves as the front-end verification range for the layout anchor point matching results. The aforementioned front-end word regions corresponding to the candidate text and the front-end pixel range of the page object are generated before or independently of the target bounding box transformation process to avoid the front-end pixel regions from self-verifying their own transformation results.

[0080] Text space verification refers to verifying whether the front-end pixel region covers the candidate text corresponding to the target bounding box, whether it falls within the page object range corresponding to the page anchor matching result, and whether it maintains the hierarchical subordinate position of the target bounding box in the hierarchical document structure tree. Covering the candidate text corresponding to the target bounding box means there is a stable overlap between the front-end pixel region and the front-end word region corresponding to the candidate text; falling within the page object range corresponding to the page anchor matching result means the front-end pixel region is located within the front-end pixel range of the page object; maintaining the hierarchical subordinate position means the page object node identifier obtained by the front-end pixel region is consistent with the page object node identifier of the target bounding box in the hierarchical document structure tree. When the front-end pixel region simultaneously covers the candidate text corresponding to the target bounding box, falls within the page object range corresponding to the page anchor matching result, and maintains the hierarchical subordinate position, the pixel verification result is recorded as consistent. When the front-end pixel region does not cover the candidate text corresponding to the target bounding box, deviates from the page object range corresponding to the page anchor matching result, or deviates from the hierarchical subordinate position, the pixel verification result is recorded as inconsistent.

[0081] "Covering candidate text" means that the overlap area between the front-end pixel region and the corresponding front-end word region of the candidate text is no less than 0.90 of the area of ​​the corresponding front-end word region of the candidate text. This ratio is the candidate text coverage threshold, used to determine whether the front-end pixel region fully covers the independently generated front-end word region of the candidate text. The value of 0.90 comes from the annotation consistency requirement in S2.1 that the intersection-union ratio of the effective annotation bounding boxes is no less than 0.90, and is used as the fixed candidate text coverage threshold in the front-end verification configuration table. "Falling into the page object range corresponding to the anchor point matching result" means that the center point of the front-end pixel region is located within the front-end pixel range of the page object, and the front-end image... The overlap area between the pixel region and the front pixel range of the page object is no less than 0.85 of the area of ​​the front pixel region. This ratio is the page object falling threshold, used to determine whether the front pixel region is still within the page object range corresponding to the page anchor point matching result. The value of 0.85 comes from the consistency verification requirement in S2.1 that the intersection-union ratio of the test text region boundary and the annotation text region boundary is no less than 0.85, and is used as the fixed page object falling threshold in the front verification configuration table. Maintaining the hierarchical subordinate position means that the page object node identifier obtained by the front pixel region back lookup is consistent with the page object node identifier of the target bounding box in the hierarchical document structure tree.

[0082] S4.3. Process the front-end pixel region based on the pixel verification results; if the pixel verification results are consistent, retain the front-end pixel region; if the pixel verification results are inconsistent, remove the front-end pixel region and reconstruct the vertical direction and display units of the target bounding box according to the page coordinate reference to obtain the corrected pixel region; the reconstructing is not an infinite adjustment of coordinates, but a correction strategy is selected according to the reason for inconsistency: if the front-end pixel region is upside down, switch the PDF vertical origin reference; if the front-end pixel region rotates in the wrong direction, use the viewport transformation matrix corresponding to the page rotation angle to reconstruct; if the front-end pixel region is translated relative to the page as a whole, switch the CropBox and MediaBox references and recalculate the page offset; if the size of the front-end pixel region is proportionally too large or too small, reread the display scaling ratio and device pixel ratio; if the above corrections still cannot meet the thresholds for covering candidate text and falling within the page object range, the target bounding box or page anchor point is determined to be incorrect, and return to S3.4 to select the next candidate box word sequence to regenerate the target bounding box.

[0083] The text space verification continues for the corrected pixel area, with a maximum of 3 corrected verifications, until the corrected pixel area covers the candidate text corresponding to the target bounding box, falls into the page object range corresponding to the page anchor point matching result, and maintains the hierarchical subordinate position, thus obtaining a front-end pixel area that has been verified.

[0084] The position of the highlighted rectangle and the page scroll position are determined by verifying the consistent front-end pixel area, thus obtaining the positioning and rendering data; the position of the highlighted rectangle includes the left pixel position, the top pixel position, the rectangle width, and the rectangle height of the target pixel area; the page scroll position is determined based on the vertical display position of the target pixel area in the PDF rendering page.

[0085] When the target bounding box is a collection of bounding boxes spanning lines, paragraphs, or pages, multiple highlight rectangle positions are generated, and the vertical center of the first highlight rectangle position is used as the page scroll position; when the target content is a field value, the highlight rectangle position corresponds to the original bounding box of the field value, and the field name bounding box is only saved as a layout anchor point.

[0086] S5. By positioning the data, mark the target pixel area in the PDF rendering page and display it in the center, generating the PDF document content positioning result.

[0087] S5.1. Extract the page number of the target content, the target pixel area, the position of the highlighted rectangle, the page scroll position, the candidate text, and the hierarchical subordinate position from the positioning and rendering data; the page number of the target content is used to determine the PDF rendering page, the target pixel area is used to determine the display range of the target content in the PDF rendering page, the position of the highlighted rectangle is used to determine the left pixel position, the top pixel position, the width and height of the rectangle, the page scroll position is used to determine the starting position of the PDF rendering page, and the candidate text and hierarchical subordinate positions are used to verify the text content and page structure position of the target display area to obtain the target display parameters.

[0088] When the target content is a keyword, the target display parameter corresponds to the keyword's bounding box in the original PDF text; when the target content is a sentence fragment, the target display parameter corresponds to multiple bounding boxes of the original text arranged in reading order; when the target content is a field name, the target display parameter corresponds to the field name's original text bounding box; when the target content is a field value, the target display parameter corresponds to the field value's original text bounding box, while retaining the field name bounding box as anchor verification information; when the semantic understanding branch hits a synonym expression, the target display parameter corresponds to the bounding box of the hit candidate text in the original PDF text, without highlighting user-input words that do not exist in the PDF page.

[0089] The target pixel area is determined in the PDF rendering page based on the target display parameters to obtain the target display area. The target display area is located in the PDF rendering page corresponding to the page number of the target content. The left pixel position, top pixel position, rectangle width and rectangle height of the target display area are consistent with the position of the highlighted rectangle. The target display area covers the front pixel area corresponding to the candidate text and is consistent with the hierarchical subordinate position in the positioning and presentation data.

[0090] S5.2. Draw a highlight mark in the target display area and adjust the display position of the PDF rendering page according to the page scroll position to obtain the target centered page; the highlight mark covers the target pixel area corresponding to the target display area, the display range of the highlight mark does not exceed the page boundary of the PDF rendering page, the page scroll position makes the target display area enter the visible range of the PDF rendering page, and the target centered page retains the correspondence between the page number of the target content, the target display area and the highlight mark.

[0091] When new target content is received, the new target content is normalized to obtain new normalized target content; the structured page text, page image data, hierarchical document structure tree and bounding box coordinates in the document parsing data are called, and the new target pixel area is re-determined by combining the candidate positioning results, the verification bounding box sequence, the target bounding box and the positioning presentation data, to obtain the new target display area.

[0092] Draw a highlight mark in the new target display area, and adjust the display position of the PDF rendering page according to the page scroll position corresponding to the new target display area to obtain the PDF document content positioning result.

[0093] In summary, this invention integrates page text, page images, bounding box coordinates, and target content into document parsing data, and preserves reading order and hierarchical relationships through a hierarchical document structure tree. This ensures that the location of target content no longer depends on a single keyword hit, but is simultaneously constrained by page objects, layout structure, and coordinate range. By processing scanned documents, image-embedded PDFs, and text-based PDFs through image recognition and semantic understanding branches respectively, it improves the coverage of candidate text for different types of PDF pages. By filtering candidate texts with matching text content but inconsistent reading order, layout object position, or hierarchical position through cross-modal alignment and layout anchor point parsing, it reduces the probability of mislocation under complex layouts. By generating target bounding boxes through bounding box lexical verification, compatible search, and geometric splicing, it can handle the incomplete positioning problems caused by full-width and half-width characters, special spaces, line breaks, and cross-text block splitting. By generating positioning presentation data through front-end coordinate transformation and text space verification, it can reduce highlight offset caused by inconsistencies between PDF coordinates and front-end pixel coordinates.

[0094] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An AI-driven method for accurate content location in PDF documents, characterized in that: include, The process involves parsing the page text, page images, bounding box coordinates, and target content of the PDF document to be located, constructing a hierarchical document structure tree, and associating the reading order and hierarchical relationship with the corresponding bounding box coordinates to generate document parsing data. The document parsing data is processed using image recognition and semantic understanding branches, and candidate texts are filtered by combining cross-modal alignment and page anchor point parsing to generate candidate positioning results; Based on the candidate localization results, page images, and hierarchical document structure tree, the local area of ​​the page number is determined, and bounding box lexical verification, compatibility search, and geometric stitching are performed to generate the target bounding box; Based on the target bounding box, page size, and layout structure, perform front-end coordinate transformation and text space verification to generate positioning and rendering data; By positioning and presenting data, the target pixel area is marked and centered on the PDF rendering page, generating the PDF document content positioning result.

2. The AI-driven method for accurate content location in PDF documents as described in claim 1, characterized in that: The construction of the hierarchical document structure tree specifically includes: Based on the line position of the page text, the text area boundary and bounding box coordinates in the page image, and combined with the font size, font attributes, numbering format, text alignment, wireframe cells, text baseline alignment, image caption number and page edge position, the page object type is identified and the page object is divided into categories, resulting in a set of page object coordinates. The reading order of page objects is determined by their vertical and horizontal positions, multi-column division results, and bounding box adjacency relationships within the page object coordinate set, thus obtaining the reading order chain. The hierarchy is determined by identifying the subordinate relationships between headings, body text, tables, image descriptions, and headers and footers based on the reading order chain, page object type, and bounding box adjacency. Establish the same-page correspondence between the reading order chain, hierarchical subordinate relationship, and page object coordinate set, and write the page object coordinate, reading order position, hierarchical subordinate position, and candidate text box set into the corresponding tree node to obtain the hierarchical document structure tree.

3. The AI-driven method for precise content location in PDF documents as described in claim 2, characterized in that: The generated document parsing data specifically includes: Extract page text, page image, and bounding box coordinates from the PDF document to be located. Perform character recognition on the text region boundary in the page image to obtain OCR text and OCR word box. Then, perform character normalization on the target content to obtain normalized target content. The hierarchical document structure tree is used to mark the reading order and hierarchical subordinate position of the page text and OCR text, resulting in structured page text; By establishing a correspondence between structured page text, page images, bounding box coordinates, OCR word boxes, and normalized target content according to page numbers, document parsing data is obtained.

4. The AI-driven method for precise content location in PDF documents as described in claim 3, characterized in that: The process of processing document parsing data using image recognition and semantic understanding branches specifically includes: By detecting page images in document parsing data through the image recognition branch, locating the text region boundaries in the page images, performing OCR recognition on the text within the text region boundaries, and mapping the OCR word boxes from page image coordinates to PDF page coordinates, and mapping the text region boundaries to the bounding box coordinates of structured page text, image candidate fragments are obtained. By comparing the normalized target content, structured page text, and OCR text through the semantic understanding branch, and retaining the reading order position and hierarchical subordinate position of the candidate text, semantic candidate fragments are obtained. Using page number, bounding box coordinates, reading order position, and hierarchical subordinate position as common indices uniformly generated by the hierarchical document structure tree, a two-branch correspondence is performed on image candidate fragments and semantic candidate fragments to obtain branch candidate results; Based on the branch candidate results, candidate text sources in text-based PDFs, scanned PDFs, image-embedded PDFs, and complex layout PDFs are labeled to obtain the branch candidate source set.

5. The AI-driven method for precise content location in PDF documents as described in claim 4, characterized in that: The generation of candidate localization results specifically includes: Cross-modal alignment is performed on candidate text, page image blocks, and bounding box coordinates in the branch candidate source set. The image pixel coordinates of the page image blocks are mapped to PDF page coordinates. Coordinate alignment, text alignment, and object alignment are performed on the candidate text, page image blocks, and bounding box coordinates to obtain the text-image alignment result. Using page objects in the hierarchical document structure tree as page anchors, the text-image alignment results are verified for page position to obtain page anchor matching results. By using the anchor point matching results, candidate texts that match the normalized target content but do not match the reading order, page object position, or hierarchical subordinate position are excluded, thus obtaining candidate positioning results.

6. The AI-driven method for precise content location in PDF documents as described in claim 5, characterized in that: The determination of the local area of ​​the page number specifically includes: Extract the candidate text, page number, preliminary bounding box, page anchor point matching result, reading order position, and hierarchical subordinate position from the candidate positioning results to obtain the candidate positioning index; Based on the candidate positioning index, a page visual multi-vector retrieval and late interaction matching mechanism is used to limit the range of page objects where candidate text is located in the page image to obtain the candidate page region. By verifying the reading order and hierarchical position of candidate page areas using the hierarchical document structure tree, a local area of ​​page number is obtained.

7. The AI-driven method for precise content location in PDF documents as described in claim 6, characterized in that: The bounding box term morphology verification specifically includes: Extract candidate text, preliminary bounding boxes, page anchor point matching results, and hierarchical subordinate positions from local areas of page numbers. Decompose the candidate text into candidate words and bind the candidate words to PDF character boxes, OCR word boxes, or word bounding boxes allocated according to character width ratio to construct candidate box word sequence. The normalized target content is split into a target word sequence, and the target word sequence is semantically mapped to the candidate box word sequence to obtain the target candidate mapping chain; Based on the textual meaning of the candidate texts compared with the target candidate corresponding chain, the page position of the preliminary bounding box, the page object position and hierarchical subordinate position of the page anchor point matching result, the semantic space consistency result is obtained; When the semantic space consistency result shows that the candidate text is consistent with the meaning of the normalized target content and the initial bounding box falls within the range of the corresponding page object, the corresponding candidate box word is retained to obtain the effective candidate box word. When the semantic space consistency result shows that the candidate text has a semantic correspondence with the normalized target content and the preliminary bounding box does not fall within the scope of the corresponding page object, the corresponding candidate box word is removed to obtain invalid candidate box word. The valid candidate box words are rearranged according to their reading order in the hierarchical document structure tree to obtain the verification bounding box sequence.

8. The AI-driven method for precise content location in PDF documents as described in claim 7, characterized in that: The compatible search and geometric stitching specifically include: Text compatibility search is performed using normalized target content and verification bounding box sequences. Differences in full-width and half-width characters, special spaces, punctuation encoding, and line breaks are handled to obtain compatibility hit results. When the compatible hit result covers the complete normalized target content, extract the bounding box corresponding to the compatible hit result to obtain the complete hit box; When the compatibility hit result does not cover the complete normalized target content, adjacent candidate box words are selected according to the reading order to obtain the group of boxes to be spliced. Based on the hierarchical subordinate position, reading order chain, and page object coordinate set, the groups of boxes to be spliced ​​are spliced ​​to obtain spliced ​​bounding boxes. A set of bounding boxes is formed by merging within the same line, splitting across lines, splitting across segments, and retaining cross-page objects separately. Duplicate areas, adjacent overlapping areas, and invalid areas in the complete hit box and spliced ​​bounding boxes are removed to obtain the target bounding box.

9. The AI-driven method for precise content location in PDF documents as described in claim 8, characterized in that: The generated location presentation data specifically includes: Extract the page size, page orientation, and PDF coordinate position of the page number corresponding to the target bounding box, and extract the visible page boundary, page rotation angle, PDF rendering scaling ratio, device pixel ratio, page scroll container offset, page spacing, and PDF rendering viewport transformation matrix to obtain the page coordinate reference. Based on the page coordinate reference, the vertical direction and display unit of the target bounding box are transformed, and the corner points of the target bounding box are transformed from PDF page coordinates to PDF rendering viewport coordinates. Combined with the device pixel ratio, page scroll container offset and page spacing, the front pixel area is obtained. The candidate text, layout anchor point matching results, and hierarchical subordinate positions corresponding to the front pixel region and the target bounding box are verified in text space to obtain pixel verification results. Based on the pixel verification results, the front-end pixel areas that are consistent with the text meaning, page object position, and hierarchical subordinate position corresponding to the target bounding box are retained. Front-end pixel areas that deviate from the page object range, hierarchical subordinate position, or candidate text corresponding area are eliminated and re-transformed. If the re-transformation still fails the text space verification, the verification bounding box sequence is returned to redetermine the target bounding box. The position of the highlight rectangle and the page scroll position are determined by the front-end pixel areas that are consistent with the verification, and the positioning and presentation data are obtained.

10. The AI-driven method for precise content location in PDF documents as described in claim 9, characterized in that: The generated PDF document content location results specifically include: Based on the positioning data, the target pixel area is determined in the PDF rendering page to obtain the target display area; Draw highlight marks in the target display area, and adjust the display position of the PDF rendering page according to the page scroll position to obtain the target centered page; When new target content is received, the document parsing data, candidate location results, verification bounding box sequence, target bounding box, and location presentation data are called to redetermine the target display area. The highlighted range is determined according to the PDF original text bounding box corresponding to keywords, sentence fragments, field names, or field values ​​to obtain the PDF document content location result.

Images