A PDF document intelligent segmentation method and system based on cross-page semantic state transmission

Abstract

The application provides a PDF document intelligent segmentation method and system based on cross-page semantic state transmission. The method comprises: obtaining a to-be-processed PDF file and converting it into page images sorted by pages, preprocessing the page images to obtain standardized page images and page meta information; inputting the standardized page images into a visual language model, combining OCR-recognized text and layout structure information to generate page multi-modal representation, page abstract and key entities; initializing a current document cluster and cross-page semantic state based on the first page; calculating the continuity score of the to-be-judged page with the current document cluster, and performing incorporation, new document start or gray page review accordingly; performing bridge attribution judgment on the abnormal page, combining initial boundary information and corrected boundary information to perform global boundary optimization, obtaining a final document boundary sequence and outputting a target document segment. The application can improve the accuracy of complex PDF document boundary recognition and intelligent segmentation.

Description

Technical Field

[0001] This application relates to the field of intelligent document processing technology, and more specifically, to a method and system for intelligent PDF document segmentation based on cross-page semantic state transmission. Background Technology

[0002] In scenarios such as reviewing bidding documents, organizing credit information, archiving contracts, digitizing archives, reviewing financial risk control materials, and transferring government documents, it is often necessary to organize lengthy PDF documents from diverse sources. These PDF documents are usually formed from scanned copies, converted electronic documents, or materials from multiple sources. A single PDF file may contain multiple independent documents such as contracts, authorization letters, credit certificates, audit reports, invoices, business licenses, attachments, and explanatory letters. Therefore, it is necessary to accurately segment them for subsequent archiving, retrieval, information extraction, and business review.

[0003] Existing technologies typically employ OCR text rules, fixed template matching, or single-page classification for PDF segmentation. OCR text rule-based methods primarily determine document boundaries based on title keywords, numbering, page numbers, fixed fields, or preset regular expressions. Template matching or single-page classification methods mainly identify page types based on page layout, table placement, headers and footers, signature areas, or image features, and then infer segmentation locations accordingly. These methods are effective for PDFs with clear text, fixed templates, and stable page quality, but they struggle to consistently identify complex PDF boundaries.

[0004] Therefore, this application provides a method and system for intelligent PDF document segmentation based on cross-page semantic state transfer to solve one of the aforementioned technical problems. Summary of the Invention

[0005] The purpose of this application is to provide a method and system for intelligent PDF document segmentation based on cross-page semantic state transfer, which can solve at least one of the technical problems mentioned above. The specific solution is as follows: According to a specific implementation of this application, in a first aspect, this application provides a method for intelligent PDF document segmentation based on cross-page semantic state transfer, including: The PDF file to be processed is obtained and converted into page images sorted by page. The page images are preprocessed to obtain standardized page images and page metadata. Standardized page images are input into a visual language model, and combined with OCR to recognize text and layout structure information, to generate multimodal representations, page summaries, and key entities for each page; Based on the page multimodal representation, page summary, key entities and page meta information of the first page, the current document cluster and the corresponding cross-page semantic state are initialized. The cross-page semantic state is used to characterize the aggregation semantics, historical entities and layout template information of the current document cluster in the multi-page range. For the next page to be judged, a continuity score is calculated based on the page multimodal representation, page summary, key entities and page meta information of the next page to be judged, and the cross-page semantic state corresponding to the current document cluster. The continuity score is used to characterize the consistency between the page to be judged and the overall context of the current document cluster. Based on the continuity score, the next page to be judged is merged into the current document cluster, used as the starting page of a new document, or used as a gray area page for review, and the cross-page semantic state is updated or reconstructed accordingly to recursively maintain the cross-page semantic state around the current document cluster and obtain the initial boundary information; For the abnormal pages identified by the initial boundary information, perform a bridging attribution determination to obtain the corrected boundary information; Global boundary optimization is performed based on the initial boundary information and the corrected boundary information to obtain the final document boundary sequence, and the target document segment is output.

[0006] Preferably, the page image is preprocessed to obtain a standardized page image and page metadata, including: After rendering each page of the PDF file to be processed into a page image, page resolution unification, edge cropping, orientation correction, tilt correction, noise reduction, shadow removal and page area standardization are performed on the page image to obtain a standardized page image; Based on the rendering and preprocessing results, record the page number, rotation angle, sharpness, resolution, and image noise level to obtain page metadata.

[0007] Preferably, standardized page images are input into a visual language model, and combined with OCR recognition of text and layout structure information, multimodal representations, page summaries, and key entities for each page are generated, including: The standardized page image is input into the visual language model to obtain image encoding features, page summary, and document type candidate results. Perform OCR recognition on a standardized page image to obtain OCR-recognized text, and encode the OCR-recognized text to obtain text encoding features; Perform layout area detection on standardized page images to obtain layout structure information for title area, header and footer, table area, paragraph area, signature area and signature area; By fusing image encoding features, text encoding features, layout structure information, and page metadata, a multimodal representation of the page is obtained. Key entities are formed by extracting document title, document type, number, project name, subject name, date, amount, page number, and attachment identifier from page summary, document type candidate results, and OCR-recognized text.

[0008] Preferably, based on the page multimodal representation, page summary, key entities, and page metadata of the first page, the current document cluster and its corresponding cross-page semantic state are initialized, including: The first page is taken as the first page of the current document cluster. The multimodal representation of the first page is taken as the initial aggregate semantic representation of the current document cluster. The page summary of the first page is written into the historical page summary cache. The key entities of the first page are written into the historical key entity set. Page number evolution information is generated based on the page meta information of the first page to obtain the initial cross-page semantic state. When the page to be judged is determined to be the starting page of the new document, the new current document cluster and the new cross-page semantic state are reconstructed in the same way.

[0009] Preferably, the cross-page semantic state is updated in the following manner: When the page to be judged is merged into the current document cluster, the page multimodal representation, page summary and key entities of the page to be judged are encoded into entity summary encoding vectors, and gating updates are performed in combination with the cross-page semantic state before merging to obtain the updated cross-page semantic state. Among them, the gating update satisfies: ; ; ; in, Indicates the first Cross-page semantic state after page merging Indicates the first Cross-page semantic state before page merging, Indicates the first Multimodal representation of a page. Indicates the first The entity summary encoding vector of a page. This indicates a state inheritance gate variable. and Represents the weight matrix. and Indicates the bias term. This indicates element-wise multiplication.

[0010] Preferably, the continuity score is calculated according to the following formula: ; in, Indicates the first Page continuity score relative to the current document family Indicates semantic continuity, Indicates structural continuity. Indicates entity consistency. Indicates visual template continuity. This indicates logical consistency in page numbers. Indicates conflict penalty items, to and Indicates the weighting parameter; Semantic continuity is calculated by combining the multimodal representation of the page to be judged with the aggregated semantic representation of the current document cluster; structural continuity is calculated by combining the layout structure information of the page to be judged with the structural template representation of the current document cluster; entity consistency is calculated by combining the key entities of the page to be judged with the historical key entity set of the current document cluster; visual template continuity is calculated by combining the image encoding features of the page to be judged with the visual template representation of the current document cluster; page number logical consistency is calculated by combining the page meta-information of the page to be judged with the page number evolution information of the current document cluster; and conflict penalty items are generated based on title mutation, subject mutation, number mutation, page number anomaly, and layout jump.

[0011] Preferably, based on the continuity score, the next page to be judged is reviewed by merging it into the current document cluster, using it as the starting page of a new document, or using it as a gray area page, including: Set the first threshold. Second threshold ,and ; For continuous scoring : when When needed, the page to be judged will be incorporated into the current document family; when When this happens, the page to be judged will be used as the starting page of the new document; when When the page to be judged is designated as a gray area page, the comparison window is expanded for pages within a preset number of pages before and after the gray area page. The title area, header area, footer area, signature area, and signature area of ​​the gray area page are partially re-identified. The gray area page is determined by cross-validating the OCR channel recognition results with the visual language model channel recognition results.

[0012] Preferably, bridging attribution determination is performed on the abnormal pages identified by the initial boundary information to obtain corrected boundary information, including: Based on the amount of text, image clarity, proportion of the signature area, and proportion of effective content in the main body of the page, blank pages, pages containing only seals, pages containing only signatures, pages with seals across the binding, blurred pages, and pages with partial images are identified as abnormal pages from the page sequence. Calculate the continuity scores of the pages before, after, and before the abnormal page relative to the relevant document clusters. When both the pages before and after the abnormal page meet the continuity condition with the same document cluster, and the signature position, page number information, or residual key entity of the abnormal page matches the document cluster, the abnormal page is bridged and merged into the document cluster and correction boundary information is generated. When the pages preceding and following the abnormal page meet the continuity condition with different document clusters, the abnormal page is identified as a boundary adjacent page or an independent transition page, and correction boundary information is generated.

[0013] Preferably, global boundary optimization is performed based on the initial boundary information and the corrected boundary information to obtain the final document boundary sequence, including: Based on the initial boundary information and the corrected boundary information, the following pages are identified as candidate boundary points: the new document start page, the gray zone page with a confidence level lower than the preset confidence threshold after verification, the title mutation page, the main body mutation page, the page number abnormal page, and the layout jump page. The continuous page intervals between candidate boundary points are defined as candidate document segments, and the interval scores of the candidate document segments are calculated. Based on interval scores, a dynamic programming formula is used to determine the first... The optimal segmentation score of the page is obtained, and the candidate document segments corresponding to the optimal segmentation score are backtracked to obtain the final document boundary sequence. The interval score is calculated using the following formula: ; The dynamic programming formula is expressed as follows: ; in, Indicates the first Pages to number The interval score of the page constituting the candidate document segment. Indicates the conflict cost within the candidate document segment. Indicates the conflict cost weight. Indicates the preceding The optimal segmentation score after the page is divided.

[0014] According to a specific embodiment of this application, in a second aspect, this application provides a PDF document intelligent segmentation system based on cross-page semantic state transfer, comprising: The page preprocessing module is used to acquire the PDF file to be processed and convert it into page images sorted by page. The page images are preprocessed to obtain standardized page images and page metadata. The page understanding module is used to input standardized page images into the visual language model, combine OCR to recognize text and layout structure information, and generate multimodal representations, page summaries and key entities for each page; The state initialization module is used to initialize the current document cluster and its corresponding cross-page semantic state based on the page multimodal representation, page summary, key entities and page meta information of the first page. The cross-page semantic state is used to characterize the aggregate semantics, historical entities and layout template information of the current document cluster in the multi-page range. The continuity judgment module is used to calculate the continuity score for the next page to be judged based on the page multimodal representation, page summary, key entities and page meta information of the next page to be judged and the cross-page semantic state corresponding to the current document cluster. The continuity score is used to characterize the consistency between the page to be judged and the overall context of the current document cluster. The status update module is used to perform a review of the next page to be judged based on the continuity score, either by merging it into the current document cluster, using it as the starting page of a new document, or using it as a gray zone page, and to update or rebuild the cross-page semantic state accordingly, so as to recursively maintain the cross-page semantic state around the current document cluster and obtain the initial boundary information. The boundary correction module is used to perform bridging attribution determination on abnormal pages identified by the initial boundary information to obtain corrected boundary information; The results output module is used to perform global boundary optimization based on the initial boundary information and the corrected boundary information, obtain the final document boundary sequence, and output the target document segment.

[0015] Compared with the prior art, the beneficial effects of this application are as follows: This application provides an intelligent PDF document segmentation method based on cross-page semantic state transfer. By converting the PDF to be processed into standardized page images, and combining visual language models, OCR recognition of text and layout structure information, a multimodal representation of the page, page summary, and key entities are generated, enabling the page content to be comprehensively represented from visual, textual, and structural dimensions. Furthermore, by calculating the continuity score between the cross-page semantic state of the current document cluster and the page to be judged, and combining gray area verification, abnormal page bridging, and global boundary optimization, the impact of weak text pages, stamped pages, blank pages, and local misjudgments on the segmentation results can be reduced, improving the accuracy of document boundary recognition and the stability of long sequence segmentation in complex PDFs.

[0016] This application improves the accuracy of document boundary recognition on real-world business datasets such as bidding documents, credit tenders, and contract archives. By cross-page semantic state modeling and multi-evidence fusion, it enhances the continuity rate of documents in the presence of interference such as blank pages, stamped pages, and signature pages. Through gray zone verification, abnormal page bridging, and global optimization, it reduces the overall error rate in long documents of hundreds of pages or more. Attached Figure Description

[0017] Figure 1 A flowchart of a PDF document intelligent segmentation method based on cross-page semantic state transfer according to this application is shown; Figure 2 The overall flowchart of the intelligent PDF document segmentation method of this application is shown; Figure 3 This application shows a flowchart of the multimodal feature extraction, fusion, and summarization and entity extraction processes. Figure 4 A flowchart illustrating the construction and updating of cross-page semantic state is shown; Figure 5 This paper presents a flowchart illustrating the continuity scoring calculation process between the current page to be judged and the cross-page semantic state in this application. Figure 6 The flowchart of the gray area page review process for this application is shown; Figure 7 The flowchart for determining the bridging attribution of abnormal pages in this application is shown; Figure 8 The flowchart of the global boundary optimization of this application is shown; Figure 9 This diagram illustrates the result of segmenting a complex, long, scanned PDF into multiple target document segments. Figure 10 A block diagram of a PDF document intelligent segmentation system based on cross-page semantic state transfer according to an embodiment of this application is shown. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] The method provided in this application is applicable to complex PDF document processing in scenarios such as reviewing bidding documents, organizing credit information, archiving contracts, digitizing archives, reviewing financial risk control materials, and transferring government documents.

[0020] In related technologies, complex scanned PDF documents often exhibit issues such as the lack of reliable text layers, OCR errors, page tilting, blurring, shadows, seal obscuring, signature pages, blank pages, pages with seals across the binding, and similar templates across different documents. Existing solutions typically rely on single-page information or local similarity between adjacent pages, making it difficult to consistently represent the overall context of the currently aggregated documents. This leads to the easy missegmentation of middle pages, attachment pages, stamped pages, or pages with weak text, as well as the easy mismerging of different documents. Furthermore, local misjudgments can propagate continuously through long PDF sequences, affecting the accuracy and stability of the final segmentation results.

[0021] In view of this, this application provides a PDF document intelligent segmentation method based on cross-page semantic state transmission. This method does not segment documents based solely on single-page titles, keywords, templates, or the similarity of adjacent pages, but instead establishes cross-page semantic states around the current document cluster, allowing the next page to be judged to be compared with the overall context corresponding to the current document cluster. This method has better performance than the segmentation schemes provided in related technologies.

[0022] The optional embodiments of this application are described in detail below with reference to the accompanying drawings.

[0023] The embodiments provided in this application are embodiments of a PDF document intelligent segmentation method based on cross-page semantic state transmission.

[0024] The following is combined Figure 1 The embodiments of this application will be described in detail.

[0025] Figure 1 The flowchart of a PDF document intelligent segmentation method based on cross-page semantic state transfer according to this application is shown, as follows: Figure 1 As shown, it includes the following steps; S101. Obtain the PDF file to be processed and convert it into page images sorted by page. Preprocess the page images to obtain standardized page images and page metadata.

[0026] S102. Input the standardized page image into the visual language model, combine it with OCR to recognize the text and layout structure information, and generate the multimodal representation of each page, page summary and key entities.

[0027] S103. Based on the page multimodal representation, page summary, key entities and page meta information of the first page, initialize the current document cluster and the corresponding cross-page semantic state.

[0028] Among them, the cross-page semantic state is used to characterize the aggregate semantics, historical entities and layout template information of the current document cluster across multiple pages.

[0029] S104. For the next page to be judged, calculate the continuity score based on the page multimodal representation, page summary, key entities and page meta information of the next page to be judged, and the cross-page semantic state corresponding to the current document cluster.

[0030] Among them, the continuity score is used to characterize the consistency between the page to be judged and the overall context of the current document cluster.

[0031] S105. Based on the continuity score, the next page to be judged is merged into the current document cluster, used as the starting page of a new document, or used as a gray area page for review, and the cross-page semantic state is updated or reconstructed accordingly to recursively maintain the cross-page semantic state around the current document cluster and obtain the initial boundary information.

[0032] S106. Perform bridging attribution determination on the abnormal pages identified by the initial boundary information to obtain the corrected boundary information.

[0033] S107. Perform global boundary optimization based on the initial boundary information and the corrected boundary information to obtain the final document boundary sequence and output the target document segment.

[0034] Using the method provided in this application, the PDF file to be processed is divided into one or more target document segments after undergoing page standardization, page understanding, cross-page state construction, continuity judgment, anomaly correction, and global optimization.

[0035] Figure 2 The overall flowchart of the intelligent PDF document segmentation method of this application is shown.

[0036] like Figure 2As shown, after obtaining the PDF file to be processed, it is converted into page images sorted by page, and standardized page images and page metadata are obtained through preprocessing. Based on the standardized page images, page multimodal representations, page summaries, and key entities are generated. The current document cluster and its corresponding cross-page semantic state are initialized using the page multimodal representation, page summary, key entities, and page metadata of the first page. The cross-page semantic state is used to characterize the aggregation semantics, historical entities, and layout template information of the current document cluster across multiple pages. For the next page to be judged in the page sequence, a continuity score is calculated based on the page multimodal representation, page summary, key entities, and page metadata of the next page to be judged, and the cross-page semantic state corresponding to the current document cluster. The continuity score is used to characterize the consistency between the page to be judged and the overall context of the current document cluster. Based on the continuity score, it is determined whether the next page to be judged is incorporated into the current document cluster, used as the starting page of a new document, or used as a gray area page for review. After obtaining the initial boundary information, a bridging attribution judgment is performed on the abnormal pages identified by the initial boundary information to obtain the corrected boundary information. Then, global boundary optimization is performed based on the initial boundary information and the corrected boundary information to obtain the final document boundary sequence, and the target document segment is output according to the final document boundary sequence.

[0037] In some embodiments, a PDF file to be processed is acquired and converted into page-ordered page images. The page images are preprocessed to obtain standardized page images and page metadata, which are used to convert the PDF document into image data that can be directly processed in subsequent page understanding processes. The PDF file to be processed may include multiple pages. In this method, each page is rendered as a page image according to the original page order of the PDF file, and a correspondence between the page images and the original page numbers is established. Here, a page image refers to bitmap data rendered from PDF pages, and the page-ordered page images are used to maintain the page sequence in the PDF file.

[0038] As a specific embodiment, page image preprocessing yields standardized page images and page metadata, including: rendering each page of the PDF file to be processed as a page image, and then performing page resolution unification, edge cropping, orientation correction, skew correction, denoising, shadow removal, and page region standardization on the page images to obtain standardized page images; recording page numbers, rotation angles, sharpness, resolution, and image noise levels based on the rendering and preprocessing results to obtain page metadata. Edge cropping is used to remove scanned borders, black borders, or invalid blank areas; orientation correction is used to adjust inverted or rotated pages to a readable orientation; skew correction is used to adjust skewed scanned pages to a uniform orientation; denoising is used to reduce copy noise, scanning graininess, and compression artifacts; shadow removal is used to reduce page creases, photographic shadows, or scanning shadows; and page region standardization is used to adjust the main content of the page to a uniform coordinate range. The page metadata is not simply additional recording, but rather input data for subsequently generating multimodal page representations, constructing page number evolution information, and calculating page number logical consistency.

[0039] In some embodiments, standardized page images are used for page visual understanding, OCR recognition, and layout structure detection. Page metadata is used to reflect page quality, page orientation, and page numbering. For low-resolution pages, the sharpness and image noise level in the page metadata can help determine whether the page is blurry; for pages with abnormal page numbers, the page numbers in the page metadata can be cross-checked with the page number text obtained from OCR recognition; for rotated pages, the rotation angle can be used to determine whether orientation correction is needed for the OCR recognition results or layout structure detection results.

[0040] In some embodiments, standardized page images are input into a visual language model, which combines OCR recognition of text and layout structure information to generate multimodal representations, page summaries, and key entities for each page. This converts each page into computable, comparable, and cross-page state update-compatible page-level data. The visual language model can be deployed on servers, workstations, or cloud computing devices with image reasoning capabilities. Its input is a standardized page image, and its output includes image encoding features, page summaries, and document type candidate results. The image encoding features characterize the page's visual content and overall layout, the page summary describes the main content, and the document type candidate results indicate the likelihood that the page belongs to a contract, authorization letter, credit certificate, audit report, invoice, business license, attachment, or explanatory letter, etc.

[0041] Figure 3 The flowchart of multimodal feature extraction, fusion, and summarization and entity extraction for this application page is shown.

[0042] like Figure 3As shown, taking a standardized page image and page metadata as input, image encoding features, OCR-recognized text, text encoding features, and layout structure information are generated, respectively. After the standardized page image is input into the visual language model, image encoding features, page summary, and document type candidate results are obtained. The standardized page image undergoes OCR recognition to obtain OCR-recognized text, which is then encoded to obtain text encoding features. The standardized page image undergoes layout region detection to obtain layout structure information for the title area, header and footer, table area, paragraph area, signature area, and closing area. Image encoding features, text encoding features, layout structure information, and page metadata are fused to obtain a multimodal representation of the page. Furthermore, document title, document type, number, project name, main body name, date, amount, page number, and attachment identifiers are extracted from the page summary, document type candidate results, and OCR-recognized text to form key entities. Figure 3 As shown, the page multimodal representation, page summary, and key entities involved in this application are output results in parallel. The page multimodal representation is used for semantic continuity calculation and cross-page semantic state update, the page summary is used to generate entity summary encoding vector and target document segment summary, and the key entities are used for entity consistency calculation and document segment structured output.

[0043] As a feasible implementation, standardized page images are input into a visual language model, and combined with OCR-recognized text and layout structure information, multimodal representations, page summaries, and key entities for each page are generated. This includes: inputting standardized page images into a visual language model to obtain image encoding features, page summaries, and document type candidate results; performing OCR recognition on the standardized page images to obtain OCR-recognized text, and encoding the OCR-recognized text to obtain text encoding features; performing layout region detection on the standardized page images to obtain layout structure information for the title area, header and footer, table area, paragraph area, signature area, and closing area; fusing image encoding features, text encoding features, layout structure information, and page metadata to obtain a multimodal representation of the page; and extracting document title, document type, number, project name, main body name, date, amount, page number, and attachment identifier from the page summary, document type candidate results, and OCR-recognized text to form key entities.

[0044] In some embodiments, OCR-recognized text is text data obtained by recognizing text content in a standardized page image, and text encoding features are features obtained by vectorizing and encoding the OCR-recognized text. Layout structure information refers to the position, type, and distribution relationship of different functional areas on the page, specifically including the coordinates, category, and confidence level of the title area, header / footer, table area, paragraph area, signature area, and closing area. Key entities refer to business fields on the page that can be used to determine document ownership and cross-page continuity, such as document title, document type, number, project name, subject name, date, amount, page number, and attachment identifier. Page multimodal representation differs from page summary and key entities. Page multimodal representation is a page vector obtained by fusing image encoding features, text encoding features, layout structure information, and page metadata; page summary and key entities are page-level semantic results generated alongside page multimodal representation.

[0045] As one specific embodiment, the first Page multimodal representation Features can be encoded from images Text encoding features Layout structure information and page metadata The fusion process involves mapping features from different sources to a unified dimension before fusion. During fusion, the mapped features are concatenated and then fused to obtain the multimodal representation of the page. This process can be represented as: ; in, Indicates the first Multimodal representation of a page. Indicates the first Image encoding features of a page Indicates the first The text encoding characteristics of a page Indicates the first Page layout and structure information, Indicates the first Page metadata, , , and These represent the dimension mapping matrices for the corresponding features. This represents the fusion mapping weight matrix. This represents the fusion mapping bias term.

[0046] In some embodiments, the page summary may be generated by a visual language model based on a standardized page image, or by OCR-recognized text, or by a combination of visual language model output and OCR-recognized text. Key entities can be extracted from the page summary, document type candidate results, and OCR-recognized text. For contract pages, key entities may include the contract number, the name of Party A, the name of Party B, the project name, and the signing date; for creditworthiness verification pages, key entities may include the name of the verification object, the issuing institution, the verification date, and the verification number; for audit report pages, key entities may include the report title, the audited entity, the reporting year, the name of the accounting firm, and the report number.

[0047] In some embodiments, the current document cluster and its corresponding cross-page semantic state are initialized based on the page multimodal representation, page summary, key entities, and page metadata of the first page. The current document cluster is the target document segment being constructed during the sequential page processing, and the cross-page semantic state is the state expression of the current document cluster. The cross-page semantic state may include aggregate semantic representation, historical page summary cache, historical key entity set, document type probability distribution, structural template representation, visual template representation, and page number evolution information. The aggregate semantic representation is used to reflect the overall semantics of the current document cluster; the historical page summary cache is used to retain the summary information of each page within the current document cluster; the historical key entity set is used to store the key entities that have appeared within the current document cluster; the document type probability distribution is used to represent the probability that the current document cluster belongs to different document types; the structural template representation is used to characterize the layout structure features of the current document cluster; the visual template representation is used to characterize the visual layout features of the current document cluster; and the page number evolution information is used to record the continuity, absence, repetition, or closure relationship of page numbers.

[0048] Figure 4 A flowchart illustrating the construction and updating of cross-page semantic state is shown.

[0049] like Figure 4 As shown, when processing the first page or the starting page of a new document, the current document cluster and its corresponding cross-page semantic state are initialized with the page's multimodal representation, page summary, key entities, and page metadata. When a page to be judged is merged into the current document cluster, the page's multimodal representation, page summary, and key entities are encoded into an entity summary encoding vector, and then combined with the cross-page semantic state before merging for gating update to obtain the updated cross-page semantic state. The cross-page semantic state is not single-page data, but rather the state data formed by the current document cluster across multiple pages, including aggregate semantic representation, historical page summary cache, historical key entity set, structural template representation, and page number evolution information. Through this embodiment, the current document cluster can be recursively updated as pages are merged and reconstructed when a new document starting page appears, thereby avoiding the incorrect propagation of the state of the previous target document to the subsequent target document.

[0050] As a feasible implementation, based on the page multimodal representation, page summary, key entities, and page metadata of the first page, the current document cluster and its corresponding cross-page semantic state are initialized. This includes: using the first page as the first page of the current document cluster, using the page multimodal representation of the first page as the initial aggregate semantic representation of the current document cluster, writing the page summary of the first page into the historical page summary cache, writing the key entities of the first page into the historical key entity set, and generating page number evolution information based on the page metadata of the first page to obtain the initial cross-page semantic state. When the page to be determined is identified as the starting page of a new document, a new current document cluster and a new cross-page semantic state are reconstructed in the same way. Thus, when page processing starts from the first page, the first page forms the first current document cluster; when a subsequent page is identified as the starting page of a new document, that page no longer uses the cross-page semantic state of the previous document cluster, but is used to reconstruct a new current document cluster and a new cross-page semantic state.

[0051] In some embodiments, the cross-page semantic state is updated as the page is merged into the current document family. The cross-page semantic state is updated as follows: when a page to be determined is merged into the current document family, the page multimodal representation, page summary, and key entities of the page to be determined are encoded into an entity summary encoding vector. This vector is then combined with the cross-page semantic state before merging and a gating update is performed to obtain the updated cross-page semantic state. The entity summary encoding vector is a vector obtained by uniformly encoding the page summary and key entities, used to introduce the semantic description and business entities of the page into the cross-page semantic state update process. The gating update controls the fusion ratio between the old state and the current page information, enabling the cross-page semantic state to retain the existing context of the current document family while absorbing information from the newly merged page.

[0052] As a specific implementation, the gating update can satisfy the following formula: ; ; ; in, Indicates the first The cross-page semantic state after page merging Indicates the first Cross-page semantic state before page merging, Indicates the first Multimodal representation of a page. Indicates the first The entity summary encoding vector of the page, This indicates a state inheritance gate variable. and Represents the weight matrix. and Indicates the bias term. This represents the Sigmoid activation function. This represents the hyperbolic tangent activation function. This indicates element-wise multiplication. Indicates by the first Candidate cross-page semantic states generated from page information. In this embodiment, using... Representing candidate cross-page semantic states can prevent the candidate states from clashing with the final updated cross-page semantic state. Confusing.

[0053] In some embodiments, when the page to be determined has a high degree of continuity with the current document cluster, the state inheritance gating variable is used. This ensures that a high proportion of the cross-page semantic state before merging is retained in the update results; when the page to be judged shows a tendency for boundary changes, the state inherits the gated variable. By reducing the inheritance ratio of old states, the influence of current page information on candidate cross-page semantic states is enhanced. In this way, cross-page semantic states can maintain the contextual continuity of multi-page documents while also remaining sensitive to the starting page or boundary page of a new document.

[0054] In some embodiments, for the next page to be judged, a continuity score is calculated based on the page multimodal representation, page summary, key entities, and page metadata of the next page to be judged, and the cross-page semantic state corresponding to the current document cluster. The continuity score is used to characterize whether the next page to be judged and the current document cluster belong to the same target document. This continuity score is not a single similarity, but is obtained by combining semantic continuity, structural continuity, entity consistency, visual template continuity, page number logical consistency, and conflict penalty terms.

[0055] Figure 5 The flowchart illustrating the continuity scoring calculation process between the current page to be judged and the cross-page semantic state in this application is shown.

[0056] like Figure 5As shown, the process receives the page multimodal representation, page summary, key entities, page metadata, image encoding features, and layout structure information of the page to be judged, and also receives the cross-page semantic state corresponding to the current document cluster. Semantic continuity is calculated based on the page multimodal representation of the page to be judged and the aggregate semantic representation of the current document cluster; structural continuity is calculated based on the layout structure information of the page to be judged and the structural template representation of the current document cluster; entity consistency is calculated based on the key entities of the page to be judged and the historical key entity set of the current document cluster; visual template continuity is calculated based on the image encoding features of the page to be judged and the visual template representation of the current document cluster; and page number logical consistency is calculated based on the page metadata of the page to be judged and the page number evolution information of the current document cluster. Conflict penalty items are generated based on title abrupt changes, body abrupt changes, numbering abrupt changes, page number anomalies, and layout jumps. A continuity score is obtained by fusing these multiple scores. For example, in the process shown in the embodiment of this application, the page to be judged is no longer compared only with the previous page, but with the overall semantics, entities, layout, and page number state of the current document cluster.

[0057] As a feasible implementation, the continuity score is calculated according to the following formula: ; in, Indicates the first Page continuity score relative to the current document family Indicates semantic continuity, Indicates structural continuity. Indicates entity consistency. Indicates visual template continuity. This indicates logical consistency in page numbers. Indicates conflict penalty items, to and This represents the weight parameters. These weight parameters can be obtained through training with sample documents or pre-configured according to the business scenario.

[0058] In some embodiments, semantic continuity is calculated by the page multimodal representation of the page to be judged and the aggregate semantic representation of the current document cluster; structural continuity is calculated by the layout structure information of the page to be judged and the structural template representation of the current document cluster; entity consistency is calculated by the key entities of the page to be judged and the historical key entity set of the current document cluster; visual template continuity is calculated by the image encoding features of the page to be judged and the visual template representation of the current document cluster; page number logical consistency is calculated by the page meta-information of the page to be judged and the page number evolution information of the current document cluster; and conflict penalty terms are generated based on title mutation, subject mutation, number mutation, page number anomaly, and layout jump.

[0059] For example, although a middle page of a contract may lack a complete title, its subject name, contract number, page number evolution, and clause format may still be consistent with the current document cluster. Therefore, this page can obtain a high continuity score through entity consistency, structural continuity, and page number logic consistency.

[0060] For example, a new document's start page may be similar to the previous document template, but its title, body name, or numbering may change abruptly. In such cases, the conflict penalty item will lower its continuity score.

[0061] In some embodiments, based on the continuity score, the next page to be determined is reviewed by merging it into the current document cluster, using it as the starting page of a new document, or using it as a gray zone page, and the cross-page semantic state is updated or reconstructed accordingly to obtain initial boundary information. The initial boundary information refers to the page attribution and candidate boundary data obtained based on the continuity score and gray zone review results, and may include the position of the new document starting page, the range of pages to be merged into the current document cluster, the gray zone page identifier, the gray zone page review result, and the boundary confidence level.

[0062] Figure 6 The flowchart for the gray area page review process of this application is shown.

[0063] like Figure 6 As shown, based on continuous scoring With the first threshold Second threshold The relationship between the two criteria determines whether the page to be judged is a gray area page. When the continuity score is between the second threshold and the first threshold, the page to be judged is designated as a gray area page, and the gray area page review process is invoked. The gray area page review process includes expanding the page comparison window, re-identifying local key areas, cross-validating the recognition results of the OCR channel and the visual language model channel, and determining the gray area page's attribution. Expanding the page comparison window means including pages within a preset number of pages before and after the gray area page in the comparison scope; local key areas include the title area, header area, footer area, signature area, and closing area; cross-validation refers to comparing the recognition results of the OCR channel and the visual language model channel for the title, body, number, page number, and signature information. For example, the process shown in the embodiment of this application is used to form a more reliable page attribution judgment when the page boundaries are unclear.

[0064] As a specific implementation, based on the continuity score, the next page to be judged is reviewed by merging it into the current document cluster, using it as the starting page of a new document, or using it as a gray area page, including: setting a first threshold. Second threshold ,and For continuous scoring :when When, the page to be judged is incorporated into the current document family; when When the page to be judged is selected, the page to be judged will be used as the starting page of the new document; when When the page to be judged is designated as a gray area page, the comparison window is expanded to include pages within a preset number of pages before and after the gray area page. Local re-identification is performed on the title area, header area, footer area, signature area, and closing area of ​​the gray area page. The gray area page's classification is determined by cross-validating the OCR channel recognition results with the visual language model channel recognition results. Among these, the first threshold... Used to determine high-confidence consecutive pages, second threshold Used to determine the starting page of a new document with high confidence; pages between the two thresholds are considered to have uncertain boundaries.

[0065] In some embodiments, when When the page to be judged is incorporated into the current document family, its multimodal representation, summary, key entities, and metadata are used to update the cross-page semantic state; when When the page to be judged is used as the starting page of the new document, a new current document cluster and a new cross-page semantic state are reconstructed using its page multimodal representation, page summary, key entities, and page metadata; when During this process, the page to be judged is treated as a gray area page for verification. The expanded comparison window in the gray area page verification refers to including the gray area page and several pages before and after it in the continuity judgment; local re-recognition refers to re-recognizing the text and structure in the title area, header area, footer area, signature area, and closing area; channel cross-validation refers to comparing whether the OCR channel recognition results are consistent with the visual language model channel understanding results. In this embodiment, through this verification process, pages with unclear boundaries can be further determined as being merged into the current document cluster, the starting page of a new document, or a boundary-adjacent page.

[0066] In some embodiments, bridging attribution determination is performed on abnormal pages identified by initial boundary information to obtain corrected boundary information. An abnormal page refers to a page whose single-page information is insufficient to reliably determine its document attribution; specifically, it may include blank pages, pages containing only seals, pages containing only signatures, pages with seals across the binding, blurred pages, and pages with partial images. Corrected boundary information refers to boundary data obtained by correcting the initial boundary information after bridging attribution determination for abnormal pages. In this embodiment, the purpose of abnormal page bridging is to avoid incorrectly cutting consecutive documents due to insufficient text, weak semantics, or poor image quality on abnormal pages.

[0067] Figure 7 The flowchart for determining the bridging attribution of the abnormal page in this application is shown.

[0068] like Figure 7As shown, abnormal pages are identified based on text quantity, image clarity, the proportion of the signature area, and the proportion of effective content in the main body of the page. The continuity scores of the pages preceding, following, and after the abnormal page relative to the relevant document clusters are calculated. When both the preceding and following pages of the abnormal page satisfy the continuity condition with the same document cluster, and the signature position, page number information, or residual key entities of the abnormal page match the document cluster, the abnormal page is bridged and merged into the document cluster, and corrected boundary information is generated. When the preceding and following pages of the abnormal page satisfy the continuity condition with different document clusters, the abnormal page is determined as a boundary adjacent page or an independent transition page, and corrected boundary information is generated. In this embodiment, by combining the relationship between the preceding and following pages of the abnormal page and the available information of the abnormal page itself, erroneous boundaries are avoided directly due to the small amount of text on the abnormal page.

[0069] As a feasible embodiment, bridging attribution judgment is performed on abnormal pages identified by initial boundary information to obtain corrected boundary information, including: identifying blank pages, pages containing only seals, pages containing only signatures, pages with seals across the binding, blurred pages, and pages with partial images as abnormal pages from the page sequence based on text quantity, image clarity, proportion of signature area, and proportion of effective content in the main body area of ​​the page; calculating the continuity scores of the page before the abnormal page, the abnormal page, and the page after the abnormal page relative to the relevant document cluster; when the page before the abnormal page and the page after the abnormal page both satisfy the continuity condition with the same document cluster, and the signature position, page number information, or residual key entity of the abnormal page matches the document cluster, the abnormal page is bridged and merged into the document cluster and corrected boundary information is generated; when the page before the abnormal page and the page after the abnormal page satisfy the continuity condition with different document clusters, the abnormal page is determined as a boundary adjacent page or an independent transition page and corrected boundary information is generated.

[0070] In some embodiments, the text quantity is used to determine whether a page is a weak text page or a blank page; image clarity is used to determine whether a page is a blurry page; the proportion of the signature area is used to determine whether a page contains only a seal, only a signature, or a seal across the binding; and the proportion of effective content in the main body of the page is used to determine whether a page contains only partial image information. When there is a ninth page containing only a seal and signature between the eighth and tenth pages of a contract, if only a single-page summary is relied upon, the ninth page may be misjudged as a document boundary. When using the abnormal page bridging attribution judgment, if the eighth, ninth, and tenth pages match the same current document cluster in terms of signature position, page number information, or residual key entities, the ninth page is bridged and merged into the current document cluster. Thus, an abnormal page will not directly trigger erroneous segmentation due to its own insufficient text.

[0071] In some embodiments, global boundary optimization is performed based on initial boundary information and corrected boundary information to obtain the final document boundary sequence, and the target document segment is output. Global boundary optimization is used to comprehensively consider local judgment results and global consistency across the entire PDF, preventing local misjudgments from propagating continuously in long PDF documents. The final document boundary sequence refers to the set of page boundaries used to determine the start and end pages of each target document segment. The target document segment is the set of document pages obtained by segmenting the PDF file to be processed according to the final document boundary sequence.

[0072] Figure 8 The flowchart of the global boundary optimization of this application is shown.

[0073] like Figure 8 As shown, the system receives initial boundary information and corrected boundary information, and determines candidate boundary points based on the new document's starting page, pages with confidence levels below a preset confidence threshold after verification (gray area pages), pages with abrupt title changes, pages with abrupt body changes, pages with abnormal page numbers, and pages with layout jumps. Continuous page intervals between candidate boundary points are defined as candidate document segments, and interval scores are calculated based on continuity scores and conflict costs within these segments. A dynamic programming approach is then used to calculate the initial boundary score. The optimal segmentation score of the page is obtained, and the candidate document segments corresponding to the optimal segmentation score are backtracked to obtain the final document boundary sequence. Figure 8 The process shown uses a global optimization approach to uniformly correct the local boundary judgment results, so that the final document boundary sequence simultaneously satisfies local continuity and global consistency.

[0074] As a specific embodiment, global boundary optimization is performed based on initial boundary information and corrected boundary information to obtain the final document boundary sequence, including: identifying candidate boundary points as the new document start page, pages with a confidence level lower than a preset confidence threshold after verification of gray zone pages, pages with title abrupt changes, pages with body abrupt changes, pages with abnormal page numbers, and pages with layout jumps, based on the initial boundary information and corrected boundary information; identifying continuous page intervals between candidate boundary points as candidate document segments; calculating the interval score of the candidate document segments; and using a dynamic programming formula based on the interval score to determine the front document boundary sequence. The optimal segmentation score of the page is obtained, and the candidate document segments corresponding to the optimal segmentation score are backtracked to obtain the final document boundary sequence.

[0075] As a feasible implementation, the interval score is calculated using the following formula: ; The dynamic programming formula is expressed as follows: ; in, Indicates the first Pages to The interval score of the page constituting the candidate document segment. Indicates the conflict cost within the candidate document segment. Indicates the conflict cost weight. Indicates the preceding The optimal partitioning score after the page is divided. Among these, the conflict cost... It can be generated based on the number of title mutations, the degree of conflict in the main name, the degree of conflict in the numbering, the degree of page number logic anomalies, and the degree of mutation in the layout structure within the candidate document segment. In this embodiment of the application, through this global boundary optimization, the final document boundary sequence does not only rely on the local judgment of a single page, but also comprehensively selects a more consistent boundary division result across the entire PDF.

[0076] In some embodiments, the target document segment may include a start page number, an end page number, a document type label, a document summary, key entity information, and boundary confidence. The document type label can be obtained by combining the document type candidate results and page summaries of each page within the target document segment; the document summary can be obtained by aggregating the page summaries within the target document segment; the key entity information can be obtained by merging, deduplicating, and conflict-checking the key entities of each page within the target document segment; and the boundary confidence can be generated based on continuity scoring, gray area verification results, abnormal page bridging results, and global boundary optimization score.

[0077] Figure 9 This diagram illustrates the result of segmenting a complex, long, scanned PDF into multiple target document segments.

[0078] like Figure 9 As shown, the PDF file to be processed is divided into multiple target document segments after final document boundary sequence partitioning. Each target document segment corresponds to a start page number and an end page number, and can be associated with document type tags, document summaries, key entity information, and boundary confidence scores. For example, if the PDF file to be processed contains a business license, bank credit certificate, audit report, contract, authorization letter, and attachments in sequence, the result shown can output the above different types of materials as target document segments respectively. In this embodiment, the target document segment is not only the page range, but also includes a structured result composed of page summaries, key entities, and boundary confidence scores, thereby providing a data foundation for subsequent archiving, retrieval, information extraction, and business review.

[0079] As one application example, the PDF file to be processed is a credit reference document of approximately 300 pages. Pages 20 to 31 are consecutive pages of the same contract, and page 32 is the starting page of another document. After generating a multimodal representation of each page, a page summary, and key entities, the system recursively maintains the cross-page semantic state of the current document cluster around pages 20 to 31. When processing page 32, due to a sudden change in the document title, subject name, number format, or page number evolution information relative to the current document cluster, the continuity score decreases, and page 32 is determined as the starting page of a new document. This achieves automatic segmentation of adjacent independent documents in the long-length credit reference PDF.

[0080] As a second application embodiment, pages 8 and 10 of the contract in the PDF file to be processed meet the continuity condition with the same contract document cluster, while page 9 only contains stamp or signature information. After identifying page 9 as an abnormal page, the system does not directly form the document boundary based on the weak text information of page 9. Instead, it combines the continuity scores of pages 8, 9, and 10 with respect to the same document cluster, and makes a bridging attribution judgment based on the signature position, page number information, or residual key entities in page 9. When the above information matches the contract document cluster, page 9 is bridged and merged into the same contract document segment.

[0081] As an application example three, in the gray zone page review scenario, the page to be judged has low semantic similarity to the current document cluster, but its key entities, main names, or layout template information are still partially consistent with the current document cluster, resulting in a continuity score between the first and second thresholds. The system marks the page to be judged as a gray zone page, expands the comparison window for pages within a preset number of pages before and after the gray zone page, performs local re-recognition of the title area, header area, footer area, signature area, and closing area, and then cross-validates the OCR channel recognition result with the visual language model channel recognition result; if the review result shows that it is not continuous with the current document cluster, then the gray zone page is confirmed as the starting page of the new document.

[0082] As a specific application example, the PDF file to be processed is a credit information document of approximately 300 pages, which includes a mix of business licenses, bank credit certificates, audit reports, contracts, authorization letters, and attachments. This application converts the PDF file page by page into page images and preprocesses them to generate standardized page images and page metadata. For each page, a multimodal representation, page summary, and key entities are generated. When processing a continuous page range within a contract document, pages within that range maintain continuity in terms of subject name, contract number, clause structure, and page number evolution, and are therefore merged into the same current document cluster. When the next page introduces a new title, a different subject name, or a new numbering format, the continuity score decreases, and that page is used as the starting page of a new document, reconstructing a new current document cluster and a new cross-page semantic state. For pages in the middle of the contract containing only signatures, these pages are identified as abnormal pages and are bridged and merged into the contract document based on the continuity relationship between the pages before and after them and the same document cluster, as well as the signature position, page number information, or residual key entities. After global boundary optimization, the PDF file is segmented into multiple target document segments.

[0083] This application also provides system embodiments that follow the above embodiments, for implementing the method steps of the above embodiments. The interpretation of the same names is the same as that of the above embodiments, and they have the same technical effects as those of the above embodiments, so they will not be repeated here.

[0084] It should be understood that this application also includes the following variations without affecting the basic technical ideas of this application, including: page summaries can be generated directly by a visual language model, or by an OCR text summarization model, or by a combination of both. Continuity scoring weights can be fixed or adaptively learned through training a model. Global boundary optimization can be performed offline or online with a sliding window. Page features can be based solely on images or combined with OCR text and layout detection results. Thresholds can be fixed or dynamically adjusted according to the predicted document type. Key entities can use general fields or set dedicated fields for document types such as contracts, invoices, audit reports, and credit certificates.

[0085] like Figure 10 As shown, this application provides a PDF document intelligent segmentation system based on cross-page semantic state transfer, including: The page preprocessing module 1001 is used to acquire the PDF file to be processed and convert it into page images sorted by page, and preprocess the page images to obtain standardized page images and page metadata. The page understanding module 1002 is used to input standardized page images into the visual language model, combine OCR to recognize text and layout structure information, and generate multimodal representations, page summaries and key entities for each page; The state initialization module 1003 is used to initialize the current document cluster and its corresponding cross-page semantic state based on the page multimodal representation, page summary, key entities and page meta information of the first page. The cross-page semantic state is used to characterize the aggregation semantics, historical entities and layout template information of the current document cluster in the multi-page range. The continuity judgment module 1004 is used to calculate the continuity score for the next page to be judged based on the page multimodal representation, page summary, key entities and page meta information of the next page to be judged and the cross-page semantic state corresponding to the current document cluster. The continuity score is used to characterize the consistency between the page to be judged and the overall context of the current document cluster. The status update module 1005 is used to perform a review of the next page to be judged based on the continuity score, either by merging it into the current document cluster, using it as the starting page of a new document, or using it as a gray zone page, and to update or rebuild the cross-page semantic state accordingly, so as to recursively maintain the cross-page semantic state around the current document cluster and obtain the initial boundary information. The boundary correction module 1006 is used to perform bridging attribution determination on the abnormal pages identified by the initial boundary information to obtain the corrected boundary information; The result output module 1007 is used to perform global boundary optimization based on the initial boundary information and the corrected boundary information, obtain the final document boundary sequence, and output the target document segment.

[0086] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0087] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for intelligent PDF document segmentation based on cross-page semantic state transfer, characterized in that, include: The PDF file to be processed is obtained and converted into page images sorted by page. The page images are preprocessed to obtain standardized page images and page metadata. The standardized page images are input into the visual language model, and combined with OCR to recognize text and layout structure information, to generate multimodal representations, page summaries, and key entities for each page; Based on the page multimodal representation, page summary, key entities and page meta information of the first page, the current document cluster and the corresponding cross-page semantic state are initialized. The cross-page semantic state is used to characterize the aggregate semantics, historical entities and layout template information of the current document cluster in the multi-page range. For the next page to be judged, a continuity score is calculated based on the page multimodal representation, page summary, key entities and page meta information of the next page to be judged, and the cross-page semantic state corresponding to the current document cluster. The continuity score is used to characterize the consistency between the page to be judged and the overall context of the current document cluster. Based on the continuity score, the next page to be judged is merged into the current document cluster, used as the starting page of a new document, or used as a gray zone page for review, and the cross-page semantic state is updated or reconstructed accordingly, so as to recursively maintain the cross-page semantic state around the current document cluster and obtain the initial boundary information; Perform a bridging attribution determination on the abnormal pages identified by the initial boundary information to obtain corrected boundary information; Global boundary optimization is performed based on the initial boundary information and the corrected boundary information to obtain the final document boundary sequence, and the target document segment is output.

2. The intelligent PDF document segmentation method according to claim 1, characterized in that, The preprocessing of the page image to obtain a standardized page image and page metadata includes: After rendering each page of the PDF file to be processed into a page image, page resolution unification, edge cropping, orientation correction, tilt correction, noise reduction, shadow removal and page area standardization are performed on the page image to obtain a standardized page image; Based on the rendering and preprocessing results, record the page number, rotation angle, sharpness, resolution, and image noise level to obtain page metadata.

3. The intelligent PDF document segmentation method according to claim 1, characterized in that, The process of inputting the standardized page image into a visual language model, combining OCR recognition of text and layout structure information, and generating multimodal representations, page summaries, and key entities for each page includes: The standardized page image is input into a visual language model to obtain image encoding features, the page summary, and document type candidate results. OCR recognition is performed on the standardized page image to obtain OCR-recognized text, and the OCR-recognized text is encoded to obtain text encoding features; The standardized page image is subjected to layout area detection to obtain the layout structure information of the title area, header and footer, table area, paragraph area, signature area and signature area; The image encoding features, text encoding features, layout structure information, and page metadata are fused to obtain the multimodal representation of the page; The document title, document type, number, project name, subject name, date, amount, page number, and attachment identifier are extracted from the page summary, the document type candidate results, and the OCR-recognized text to form the key entity.

4. The intelligent PDF document segmentation method according to claim 1, characterized in that, The initialization of the current document cluster and its corresponding cross-page semantic state based on the page multimodal representation, page summary, key entities, and page metadata of the first page includes: The first page is taken as the first page of the current document cluster. The multimodal representation of the first page is taken as the initial aggregate semantic representation of the current document cluster. The page summary of the first page is written into the historical page summary cache. The key entities of the first page are written into the historical key entity set. Page number evolution information is generated based on the page meta information of the first page to obtain the initial cross-page semantic state. When the page to be judged is determined to be the starting page of the new document, the new current document cluster and the new cross-page semantic state are reconstructed in the same way.

5. The intelligent PDF document segmentation method according to claim 1, characterized in that, The cross-page semantic state is updated in the following manner: When the page to be judged is merged into the current document cluster, the page multimodal representation, page summary and key entities of the page to be judged are encoded into entity summary encoding vectors, and gating updates are performed in combination with the cross-page semantic state before merging to obtain the updated cross-page semantic state. Wherein, the gating update satisfies: ； ； ； in, Indicates the first Cross-page semantic state after page merging Indicates the first Cross-page semantic state before page merging, Indicates the first Multimodal representation of a page. Indicates the first The entity summary encoding vector of a page. This indicates a state inheritance gate variable. and Represents the weight matrix. and Indicates the bias term. This indicates element-wise multiplication.

6. The intelligent PDF document segmentation method according to claim 1, characterized in that, The continuity score is calculated according to the following formula: ； in, Indicates the first Page continuity score relative to the current document family Indicates semantic continuity, Indicates structural continuity. Indicates entity consistency. Indicates visual template continuity. This indicates logical consistency in page numbers. Indicates conflict penalty items, to and Indicates the weighting parameter; Semantic continuity is calculated by combining the multimodal representation of the page to be judged with the aggregated semantic representation of the current document cluster; structural continuity is calculated by combining the layout structure information of the page to be judged with the structural template representation of the current document cluster; entity consistency is calculated by combining the key entities of the page to be judged with the historical key entity set of the current document cluster; visual template continuity is calculated by combining the image encoding features of the page to be judged with the visual template representation of the current document cluster; page number logical consistency is calculated by combining the page meta-information of the page to be judged with the page number evolution information of the current document cluster; and conflict penalty items are generated based on title mutation, subject mutation, number mutation, page number anomaly, and layout jump.

7. The intelligent PDF document segmentation method according to claim 1, characterized in that, The step of reviewing the next page to be judged based on the continuity score, including merging it into the current document cluster, using it as the starting page of a new document, or using it as a gray area page, includes: Set the first threshold. Second threshold ,and ; For continuous scoring : when When needed, the page to be judged will be incorporated into the current document family; when When this happens, the page to be judged will be used as the starting page of the new document; when When the page to be judged is designated as a gray area page, the comparison window is expanded for pages within a preset number of pages before and after the gray area page. The title area, header area, footer area, signature area, and signature area of ​​the gray area page are partially re-identified. The gray area page is determined by cross-validating the OCR channel recognition results with the visual language model channel recognition results.

8. The intelligent PDF document segmentation method according to claim 1, characterized in that, The step of performing a bridging attribution determination on the abnormal pages identified by the initial boundary information to obtain corrected boundary information includes: Based on the amount of text, image clarity, proportion of the signature area, and proportion of effective content in the main body of the page, blank pages, pages containing only seals, pages containing only signatures, pages with seals across the binding, blurred pages, and pages with partial images are identified as abnormal pages from the page sequence. Calculate the continuity scores of the pages before, after, and before the abnormal page relative to the relevant document clusters. When both the pages before and after the abnormal page meet the continuity condition with the same document cluster, and the signature position, page number information, or residual key entity of the abnormal page matches the document cluster, the abnormal page is bridged and merged into the document cluster and correction boundary information is generated. When the pages preceding and following the abnormal page meet the continuity condition with different document clusters, the abnormal page is identified as a boundary adjacent page or an independent transition page, and correction boundary information is generated.

9. The intelligent PDF document segmentation method according to claim 1, characterized in that, The step of performing global boundary optimization based on initial boundary information and corrected boundary information to obtain the final document boundary sequence includes: Based on the initial boundary information and the corrected boundary information, the following pages are identified as candidate boundary points: the new document start page, the gray zone page with a confidence level lower than the preset confidence threshold after verification, the title mutation page, the main body mutation page, the page number abnormal page, and the layout jump page. The continuous page intervals between candidate boundary points are defined as candidate document segments, and the interval scores of the candidate document segments are calculated. Based on the interval scores, a dynamic programming formula is used to determine the preceding intervals. The optimal segmentation score of the page is obtained, and the candidate document segments corresponding to the optimal segmentation score are backtracked to obtain the final document boundary sequence. The interval score is calculated using the following formula: ； The dynamic programming formula is expressed as follows: ； in, Indicates the first Pages to The interval score of the page constituting the candidate document segment. This represents the conflict cost within a candidate document segment. Indicates the conflict cost weight. Indicates the preceding The optimal segmentation score after the page is divided.

10. A PDF document intelligent segmentation system based on cross-page semantic state transmission, characterized in that, include: The page preprocessing module is used to acquire the PDF file to be processed and convert it into page images sorted by page, and preprocess the page images to obtain standardized page images and page metadata; The page understanding module is used to input the standardized page image into the visual language model, and combine it with OCR to recognize text and layout structure information to generate multimodal representations, page summaries and key entities for each page; The state initialization module is used to initialize the current document cluster and its corresponding cross-page semantic state based on the page multimodal representation, page summary, key entities and the page meta information of the first page. The cross-page semantic state is used to characterize the aggregate semantics, historical entities and layout template information of the current document cluster in the multi-page range. The continuity judgment module is used to calculate a continuity score for the next page to be judged, based on the page multimodal representation, page summary, key entities and page meta information of the next page to be judged, and the cross-page semantic state corresponding to the current document cluster. The continuity score is used to characterize the consistency between the page to be judged and the overall context of the current document cluster. The status update module is used to perform the following checks on the next page to be judged based on the continuity score: merge it into the current document cluster, use it as the starting page of a new document, or use it as a gray zone page. The module also updates or reconstructs the cross-page semantic state accordingly, so as to recursively maintain the cross-page semantic state around the current document cluster and obtain the initial boundary information. The boundary correction module is used to perform bridging attribution determination on the abnormal pages identified by the initial boundary information to obtain the corrected boundary information; The result output module is used to perform global boundary optimization based on the initial boundary information and the corrected boundary information to obtain the final document boundary sequence and output the target document segment.

Images