Automatic document image segmentation method and system based on layout partition and sequence constraint

Abstract

The application discloses a kind of based on layout partition and sequence constraint's archive image automatic division method and system, belong to archive image processing technical field.The method obtains the archive image and division rule according to page sequence, determines title area, text area and footer area using layout analysis result, and carries out optical character recognition and text normalization respectively;According to the matching candidate piece class of the exclusion relationship of the text condition chain in area, cross-area containing relationship and merged area text, and combine previous page division state, last page number, consecutive page number and same volume occurrence number Sequence constraints determine each page piece name and division boundary;Finally, the corresponding relationship of page and target piece class and determination basis are output.The application can reduce the interference of whole page unrelated text, and keep the page sequence consistency of continuous archive material, suitable for automatic division and review of scanned archives.

Description

Technical Field

[0001] This invention relates to the fields of archival image processing, document layout analysis, and optical character recognition, and particularly to an automatic segmentation method and system for archival images based on layout partitioning and sequence constraints. Background Technology

[0002] Digitizing archives typically requires scanning paper files into page-by-page images, then dividing consecutive pages into multiple document files according to material type such as applications, approval forms, contracts, identification documents, drawings, and vouchers. In current practices, the boundaries and names of these file divisions are usually determined manually by operators, page by page. This process is susceptible to variations in speed and consistency due to factors such as operator experience, scan quality, and differences in material format.

[0003] Existing automatic classification schemes often use full-page optical character recognition (OCR) text for keyword matching or page-by-page general image classification models. Titles, body text, and footers on archival pages have different discriminative functions. Full-page text recognition is easily mixed with interfering content such as clauses, seals, headers, and page numbers; furthermore, subsequent pages of the same material may no longer contain the material title, making it easy to break up continuous materials based solely on single-page content. On the other hand, similar archives often exhibit sequential patterns such as consecutive page numbers, last page number, and frequency of occurrence within the same volume, which existing single-page classification schemes struggle to utilize simultaneously.

[0004] Therefore, there is a need for an automatic component segmentation technology that can combine the local recognition results of page layout areas with the page sequence status within a volume, in order to improve the stability of component name and boundary judgment in complex scanned archives and retain verifiable judgment criteria. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for automatic segmentation of archival images based on page layout and sequence constraints, in order to solve the problems of full-page text matching being easily interfered with by irrelevant content, continuation pages lacking clear titles, and single-page classification results lacking consistency in the order within the volume.

[0006] To achieve the above objectives, the present invention adopts the following technical solution.

[0007] An automatic segmentation method for archival images based on page layout and sequence constraints is executed by an electronic device. First, a sequence of archival images arranged in page order within a volume and a set of segmentation rules are acquired. The segmentation rules are configured with a starting page and optional ending page conditions according to the target document type. Text conditions are associated with the title area, body text area, or footer area, and at least one sequence constraint is configured from the following: maximum consecutive page count, minimum page count, maximum occurrence of one segment within the same volume, and page number determination.

[0008] For each page of the archival image, layout analysis is used to obtain the category, confidence level, reading order, and bounding box coordinates of layout elements. The title area, body text area, and footer area are determined based on the category, geometry, and relative position of the layout elements. If the layout model does not output a reliable area, a fixed-proportion backtracking area is generated according to the page size; for title areas with insufficient recognition information, the cropping range can be expanded for re-recognition.

[0009] Optical character recognition (OCR) and text normalization are performed on the title area, body area, and footer area respectively. For multiple text fragments within the same area, a condition chain is formed according to the configured AND, OR, and OR logic. For the same target element class with inclusion conditions configured in different areas, a cross-area relationship is adopted where at least one area can be determined to satisfy the condition. For conditions that must not be included, the text from all areas is merged and uniformly rejected. This yields the candidate element classes that may correspond to the current page.

[0010] The system maintains the current component class, its closure status, already appearing component classes, and consecutive page count according to page order. When the current component class has a last page condition configured, it first checks if the current page is the last page. If not closed, it retains the current component class; if closed, it re-matches open component rules from the next page. When the current component class does not have a last page condition configured, it can switch components if the starting condition of another target component class is met; otherwise, it uses the component class from the previous page. Constraints such as maximum consecutive page count, maximum number of segments within the same volume, and minimum page count are then applied, and the component sorting results are output.

[0011] The present invention also provides an automatic file image sorting system based on page layout and sequence constraints, including an image sequence acquisition module, a page layout module, a partition recognition module, a rule matching module, a sequence constraint module, and a result output and traceability module, each module being configured to execute the above methods.

[0012] Compared with existing technologies, the present invention has at least the following beneficial effects: First, by recognizing the page title area, body text area, and footer area separately, the document name clues and page number clues participate in the judgment within their corresponding areas, reducing the interference of irrelevant content such as body text and seals on title matching; Second, by combining text condition chains, cross-area inclusion relationships, and merged text exclusion relationships, it can adapt to changes in title position and scanning recognition errors of similar materials; Third, by maintaining the sequence consistency of continuous materials through previous page status, last page, consecutive page number, and uniqueness constraints within the same volume, it reduces the probability of continuation pages being incorrectly split or different materials being incorrectly merged; Fourth, it provides the basis for saving area text, rule hits, and status changes for each page, facilitating manual review and rule optimization. Attached Figure Description

[0013] Figure 1 A flowchart of an automatic file image segmentation method provided in an embodiment of the present invention; Figure 2 This is a structural diagram of an automatic file image sorting system provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the file page layout partitioning provided in an embodiment of the present invention; Figure 4 A flowchart of sequence state constraints provided for embodiments of the present invention. Detailed Implementation

[0014] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments are for illustrative purposes only and not for limiting the scope of protection of the present invention. Where there is no conflict, the technical features of the various embodiments can be combined with each other.

[0015] like Figure 1 As shown, the method in this embodiment includes steps S101 to S107.

[0016] Step S101: Obtain the ordered sequence of archival images and the set of partitioning rules. The electronic device reads multiple image files from a volume folder and arranges the images as P1, P2, ..., Pn according to the page number in the filename, the scanning order, or a pre-established page-level index. Each page records the volume identifier, relative path, and page number. The partitioning rule set is saved according to the archival type so that different business archives can use their own rule templates.

[0017] Each component segmentation rule includes the target component class name and a start page condition. The start page condition can be configured as follows: the title area must contain a condition chain, the title area must not contain a condition chain, the body area must contain a condition chain, the body area must not contain a condition chain, the footer area must contain a condition chain, and the footer area must not contain a condition chain. Component segmentation rules may also include a corresponding end page condition chain, a page number determination switch, a maximum number of consecutive pages, a minimum number of pages, and a flag indicating that at most one paragraph can appear within the same volume.

[0018] Step S102: Perform layout analysis on the current page image and determine the target layout area. Input the RGB pixel matrix of the current page image into the document layout analysis model. The model outputs multiple layout elements, each of which includes at least a category label, confidence score, reading order, and a bounding box in pixel coordinates [x0, y0, x1, y1]. Category labels include, for example, document title, paragraph title, figure title, header, body text, table, footer, and page number. The model used can be a neural network layout detection model capable of outputting the above data, and is not limited to a specific model name.

[0019] Candidate title boxes are selected from layout elements. For each candidate title box, a candidate score is calculated based on category priority, model confidence, the ratio of candidate box width to page width, the ratio of candidate box area to page area, and the position of the candidate box's vertical center relative to the page height. Headers that are too close to the top of the page and have insufficient height, and paragraph titles located in the lower middle of the page and have insufficient width, receive lower scores. The candidate box with the highest score and whose height and bottom edge position meet the confidence threshold are selected as the title seed box.

[0020] For document title boxes or figure title boxes that are vertically adjacent to the title seed box and overlap horizontally by a preset proportion, their bounding boxes are iteratively merged. If the height of the merged box exceeds a first proportional threshold of the page height, or if the bottom edge exceeds a second proportional threshold of the page height, merging is stopped or the bottom edge is clipped. This adapts to multi-line main titles while avoiding merging large blocks of body text into the title area.

[0021] Select a footer candidate box from layout elements categorized as footer or page number and located within a preset range at the bottom of the page. If no footer candidate box exists, a fixed-ratio area at the bottom of the page can be used as the footer area. The body text area uses the full page width, with its top edge following the bottom edge of the title area and its bottom edge preceding the top edge of the footer area. When the layout analysis model does not produce usable results, the title area can use a fixed area between the top of the page and a preset ratio of the page height, and the footer area can use an area below another preset ratio of the page height. All of these ratios can be configured according to the file type.

[0022] Figure 3 The relationship between the title area 201, body area 202, and footer area 203 is illustrated. Title area 201 is used to extract starting page clues such as material names; body area 202 is used to extract document features from the body text; and footer area 203 is used to extract page numbers and ending page clues. For multiple paragraph title boxes output by the layout model, they can also be identified separately according to the reading order, and the identified text that does not highly overlap with the main title area can be added to the title area text.

[0023] Step S103: Perform optical character recognition (OCR) on the three target page regions respectively. Specifically, crop the region image from the current page image according to the region bounding box and input it into the OCR model to obtain the text lines and their confidence scores. To avoid truncating characters at the cropping boundary, the bounding box can be extended in the horizontal and vertical directions according to the region type. For text areas with a large height, they can be divided into multiple horizontal strips with overlapping parts, and the results can be merged after recognition.

[0024] The recognized text undergoes normalization, such as converting to uniform case, removing spaces, line breaks, and punctuation, and retaining only Unicode letters and numbers. Normalization reduces the impact of differences in whitespace and punctuation in the scanned layout and allows text fragments in the rule base to be compared with the recognition results in a consistent manner. When the original recognized text in the title area is empty or the number of normalized characters is less than the threshold, a fallback area covering a larger area at the top of the page is used for re-recognition, and the result with more valid characters is selected.

[0025] Step S104: Match candidate component classes according to the component segmentation rules. Each condition chain consists of a text segment array `segments` and a logical operator array `ops`. Each element in `ops` represents the AND, OR, or OR logic applied between adjacent text segments. The system first removes null values ​​from the text segments and corrects the number of operators, then evaluates them by left associativity according to their order of arrangement. For example, condition chain A AND B OR C is evaluated using `((A AND B) OR C)`.

[0026] For the start page rule of the same target element class, if one or more areas in the title area, body area, and footer area have a must-include condition chain configured, then that area is considered a decidable area only if the area text is not empty, and the inclusion condition is considered to be true if at least one decidable area's must-include condition chain is matched. If a must-include condition is configured but all corresponding area texts are empty, then the inclusion condition is considered not true.

[0027] For any of the "must-not-exclude" condition chains configured in the title, body, or footer areas, the system merges the text from all three areas before searching. If any "must-not-exclude" condition chain evaluates to true, the current target category is rejected. This prevents incorrect matches even if prohibited clues fall into adjacent areas due to layout detection errors. Multiple target categories are scanned according to the rule list, and the first target category that fully satisfies the starting page conditions is selected. Alternatively, a preset priority can be used for selection.

[0028] Taking real estate registration archives as an example, the title area of ​​the first page of the target document type "Registration Application Form" must contain the text fragment "Registration Application Form," with both the maximum and minimum number of consecutive pages set to 2, and a maximum of one instance appearing in the same volume. The body area of ​​the target document type "ID Card Copy" must contain both "Resident ID Card" and "Citizen Identification Number," with a maximum number of consecutive pages set to 2. The body area or footer area of ​​the target document type "Issuance Map" can be configured with any text fragment from "CGCS2000," "National Geodetic Coordinate System," or the name of a designated surveying and mapping unit. The above text is merely an example of rule configuration and does not constitute a restriction on the applicable archive types.

[0029] Step S105: Determine the starting page, continuing page, and ending page based on the previous page status. The system processes pages in the order of P1 to Pn, and maintains the current item class currentPiece, the closed set of item classes that are closed and have at most one segment in the same volume, the number of consecutive pages of the current item class streak, and the flag resetNext indicating whether the next page should be forcibly rematched.

[0030] If no current element class exists or `resetNext` is true, match the start condition of the current page from the open rules that are not part of `closedSet`. If a match is found, set the current page as the start page of the corresponding target element class; if no match is found, mark the page as pending review.

[0031] When a current element class exists and that element class is configured with a last page condition, the last page condition of the current element class is used first to determine the current page. The last page condition (must include or must not include) is evaluated using the same partitioning method as the starting page. If page number determination is also enabled, page numbers in the form of "Page x of y", "Page x of y", or "x / y" are parsed from the footer text. When x equals y, the current page is determined to be the last page; if the last text condition is met but the parsed x does not equal y, the overly broad last text condition can be rejected by the numerical determination result.

[0032] If the current page does not meet the end-page condition, it is retained as the current page class to ensure that untitled continuation pages are not broken up; if the current page meets the end-page condition, it is still assigned to the current page class, and currentPiece is cleared before processing the next page. If the current page class is configured to have at most one segment in the same volume, it is added to closedSet after closing.

[0033] When the current item class does not have a specified end page condition, the system checks if the current page matches the start page condition of a different opening rule. If it does, the current page is used as the start page for the new item, and the previous item is closed; otherwise, the previous item class is used. This state transition process is as follows: Figure 4 As shown.

[0034] Step S106: Execute sequence constraints. When the current page is assigned to a certain item class, count the number of consecutive pages of the same item class preceding the current page. If the number exceeds the maximum number of consecutive pages configured for that item class, stop assigning the current page to that item class, mark the current page as awaiting re-identification or manual review, and re-execute the starting page matching for subsequent pages. After the entire volume is processed, scan the number of pages in each segment. If the number is less than the configured minimum number of pages, generate a verification alarm. For item classes that have at most one segment in the same volume, after closure, they are no longer allowed to be candidates for new items.

[0035] Step S107: Output the page segmentation results and traceability basis. The output results should include at least the volume identifier, page relative path, page number, target part type, and boundary status. The system can also save the coordinates and identified text of the title area, body area, and footer area, the evaluation results of the rule condition chain, the part type of the previous page, the determination of the last page, the verification of consecutive page numbers, and the final determination reason. The segmentation results can be written to the local database or the server database and synchronized to the manual quality inspection interface; when manually modifying the part name, the values ​​before and after the modification, the operator, and the time are recorded for auditing and reverse optimization of rules.

[0036] like Figure 2As shown, the automatic part sorting system of this embodiment includes an image sequence acquisition module 110, a layout partitioning module 120, a partitioning recognition module 130, a rule matching module 140, a sequence constraint module 150, a result output and traceability module 160, and a part sorting rule base 170. The image sequence acquisition module 110 establishes the sequential index of volumes and pages; the layout partitioning module 120 performs layout analysis and region determination; the partitioning recognition module 130 performs region cropping, optical character recognition, and text normalization; the rule matching module 140 reads the part sorting rule base 170 and determines the candidate part classes; the sequence constraint module 150 maintains the current part status and verifies the sequence constraints; and the result output and traceability module 160 saves the part sorting results and the judgment criteria.

[0037] The above system can be deployed on a personal computer, server, or dedicated document processing equipment. The electronic device includes a processor, memory, and a communication interface. When the computer program stored in the memory is executed by the processor, steps S101 to S107 are implemented. The layout analysis model and optical character recognition model can run locally or through a controlled model service interface.

[0038] This invention is not limited to real estate registration archives, but also applies to scanned images of engineering approval archives, contract archives, document archives, accounting archives, and other materials composed of multiple materials arranged in consecutive pages. The target page area can also include a header area, a stamp area, or a table area in addition to the title area, body text area, and footer area; as long as the page area recognition result and page sequence constraints are used to jointly determine the component division result, it can fall under the concept of this invention.

Claims

1. A method for automatically segmenting archival images based on page layout and sequence constraints, characterized in that, Performed by an electronic device, the process includes: acquiring a sequence of archival images arranged according to the page order within a volume and a set of segmentation rules, wherein the set of segmentation rules includes multiple segmentation rules corresponding to target document categories, and each segmentation rule includes at least a starting page text condition chain and sequence constraints associated with the page layout area; performing layout analysis on the current page image in the archival image sequence to obtain the category and coordinates of the layout elements, and determining multiple target layout areas, including a title area, a body text area, and a footer area, based on the category, coordinates, and size of the current page image; performing optical character recognition on the multiple target layout areas respectively, and normalizing the obtained regional text; and then processing the normalized regional text. The text is matched with the segmentation rules. For multiple text-containing conditions within the same target page area, the logical operation relationship configured in the condition chain is evaluated. For text-containing conditions configured in different target page areas, at least one region is matched. The text from multiple target page areas is merged, and conditions that must not contain text are uniformly rejected to obtain the candidate component class corresponding to the current page image. Based on the page order of the archive image sequence, combined with the segmentation status of the previous page image, the candidate component class, and the sequence constraints, the target component class and segmentation boundary of the current page image are determined. The correspondence between each page image in the archive image sequence and the target component class is output and stored.

2. The method according to claim 1, characterized in that, The step of performing layout analysis on the current page image includes: inputting the pixel data of the current page image into a layout analysis model to obtain layout analysis results containing layout element categories, confidence levels, reading order, and bounding box coordinates; filtering title candidate boxes and footer candidate boxes from the layout analysis results, and determining the title area and footer area based on the filtering results; and determining the area between the lower edge of the title area and the upper edge of the footer area as the body text area.

3. The method according to claim 2, characterized in that, Determining the title area includes: calculating candidate scores based on the category, confidence level, width as a percentage of page width, area as a percentage of page area, and vertical position of the candidate title boxes; selecting candidate title boxes whose scores meet preset conditions; merging at least one candidate title box that is vertically adjacent to the selected candidate title box and horizontally overlaps it, and restricting the height and bottom edge position of the merged box to obtain the title area.

4. The method according to claim 2, characterized in that, When no candidate title or footer box that meets the credibility criteria is obtained, a corresponding fixed layout area is generated according to the preset page height ratio. When the recognized text in the title area is empty or its normalized character count is less than the preset threshold, the cropping range of the title area is expanded and optical character recognition is performed again, selecting the recognition result with a larger number of normalized characters.

5. The method according to claim 1, characterized in that, The normalization process includes: converting the regional text into a unified character format, removing whitespace characters and symbols, and retaining letters and numbers; the text condition chain includes multiple text segments and AND and OR logical operators located between adjacent text segments, and left-associative evaluation is performed according to the order of the text condition chain.

6. The method according to claim 1, characterized in that, Each component classification rule also includes at least one of the following: a last page text condition chain and a page number determination condition. When the previous page image has been assigned to the current component class, the region text of the current page image is matched with the last page text condition chain of the current component class, and / or the current page number and the total number of pages are parsed from the region text in the footer area, and the current page image is marked as the last page when the current page number equals the total number of pages.

7. The method according to claim 6, characterized in that, When the current component class is configured with an end page condition and the current page image does not meet the end page condition, the current page image is assigned to the current component class; When the current page image meets the end page condition, the current element class is closed and the start page matching is re-executed for the next page image; When the current component class is not configured with an end page condition, if the current page image matches the start page text condition chain of different target component classes, then the current page image is used as the start page of the different target component classes.

8. The method according to claim 1, characterized in that, The sequence constraints include at least one of the following: the maximum number of consecutive pages in a target component class, the minimum number of pages, and the maximum number of segments that can appear within the same volume. When the number of consecutive pages assigned to the same target component class exceeds the maximum number of consecutive pages, the assignment of the current page image to that target component class is stopped. After a target component class is closed, the target component class configured to have at most one segment within the same volume is set to a non-open state. After the entire volume is processed, the component verification result is generated based on the minimum number of pages.

9. The method according to claim 1, characterized in that, Also includes: For each page of image records, at least one of the following is recorded: page area coordinates, area text, matching segmentation rules, condition chain hit results, previous page segmentation status, last page judgment result, and sequence constraint verification results, forming a traceable record associated with the page segmentation result; The page segmentation results are provided to the manual review interface, and manual modification records are stored.

10. An automatic file segmentation system based on page layout and sequence constraints, characterized in that, include: The image sequence acquisition module is used to acquire the archive image sequence arranged according to the page order within the volume and the set of component division rules; The page layout module is used to perform page layout analysis on the current page image, obtain the category and coordinates of page elements, and determine the title area, body text area, and footer area; The partition recognition module is used to perform optical character recognition on the title area, body area and footer area respectively and to normalize the text in the area. The rule matching module is used to determine the candidate component class of the current page image according to the logical operation relationship of the text condition chain in the same area, the relationship of at least one region hit between different areas, and the unified rejection relationship based on the condition of the text in the merged area not containing the text. The sequence constraint module is used to determine the target component class and component boundary of the current page image according to the page order, the component status of the previous page image, the candidate component class, and the sequence constraint. The result output and traceability module is used to output and store the correspondence between each page image and the target component class and the corresponding judgment criteria.

Images