Document quality inspection method and system based on multi-modal model

Abstract

The invention belongs to the technical field of document intelligent processing, and particularly relates to a document quality inspection method and system based on a multi-modal model, which can solve the problems of high missed detection and false detection, weak interpretability and insufficient suitability in the existing document quality inspection, and the method comprises the following steps: performing paging processing on an input target document to obtain a paging image; performing text area detection on the paging image to generate a text detection box list including coordinate information in pages; performing identification and rule verification on the content in each text detection box through the first channel, and generating a suspicion error list; dense block cutting is carried out on the paging image through a second channel, whether character missing or errors exist in each block is judged, and a block error list is generated; according to a preset quality inspection mode, performing aggregation processing on the suspicion error list and the block cutting error list; and performing de-duplication processing on the aggregated error information to generate a quality inspection result.

Description

Technical Field

[0001] This invention belongs to the field of document intelligent processing technology, and in particular relates to a document quality inspection method and system based on a multimodal model. Background Technology

[0002] In the field of document quality inspection, existing methods for detecting text errors or missing text mostly rely on a single detection channel, relying solely on text recognition or simple rule-based validation to identify errors, making it difficult to balance accuracy and comprehensiveness. For example, sentence-by-sentence detection may miss global issues such as missing entire lines of text, and the lack of targeted validation logic leads to frequent problems such as formatting errors and misjudgments of technical terms. Furthermore, traditional methods have poor adaptability to low-quality documents such as blurry scans, and text recognition accuracy is significantly affected by image quality. They also lack flexible detection mode switching mechanisms, failing to meet the needs of different scenarios such as efficient daily quality inspection and zero-missing compliance. In addition, the detection results suffer from low error location accuracy and insufficient visual evidence, leading to low efficiency in manual review and making it difficult to adapt to the diverse quality inspection needs in practical applications. Summary of the Invention

[0003] The purpose of this invention is to provide a document quality inspection method and system based on a multimodal model to solve the above-mentioned problems.

[0004] To achieve the above objectives, in a first aspect of the present invention, a document quality inspection method based on a multimodal model is provided, comprising the following steps: The input target document is paginated to obtain a pagination image; The paginated image is subjected to text region detection to generate a list of text detection boxes including in-page coordinate information; The first channel identifies and validates the content within each text detection box according to rules, generating a list of suspected errors; the second channel performs dense segmentation of the paginated image, determines whether there are missing or incorrect texts in each segment, and generates a segmentation error list. According to the preset quality inspection mode, the suspected error list and the block error list are aggregated. The aggregated error messages are deduplicated to generate quality inspection results.

[0005] Furthermore, in the document quality inspection method based on the multimodal model, the target document is a PDF document; after the pagination process, the method further includes enhancing the pages with insufficient readability in the pagination image.

[0006] Furthermore, in the document quality inspection method based on the multimodal model, the text region detection is implemented by initializing and reusing the OCR detector in a thread-local manner.

[0007] Furthermore, in the document quality inspection method based on the multimodal model, the processing steps of the first channel include: A multimodal model is used to perform read-only recognition of the content within the text detection box, and the recognized text and confidence score are output. When the confidence level is lower than the preset confidence threshold, the text length is lower than the preset length threshold, or the difference between the two recognition results is higher than the preset difference threshold, the text detection box area is sharpened and re-recognized to obtain stable recognized text; The stable identified text is verified based on preset rules, and a list of suspected errors is generated for texts with errors, including error type, basis, coordinates, and cropped evidence image.

[0008] Furthermore, in the document quality inspection method based on the multimodal model, the processing steps of the second channel include: The paginated image is cut into blocks using a multi-scale overlapping method, and each block is labeled with its original size, block size, and block start coordinates. For each of the segments, determine whether there is missing or incorrect text, and generate a segment error list including coordinates, error reasons and evidence screenshots; A visual image marked with a red box is generated for the erroneous slice.

[0009] Furthermore, in the document quality inspection method based on the multimodal model, the preset quality inspection modes include a precision priority mode and a limit recall mode. The precise priority mode outputs the suspected error list and the chunking error list separately; the extreme recall mode converts the chunking error list into a format that matches the suspected error list and then merges them for output.

[0010] Furthermore, in the document quality inspection method based on the multimodal model, the step of deduplicating the aggregated error information and generating the quality inspection result includes: deduplicating using the identified text content and coordinate information as a combination key, retaining the error information with the highest confidence; the generated quality inspection result includes error evidence and structured information, the error evidence includes cropped evidence images and visual images marked with red boxes, and the structured information includes error type, basis, and coordinate information.

[0011] In another aspect of the present invention, a document quality inspection system based on a multimodal model is also proposed, including a document preprocessing module, a text detection module, a dual-channel interpretation module, a pattern aggregation module, and a deduplication output module; The document preprocessing module is used to paginate the input target document to obtain a paginated image; the text detection module is used to perform text region detection on the paginated image and generate a list of text detection boxes including page coordinate information; the dual-channel interpretation module includes a first channel unit and a second channel unit. The first channel unit is used to identify and verify the content within each text detection box and generate a list of suspected errors. The second channel unit is used to perform dense segmentation of the paginated image, determine whether there are missing or incorrect characters in each segment, and generate a segmentation error list; the pattern aggregation module is used to aggregate the list of suspected errors and the segmentation error list according to a preset quality inspection pattern; the deduplication output module is used to deduplicate the aggregated error information and generate a quality inspection result including error evidence and structured information.

[0012] Furthermore, in the document quality inspection system based on the multimodal model, the document preprocessing module also includes an image enhancement submodule, which is used to enhance pages with insufficient readability in the paginated image; the text detection module uses a thread-local storage method to reuse the OCR detector.

[0013] Furthermore, in the document quality inspection system based on the multimodal model, the first channel unit includes an identification subunit, a verification subunit, and a suspect generation subunit. The identification subunit uses a multimodal model to perform text recognition and determines stable recognition text through a sharpening backoff mechanism. The verification subunit performs error verification based on preset rules. The second channel unit includes a slicing subunit, a judgment subunit, and a rendering subunit. The slicing subunit slices text in a multi-scale overlapping manner, and the rendering subunit generates a visual image with a red box for erroneous slices.

[0014] Compared with the prior art, the present invention has at least the following technical effects: This invention, through a complete process of pagination processing, text region detection, dual-channel interpretation, pattern aggregation, and deduplication output, solves the problem of high rates of missed and false detections in document quality inspection compared to existing technologies. Its collaborative design of precise frame-by-frame recognition and rule verification in the first channel, and dense block-based supplementary inspection in the second channel, ensures accurate identification of text errors while avoiding omissions of missing text. The preset quality inspection modes can flexibly adapt to different business scenarios, and deduplication reduces redundant error information. The final quality inspection results combine visual evidence with structured information, thereby improving the efficiency of manual review and the traceability of the quality inspection process. Attached Figure Description

[0015] Figure 1 This is a flowchart of a document quality inspection method based on a multimodal model in one embodiment of the present invention; Figure 2This is a detailed flowchart of a document quality inspection method based on a multimodal model in one embodiment of the present invention; Figure 3 This is a flowchart of a document quality inspection system based on a multimodal model in one embodiment of the present invention. Detailed Implementation

[0016] The following will describe in more detail a document quality inspection method and system based on a multimodal model according to the present invention, with reference to the accompanying diagrams, which illustrate preferred embodiments of the invention. It should be understood that those skilled in the art can modify the invention described herein while still achieving its advantageous effects. Therefore, the following description should be understood as being of general knowledge to those skilled in the art and is not intended to limit the invention.

[0017] For clarity, not all features of the actual embodiments are described. In the following description, well-known functions and structures are not detailed in detail, as they would obscure the invention with unnecessary detail. It should be understood that in the development of any actual embodiment, numerous implementation details must be made to achieve the developer's specific objectives, such as changes from one embodiment to another according to limitations related to the system or business. Furthermore, it should be understood that such development work may be complex and time-consuming, but is merely routine work for those skilled in the art.

[0018] The invention is described more specifically by way of example in the following paragraphs with reference to the accompanying drawings. The advantages and features of the invention will become clearer from the following description. It should be noted that the drawings are in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the invention.

[0019] Based on the teachings of this specification, those skilled in the art can form new technical solutions through cross-combination of different implementation methods without creating technical contradictions. Such variations should all be considered to fall within the protection scope of this invention.

[0020] Example 1 Existing document quality inspection technologies generally suffer from four core problems: First, OCR recognition accuracy is greatly affected by image quality, easily leading to missed detections and false detections; second, they are insufficiently adaptable to scenarios such as formatting errors and mixing of full-width and half-width characters, with incomplete rule coverage; third, error location is vague and lacks visual evidence, requiring manual review to check page by page, resulting in extremely low efficiency; fourth, when processing multi-page documents concurrently, repeated detector initialization leads to resource waste and performance degradation. To solve these problems, such as... Figure 1-2 As shown, this embodiment proposes a document quality inspection method based on a multimodal model, which balances detection accuracy, recall, and practicality. Specifically, it includes the following steps: S1: Perform pagination on the input target document to obtain a pagination image; S2: Perform text region detection on the paginated image to generate a list of text detection boxes including page coordinate information; S3: The first channel is used to identify and verify the content within each text detection box, generating a list of suspected errors; the second channel is used to densely slice the paginated image, determine whether there are missing or incorrect text in each slice, and generate a slice error list. S4: Aggregate the suspected error list and the segmentation error list according to the preset quality inspection mode; S5: Perform deduplication on the aggregated error information and generate quality inspection results.

[0021] This embodiment uses a 10-page corporate lease contract PDF (including 3 scanned pages, where page 3 is a blurry copy, page 7 has missing text, and page 5 has a formatting error) as an example to explain the above steps S1-S5 in detail.

[0022] It should be noted that the target document in step S1 is a PDF document. In this embodiment, a 10-page corporate lease contract PDF is used as the processing object. First, the system calls the PDF rendering engine to perform pagination processing on the input PDF document. The complete document is split into 10 independent pagination images according to the natural page number order, which are respectively denoted as PageImage[0] (corresponding to page 1 of the original document) to PageImage[9] (corresponding to page 10 of the original document). This page-level splitting achieves independent processing of each page, avoids interference between multiple pages, and ensures the accuracy of subsequent detection and interpretation.

[0023] Furthermore, after completing the pagination process, the system will automatically perform readability detection on each generated pagination image, that is, determine whether the page has problems such as low contrast, blurry text, and unclear details that affect subsequent recognition, and perform enhancement processing on pages with insufficient readability. For example, in this embodiment, page 3 is a scanned document. Due to multiple photocopyings, the text edges of the "lease term" clause are blurred and the recognizability is reduced, which is a case of insufficient readability. At this time, optional image enhancement processing will be triggered. Specifically, the CLAHE (contrast-limited adaptive histogram equalization) algorithm is used to specifically brighten the text area, and at the same time, the text edge details are optimized by light sharpening. This processing method is similar to the clarity enhancement function in commonly used image editing tools, which can effectively improve the recognizability of blurry text, and finally generate the enhanced pagination image EnhancedPageImage[2] (corresponding to page 3 of the original document). As for pagination images such as pages 1, 2, and 4 that meet the clarity standards, the system directly reuses the original pagination images without performing additional enhancement processing, thereby avoiding unnecessary computing power consumption and taking into account both processing effect and efficiency.

[0024] This step ultimately outputs 10 basic pagination images (PageImage[0-9]) and 1 enhanced pagination image (EnhancedPageImage[2]). All output images maintain the integrity of the original document's content and the page number correspondence.

[0025] Furthermore, step S2 performs text region detection processing on the paginated image (including the enhanced PageImage[2]) output in step S1, and finally generates a list of text detection boxes including page coordinate information. In this embodiment, text region detection is implemented by initializing and reusing the OCR detector in a thread-local manner.

[0026] Specifically, upon system startup, a single OCR detector instance is initialized. This instance is then bound to a processing thread via a thread-local mechanism, forming a dedicated "thread-detector" association. When processing multi-page documents concurrently, all paged images share this detector instance, eliminating the need to reinitialize the detector for each page. This is analogous to printing multiple documents consecutively after a single power-on, rather than restarting the printer after each print job. This avoids the resource waste and time consumption caused by repeated initialization, reduces resource contention between threads, and significantly improves overall processing efficiency.

[0027] Furthermore, during the actual detection process, the system filters out interfering elements on the page using adjustable thresholds, such as contract borders, watermarks, table lines, and other irrelevant information, ensuring focus is solely on the text area. Simultaneously, an expansion rule is employed to extend the identified text area by 1-2 pixels, effectively avoiding missed detections caused by text edge cropping. After scanning each paginated image, a corresponding list of text detection boxes is generated, denoted as DetBBoxes[i] (where i is the page index). Each text detection box contains complete in-page coordinate information. This in-page coordinate information includes the coordinates of the top-left corner (x1, y1) and bottom-right corner (x2, y2), the polygonal outline of the text, and the recognition confidence level of the text detection box. For example, the clause "Rental amount: 5000 yuan / month" on page 5 has coordinate information x1=150, y1=320, x2=420, and y2=350 in the corresponding text detection box DetBBoxes[4]. The polygonal outline of the text accurately fits the edge of the clause text, and the detection reliability is as high as 96%, ensuring the accuracy of subsequent interpretation.

[0028] Step S2 ultimately outputs 10 sets of text detection box lists (DetBBoxes[0-9]), each set of lists corresponding to the position information of all text areas on a page of document.

[0029] Furthermore, step S3 uses the collaborative interpretation of the first channel (bbox channel) and the second channel (tile channel) to complete the functions of precise error checking for each text box and comprehensive supplementary inspection of the entire page, and finally generates a list of suspected errors and a list of segmented errors, which not only ensures the accuracy of error judgment, but also avoids the risk of missed detection.

[0030] The processing steps for the first channel are as follows: First, a Visual-Language Model (VLM) is used to perform read-only recognition on the content within each text detection box generated in step S2. The entire process adheres to the principle of read-only reading and no rewriting, outputting the recognized text and its corresponding confidence score to ensure the original information is not tampered with. For example, for the text detection box corresponding to "Lease period: 2024.13.01-2025.13.01" on page 3, the model outputs the recognized text and an 88% confidence score. Next, the system automatically determines whether the recognition result meets the stability conditions. If the confidence level is lower than the preset confidence threshold (e.g., 90%), the text length is lower than the preset length threshold (e.g., 3 characters), or the difference between the two recognition results is higher than the preset difference threshold (e.g., the character matching rate is lower than 80%), the text detection box area will be sharpened separately and re-recognized. The stable recognition text is determined through secondary verification. In the example above, the results were consistent after sharpening and rollback and secondary recognition, and the stable recognition text was finally locked. Subsequently, a comprehensive verification of the stable recognized text is carried out based on preset rules. These preset rules include key scenarios such as misspellings, formatting, punctuation, full-width and half-width characters, and date numbering. For example, 2024.13.01 was judged to be a date format error because the month exceeded the legal range of 1-12. Finally, for all texts identified as containing errors, a list of suspected errors is generated, including the error type, the basis for the error, the coordinates of the text detection box, and the cropped evidence image. All entries are then compiled to form a list of suspected errors.

[0031] The processing steps for the second channel are as follows: First, perform a multi-scale overlapping block cutting operation on the paginated image output in step S1. During the cutting process, each block is labeled with the original page size, the block size itself, and the starting coordinates of the block. For example, the image on page 7 is cut into blocks using a "2×2 large grid + 3×3 small grid" method, with a grid overlap rate of about 50%, which effectively avoids missing edge text detection. Next, a multimodal model is used to determine whether each segment contains missing or incorrect text. During the determination process, coordinate context information is injected to assist semantic analysis. For example, for a segment containing "the client needs to provide", the model combines the semantic logic that "provide" as a transitive verb needs to be paired with an object to determine that there is missing text and generates segment error entries including segment coordinates, error reasons and evidence screenshots. All entries are summarized to form a segment error list. Finally, for the slices identified as having errors, red boxes are overlaid on the image to mark the error locations, generating a visual image (overlay image) with red boxes. All slice images with red boxes are summarized into a list of content_tiles, making the error locations visually visible.

[0032] Through the above dual-channel collaborative processing, step S3 finally outputs a list of suspected errors (such as date format errors on page 3), a list of slicing errors (such as text missing errors on page 7), and a list of content_tiles (such as slicing images with red boxes on page 7).

[0033] Step S4 involves performing targeted aggregation processing on the suspected error list and segmented error list generated in step S3 according to a preset quality inspection mode. This differentiated result integration strategy adapts to the balanced needs of different business scenarios regarding detection accuracy, recall completeness, and review costs. In this embodiment, the preset quality inspection modes specifically include a precision-first mode and a maximum recall mode. These two modes design two different aggregation rules based on different business objectives. Specifically, the precision-first mode outputs the suspected error list and the segmented error list separately; the maximum recall mode converts the segmented error list into a format matching the suspected error list before merging and outputting them.

[0034] Specifically, the precision-priority mode is mainly suitable for scenarios that are sensitive to false alarm rates and pursue efficient and low-cost review, such as daily batch quality inspection of contracts and rapid review of routine documents. Its core idea is to avoid uncertain errors from interfering with the main process by "separate output". Specifically, the system will mark the suspected error list generated by the first channel as "high confidence error" and push it directly to the manual review queue, so that reviewers can focus on errors with higher certainty; while the block error list generated by the second channel is marked as suspected error and is not actively mixed into the main review process. It is stored separately in the background and needs to be actively retrieved and viewed by humans according to actual needs. This approach can both preserve the traceability of potential errors and effectively control the false alarm rate, thereby significantly reducing the time cost and workload of manual review.

[0035] The extreme recall mode is specifically designed for scenarios with strict requirements for zero omissions, such as compliance audits, critical contract reviews, and high-risk document verification. Its main goal is to maximize the recall of all potential errors, allowing for the addition of some suspected errors and increased review workload at the cost of this. The aggregation process in this mode consists of two steps: First, the segmented error list is standardized and transformed by supplementing coordinate alignment, error type mapping, and evidence format unification to ensure that its structure fully matches the suspected error list, providing a basis for merging the two types of error information; then, a union operation is performed on the transformed segmented error list and the original suspected error list to integrate all potential errors into a complete total suspected set, avoiding missed detection problems caused by channel separation.

[0036] In this embodiment, since the focus is on the compliance review of corporate lease contracts, a zero-miss requirement must be met. Therefore, the extreme recall mode is selected for aggregation processing. After format conversion and set merging, the final result is a total suspect set containing the date format errors and missing text errors detected in step S3.

[0037] Step S5 involves deduplicating and optimizing the error information aggregated in step S4, and generating standardized and usable final quality inspection results. This ensures the accuracy of the error information and provides complete support for subsequent manual review and audit traceability.

[0038] In the deduplication process, the system uses a combination of text content and coordinate information as the basis for deduplication judgment to identify duplicate records of the same error. For example, the missing text "Party A needs to provide" on page 7 is identified by both the first channel (bbox channel) through text detection and the second channel (tile channel) through tile detection. The system extracts the combination of the two error entries: the text content is "Party A needs to provide," and the coordinate information (text detection box coordinates and tile coordinates) highly overlaps because they point to the same document location, thus determining it as a duplicate error. At this point, the system compares the confidence levels of the two error entries, retaining the version with higher confidence (e.g., the second channel determines a confidence level of 99%), and deleting the duplicate entry, making the final error list free of redundancy and reducing the unnecessary workload of manual review.

[0039] After deduplication, the system generates quality inspection results including error evidence and structured information. Error evidence includes cropped evidence images generated by the first channel for frame-by-frame errors (e.g., a partial screenshot of the date format error on page 3), and visual images marked with red boxes generated by the second channel for segmentation errors (e.g., a red-boxed segmentation image of the missing text location on page 7). This visual evidence allows for quick location of errors during manual review and provides an intuitive understanding of the error scenario. Structured information includes error type (e.g., incorrect date format, missing text), error basis (e.g., month exceeding the legal range of 1-12, incomplete verb-object structure), and coordinate information (the specific page coordinates of the corresponding text detection box or segment). This structured presentation makes error information traceable, facilitating subsequent statistical analysis or problem tracing.

[0040] It should be noted that all information is organized into a JSON format file according to a unified standard. This file can be directly pushed to the front-end system (such as a contract review platform) and displayed as a contract preview marked with a red box, allowing reviewers to view it in real time. Alternatively, it can be archived to meet the traceability requirements of compliance audits. The final output JSON format quality inspection results clearly display complete information: a total of 2 errors, including 1 date format error and 1 missing text error, coupled with corresponding visual evidence and structured details, ultimately achieving the goals of error localization, evidence viewing, and information traceability.

[0041] Example 2 like Figure 3 This embodiment provides a document quality inspection system based on a multimodal model, which is suitable for various scenarios that require accurate detection of text errors or omissions, such as enterprise contracts and compliance documents. The system solves the problems of high false negatives and high omissions, weak interpretability and insufficient adaptability of existing quality inspection technologies by working together through five modules: document preprocessing module, text detection module, dual-channel interpretation module, pattern aggregation module and deduplication output module. The following describes in detail the functions of each module and the overall operation logic of the system by taking the quality inspection process of a 10-page enterprise lease contract PDF.

[0042] The document preprocessing module is used to paginate the input target document to obtain paginated images. This module provides a high-quality image foundation for subsequent detection. It can not only paginate the input target document (PDF format in this embodiment) and split it into 10 independent paginated images (PageImage[0] to PageImage[9]), but also includes an image enhancement submodule. It can automatically identify pages with insufficient readability in the paginated images (such as the lease term on page 3 due to photocopying), use the CLAHE algorithm to brighten the text area and lightly sharpen the edge details to generate enhanced paginated images (EnhancedPageImage[2]). Other clear pages directly reuse the original paginated images, which ensures the processing effect and saves computing power.

[0043] The text detection module receives the output from the document preprocessing module and is responsible for accurately locating text regions in the paginated images. This module uses a thread-local storage method to reuse the OCR detector. At startup, a single OCR detector instance is initialized and bound to the processing thread. When processing multiple pages concurrently, there is no need for repeated initialization, which significantly improves processing efficiency. During the detection process, interference elements such as contract borders and watermarks are filtered out by an adjustable threshold. The rule of expanding the text box by 1-2 pixels is used to avoid missing edge text. Finally, a list of text detection boxes (DetBBoxes[i]) containing the coordinate information within the page is generated for each paginated image. For example, the text detection box corresponding to "Rental amount: 5000 yuan / month" on page 5 clearly marks the coordinates of the upper left corner (x1=150, y1=320), the lower right corner (x2=420, y2=350), and a detection confidence of 96%.

[0044] The dual-channel interpretation module comprises a first-channel unit and a second-channel unit, which work together to achieve accurate error detection and correct missed detections. The first-channel unit consists of an identification subunit, a verification subunit, and a suspect generation subunit. The identification subunit uses a multimodal model to perform read-only identification of the content within each text detection box, outputting the identified text and confidence level. If the confidence level is below a threshold, the text is too short, or the difference between two identifications is too large, a sharpening backtracking mechanism is used to re-identify and determine a stable identified text. The verification subunit performs error verification based on preset rules such as typos, formatting, punctuation, full-width / half-width characters, and date numbering. For example, it identifies a date format error in "2024.13.01" on page 3. The suspect generation subunit packages the error information to generate a list of suspected errors including error type, basis, coordinates, and cropped evidence images. The second-channel unit also includes a segmentation subunit, a judgment subunit, and a rendering subunit. The slicing subunit slices the paginated image into blocks using a multi-scale overlap of "2×2 large grids + 3×3 small grids," with each block labeled with its original size, block size, and starting coordinates. The judgment subunit injects coordinate context into each block to determine if there is any missing or incorrect text, such as detecting a missing text in "Party A needs to provide" on page 7. The rendering subunit then overlays a red box mark on the erroneous blocks, generating a visual image with a red box, which is then summarized into a list of content_tiles.

[0045] The pattern aggregation module is responsible for flexibly processing the output results of the dual-channel judgment module according to business needs. This module supports preset precise priority mode and extreme recall mode: in daily quality inspection scenarios, the precise priority mode is selected to output the suspected error list and the segmented error list separately to avoid increasing the manual review burden of suspected errors; in compliance audit and other zero-missing-report scenarios, the extreme recall mode is selected to convert the segmented error list into a format that matches the suspected error list and then merge it for output, so that no potential errors are missed. In this embodiment, due to the need to meet compliance requirements, the extreme recall mode is used to generate the total suspected set.

[0046] The deduplication output module is responsible for optimizing error messages and generating the final quality inspection results. This module uses the identified text content and coordinate information as a key to deduplicate the aggregated error messages. For example, if a missing text error on page 7 is detected simultaneously by both channels, the system will retain the entry with the higher confidence level (99% confidence level for the second channel) and delete duplicates. The deduplicated error messages are then packaged into a quality inspection result that includes error evidence and structured information. The error evidence includes cropped evidence images from the first channel and a visual image with a red box from the second channel. The structured information covers the error type, basis, coordinates, and confidence level. All information is organized into a JSON file, which can be directly pushed to the front-end review platform for display or archived for audit traceability.

[0047] In summary, through the close collaboration of its various modules, the entire system achieves fully automated quality inspection from document input to result output. It reduces the rate of missed and false positives by relying on the dual-channel design, adapts to different business scenarios through flexible mode switching, and improves the interpretability of the results by leveraging visual evidence and structured information, thereby significantly reducing the workload of manual review and improving the efficiency and accuracy of document quality inspection.

[0048] The above are merely preferred embodiments of the present invention and do not constitute any limitation on the present invention. Any equivalent substitutions or modifications made by those skilled in the art to the technical solutions and content disclosed in the present invention without departing from the scope of the present invention shall be deemed to have remained within the protection scope of the present invention.

Claims

1. A document quality inspection method based on a multimodal model, characterized in that, Includes the following steps: The input target document is paginated to obtain a pagination image; The paginated image is subjected to text region detection to generate a list of text detection boxes including in-page coordinate information; The first channel identifies and validates the content within each text detection box according to rules, generating a list of suspected errors. The paginated image is densely segmented using the second channel to determine whether there are missing or incorrect texts in each segment and to generate a segmentation error list. According to the preset quality inspection mode, the suspected error list and the block error list are aggregated. The aggregated error messages are deduplicated to generate quality inspection results.

2. The method according to claim 1, characterized in that, The target document is a PDF document; after the pagination process, the process also includes enhancing the pages in the pagination image that are not readable.

3. The method according to claim 1, characterized in that, The text region detection is implemented by initializing and reusing the OCR detector in a thread-local manner.

4. The method according to claim 1, characterized in that, The processing steps for the first channel include: A multimodal model is used to perform read-only recognition of the content within the text detection box, and the recognized text and confidence score are output. When the confidence level is lower than the preset confidence threshold, the text length is lower than the preset length threshold, or the difference between the two recognition results is higher than the preset difference threshold, the text detection box area is sharpened and re-recognized to obtain stable recognized text; The stable identified text is verified based on preset rules, and a list of suspected errors is generated for texts with errors, including error type, basis, coordinates, and cropped evidence image.

5. The method according to claim 1, characterized in that, The processing steps for the second channel include: The paginated image is cut into blocks using a multi-scale overlapping method, and each block is labeled with its original size, block size, and block start coordinates. For each of the segments, determine whether there is missing or incorrect text, and generate a segment error list including coordinates, error reasons and evidence screenshots; A visual image marked with a red box is generated for the erroneous slice.

6. The method according to claim 1, characterized in that, The preset quality inspection modes include a precision priority mode and an extreme recall mode; The precise priority mode outputs the suspected error list and the chunking error list separately; the extreme recall mode converts the chunking error list into a format that matches the suspected error list and then merges them for output.

7. The method according to claim 1, characterized in that, The step of deduplicating the aggregated error information to generate quality inspection results includes: deduplicating using the identified text content and coordinate information as a combination key, retaining the error information with the highest confidence; the generated quality inspection results include error evidence and structured information, the error evidence includes cropped evidence images and visual images marked with red boxes, and the structured information includes error type, basis, and coordinate information.

8. A document quality inspection system based on a multimodal model, characterized in that, It includes a document preprocessing module, a text detection module, a dual-channel interpretation module, a pattern aggregation module, and a deduplication output module; The document preprocessing module is used to paginate the input target document to obtain a paginated image; the text detection module is used to perform text region detection on the paginated image and generate a list of text detection boxes including page coordinate information; the dual-channel interpretation module includes a first channel unit and a second channel unit. The first channel unit is used to identify and verify the content in each text detection box and generate a list of suspected errors. The second channel unit is used to perform dense segmentation of the paginated image, determine whether there are missing or incorrect characters in each segment, and generate a segmentation error list. The pattern aggregation module is used to aggregate the suspected error list and the block error list according to the preset quality inspection pattern. The deduplication output module is used to deduplicatize the aggregated error information and generate quality inspection results that include error evidence and structured information.

9. The system according to claim 8, characterized in that, The document preprocessing module also includes an image enhancement submodule, which is used to enhance pages with insufficient readability in the paginated images; the text detection module uses a thread-local storage method to reuse the OCR detector.

10. The system according to claim 8 or 9, characterized in that, The first channel unit includes an identification subunit, a verification subunit, and a suspect generation subunit. The identification subunit uses a multimodal model to perform text recognition and determines stable recognition text through a sharpening backoff mechanism. The verification subunit performs error verification based on preset rules. The second channel unit includes a slicing subunit, a judgment subunit, and a rendering subunit. The slicing subunit slices the blocks in a multi-scale overlapping manner, and the rendering subunit generates a visualization image with a red box for incorrect slices.

Images