Document identification method and computer program product

Abstract

The invention discloses a document recognition method and a computer program product, and relates to the technical field of data processing.The document recognition method comprises the steps that the document complexity of a target document is recognized, and a text analysis strategy matched with the document complexity is determined; the text analysis strategy comprises at least one of a lightweight model analysis strategy, a large model analysis strategy and a lightweight collaborative large model analysis strategy; and analyzing the target document according to the text analysis strategy to obtain a document identification result of the target document. According to the method, the problem that the recognition precision of the mixed elements in the complex document is insufficient is effectively solved, low-complexity content can be rapidly processed locally by establishing a complexity evaluation and strategy matching mechanism, high-complexity content can obtain sufficient computing resources, efficiency and precision of medium-complexity content are balanced through a collaborative mechanism, and the recognition precision of the mixed elements in the complex document is improved. Therefore, the accuracy and the real-time performance of recognizing the complex document are greatly improved, and meanwhile, the resource utilization rate of the edge end and the cloud end is also greatly improved.

Description

Technical Field

[0001] This disclosure relates to the field of large language model technology, and in particular to a document recognition method and computer program product. Background Technology

[0002] With the rapid development of Artificial Intelligence (AI) technology, AI knowledge bases have become a core infrastructure for enterprise intelligent transformation. These AI knowledge bases contain a vast amount of complex documents with diverse formats, layouts, and highly specialized content. Therefore, the ability to identify complex documents is particularly important.

[0003] In existing technologies, pre-trained models with a single parsing method are typically used for document recognition. These pre-trained models are either deployed purely in the cloud or purely locally, resulting in low recognition accuracy for complex documents and an inability to effectively utilize local and cloud computing resources. Summary of the Invention

[0004] In view of the above-mentioned defects or deficiencies in the existing technology, it is desirable to provide a document recognition method and computer program product that not only effectively solves the problem of insufficient recognition accuracy of mixed elements in complex documents, but also enables low-complexity content to be processed quickly locally by establishing a complexity assessment and strategy matching mechanism, high-complexity content to obtain sufficient computing resources, and medium-complexity content to balance efficiency and accuracy through a collaborative mechanism. This significantly improves the accuracy and real-time performance of complex document recognition, and also significantly improves the resource utilization of edge devices and the cloud.

[0005] Firstly, this application provides a document recognition method. The method includes: Identify the document complexity of the target document and determine a text parsing strategy that is suitable for the document complexity; the text parsing strategy includes at least one of a lightweight model parsing strategy, a large model parsing strategy, and a lightweight collaborative large model parsing strategy. The target document is parsed according to the text parsing strategy to obtain the document recognition result of the target document.

[0006] In conjunction with the first aspect, in one possible implementation, identifying the document complexity of the target document and determining a text parsing strategy adapted to the document complexity includes: Identify one or more target content elements in the target document; the target content elements are content elements related to the complexity of the document. Determine the content complexity of one or more types of target content elements in the target document, and determine the text parsing strategy corresponding to each content complexity. Based on the text parsing strategy corresponding to each of the aforementioned content complexities, a text parsing strategy adapted to the document complexity is determined.

[0007] In conjunction with the first aspect, in one possible implementation, determining the content complexity of one or more target content elements in the target document includes: If the target content element includes a table, then the content complexity of the table is determined based on the table's layout complexity and content density.

[0008] In conjunction with the first aspect, in one possible implementation, determining the content complexity of one or more target content elements in the target document includes: If the target content element includes an image, then the content complexity corresponding to the image is determined based on the element complexity of the image.

[0009] In conjunction with the first aspect, in one possible implementation, determining a text parsing strategy adapted to the document complexity includes: If the document complexity is lower than the preset minimum complexity, then determine the lightweight model parsing strategy that is compatible with the document complexity. If the document complexity is higher than the preset maximum complexity, then determine the large model parsing strategy that is compatible with the document complexity; If the document complexity is between the preset minimum complexity value and the preset maximum complexity value, then the lightweight collaborative large model parsing strategy adapted to the document complexity is determined.

[0010] In conjunction with the first aspect, in one possible implementation, the method further includes: Determine the confidence level of the document recognition result, wherein the confidence level is used to characterize the accuracy of the document recognition result; If the confidence level is less than a preset confidence threshold, the document recognition result will be processed as an anomaly.

[0011] In conjunction with the first aspect, in one possible implementation, the anomaly handling of the document recognition result includes: If the confidence level is within a preset abnormal confidence level range, the document recognition result is corrected according to the rule engine; the preset confidence threshold is greater than the maximum value of the preset abnormal confidence level range.

[0012] In conjunction with the first aspect, in one possible implementation, the anomaly handling of the document recognition result includes: If the confidence level is less than the minimum value of the preset abnormal confidence level range, the document recognition result is sent to the manual review platform for manual correction to obtain the manual correction result; the manual correction result is used as training data to participate in at least one of rule engine optimization, lightweight model incremental training and large model incremental training.

[0013] In conjunction with the first aspect, in one possible implementation, the method further includes: If the confidence level is greater than or equal to the preset confidence threshold, the document recognition result is converted into structured data with coordinates and stored in the AI ​​knowledge base; or, the target image region in the target document is subjected to secondary analysis; the target image region is the image region in the target document that is related to the confidence level being greater than or equal to the preset confidence threshold.

[0014] Secondly, this application also provides a computer program product. This computer program product includes a computer program that, when executed by a processor, implements the document recognition method described in the first aspect.

[0015] This application provides a document recognition method and computer program product. The document recognition method first identifies the document complexity of the target document and determines a text parsing strategy adapted to the document complexity. Since the text parsing strategy includes at least one of a lightweight model parsing strategy, a large model parsing strategy, and a lightweight collaborative large model parsing strategy, parsing the target document according to the text parsing strategy not only effectively solves the problem of insufficient recognition accuracy of mixed elements in complex documents, but also, by establishing a complexity evaluation and strategy matching mechanism, enables low-complexity content to be processed quickly locally, high-complexity content to obtain sufficient computing resources, and medium-complexity content to balance efficiency and accuracy through a collaborative mechanism. This significantly improves the accuracy and real-time performance of complex document recognition, and also significantly improves the resource utilization of edge devices and the cloud. Attached Figure Description

[0016] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is one of the flowcharts illustrating a document recognition method in one embodiment; Figure 2 This is one of the schematic diagrams illustrating the process of determining a text parsing strategy in one embodiment; Figure 3 This is a second flowchart illustrating a document recognition method in one embodiment; Figure 4 This is a schematic diagram of a code-first workflow in one embodiment; Figure 5 This is a schematic diagram of the table content parsing process in one embodiment; Figure 6 This is a second schematic diagram illustrating the process of determining a text parsing strategy in one embodiment; Figure 7 This is a schematic diagram of the image content parsing process in one embodiment; Figure 8 This is the third illustration of the process of determining a text parsing strategy in one embodiment; Figure 9 This is the third flowchart of a document recognition method in one embodiment; Figure 10 This is the fourth flowchart of a document recognition method in one embodiment; Figure 11 This is a schematic diagram of AI workflow priority processing in one embodiment; Figure 12 This is a schematic diagram of an exception handling process in one embodiment; Figure 13 This is a schematic diagram of the complete closed-loop feedback learning process in one embodiment; Figure 14 This is a schematic diagram illustrating the application and processing of document recognition results in one embodiment. Figure 15 This is a schematic diagram of the overall workflow of a document recognition method in one embodiment. Figure 16 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0017] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0018] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present application will now be described in detail with reference to the accompanying drawings and embodiments. Furthermore, the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The terms "first" and "second," etc., in the specification and claims of the embodiments of this application are used to distinguish different objects, not to describe a specific order of objects.

[0019] With the rapid development of AI technology, AI knowledge bases have become a core infrastructure for enterprise intelligent transformation. Enterprise AI knowledge bases collect and aggregate knowledge documents from various aspects of the enterprise, combining AI technology to provide users with diversified services. However, the massive amount of documents in AI knowledge bases contains complex documents with diverse formats, layouts, and highly specialized content, making document parsing and information extraction a key bottleneck restricting their efficiency.

[0020] In existing technologies, pre-trained models with a single parsing method are typically used for document recognition. These pre-trained models are either deployed purely in the cloud or purely locally, resulting in low recognition accuracy for complex documents and an inability to effectively utilize local and cloud computing resources.

[0021] For example, a common method for document parsing and information extraction is the Optical Character Recognition (OCR) model. However, the OCR model faces four challenges when processing such complex documents: First, the model is rigid and has poor adaptability: it relies on a single training model, lacks the ability to dynamically adjust, and is difficult to effectively handle multilingual mixed text, complex structured layouts (such as multi-column formats and nested tables), and low-quality scanned images (such as blur, tilt, and shadow interference).

[0022] Second, the process is rigid and lacks intelligence: The traditional "preprocessing-recognition-postprocessing" pipeline is static and linear, and cannot dynamically arrange the optimal processing strategy according to the specific type of document (such as technical drawings, medical reports, financial documents) and the real-time recognition status (such as real-time text recognition, real-time extraction of structured data, complex scenario adaptation and error correction mechanism, etc.), resulting in unstable accuracy.

[0023] Third, lack of collaboration and low efficiency: Inflexible scheduling of computing resources and lack of effective collaboration between edge computing and cloud computing power make it difficult to meet the high requirements of real-time performance, low bandwidth and data privacy in scenarios such as the Industrial Internet of Things.

[0024] Fourth, the system is closed and cannot evolve: lacking effective error feedback and self-learning mechanisms, the system cannot learn from human intervention, resulting in the recurrence of the same errors, stagnant recognition capabilities, and high maintenance costs.

[0025] In summary, existing OCR models have five main drawbacks in practical document recognition: Disadvantage 1: Insufficient recognition accuracy: The fixed model cannot be dynamically switched, making it difficult to handle multimodal elements in documents (such as charts, stamps, and handwritten annotations), resulting in low overall recognition accuracy for complex documents (such as medical bills and legal contracts), and a persistently high error rate in extracting key information.

[0026] Disadvantage 2: Inflexible and inefficient processing flow: The workflow lacks intelligent arrangement capabilities and cannot automatically optimize the order and parameters of model calls based on document content characteristics (such as performing layout analysis first and then calling a dedicated table recognition model), resulting in a lengthy processing chain and a single-page processing time that far exceeds the industry's advanced level.

[0027] Disadvantage 3: Poor system scalability and flexibility: The parsing method is singular, usually relying on a single model or pure cloud / pure local deployment. It cannot flexibly combine multiple models and computing resources according to business needs, resulting in poor system scalability and difficulty in adapting to rapidly changing business scenarios.

[0028] Disadvantage 4: Lack of dynamic decision-making ability: It is unable to quantitatively assess the complexity of documents, and it lacks automated and intelligent processing path decision rules based on the assessment results, relying heavily on manual presets and interventions.

[0029] Fifthly, the system lacks error correction and evolution mechanisms: the lack of a systematic anomaly handling, human intervention and feedback learning loop means that identified errors cannot be effectively corrected and used for system optimization, resulting in the system maintaining a low level of operation for a long time and the inability to continuously improve its intelligence level.

[0030] To address the aforementioned technical problems, this application provides a document recognition method and a computer program product. The document recognition method can be executed by a document recognition system that communicates with an AI knowledge base. This document recognition system can be installed in an electronic device or installed independently outside of the electronic device. The electronic device can be a personal computer, server, embedded system, or other devices. This application does not specifically limit its functionality. The following description uses a document recognition system as an example to illustrate the document recognition method.

[0031] Below, in conjunction with Figures 1 to 15 This application describes a document recognition method. To facilitate understanding of the document recognition method provided, the following exemplary embodiments will be used to illustrate the method in detail. It is understood that these exemplary embodiments can be combined with each other, and similar concepts or processes may not be repeated in some embodiments.

[0032] Reference Figure 1 This is a flowchart illustrating the document recognition method provided in an embodiment of this application, as shown below. Figure 1 As shown, the document recognition method includes the following steps 101 and 102.

[0033] Step 101: Identify the document complexity of the target document and determine a text parsing strategy that matches the document complexity; the text parsing strategy includes at least one of the following: lightweight model parsing strategy, large model parsing strategy, and lightweight collaborative large model parsing strategy.

[0034] The target document can be any document from the massive amount of documents contained in the AI ​​knowledge base, and the target document can be a document in PDF format, Word format, or other formats such as image format; in addition, the target document can be a plain text PDF document, a document in Word format containing tables and handwritten annotations, or a multi-column text and image document.

[0035] Document complexity can be understood as a comprehensive assessment of the difficulty of processing various content elements in a target document, reflecting the magnitude of computational resources required for document parsing.

[0036] The lightweight model parsing strategy can be understood as a strategy of using lightweight edge models for text parsing, also known as a fast parsing loop, which is suitable for processing low-complexity text content; for example, the lightweight edge model can be a Gemma3 model or a MobileNet model deployed locally or at the edge to implement basic OCR functions.

[0037] The large model parsing strategy can be understood as a strategy of using a large cloud model for text parsing, also known as a deep parsing strategy, which is suitable for processing highly complex content; for example, the large cloud model can be a Gemma3 network model deployed on a cloud server to parse the text content in a table. The lightweight collaborative large model parsing strategy can be understood as a strategy for text parsing by collaborating a local lightweight model and a cloud-based large model. It can be called a hybrid parsing loop, which optimizes efficiency and cost: the lightweight collaborative large model parsing strategy reduces cloud computing power consumption, reduces single-page processing time by more than 30%, and dynamically schedules the model library (such as giving priority to using edge lightweight models), reducing hardware costs by 40%. For example, after the local lightweight model completes the text parsing of simple icons in the target document, the cloud-based large model completes the text parsing of tables in the target document.

[0038] Specifically, to assess the document complexity of a target document, the document recognition system can first identify the document features of the target document, such as document features in the dimensions of page layout, content modality, image quality, and domain specialization. These document features can be preprocessed and preliminarily structurally analyzed using tools such as OpenCV and LayoutLM. Then, the document complexity of the target document is evaluated based on the identified document features.

[0039] For example, a document recognition system can calculate a complexity score (CS) by weighting the text features of each dimension shown in Table 1 and use it as the document complexity of the target document.

[0040] Table 1

[0041] As can be seen from Table 1, the CS score can be calculated based on the specific document features of different dimensions in the target document and the weight of each specific document feature, which means that the document complexity of the target document has been identified.

[0042] It should be noted that the document recognition system can include a hybrid programming engine, a multimodal model library, an intelligent scheduler, and a path decision rule library. The hybrid programming engine supports seamless integration of code scripts (such as Python) and AI workflows (such as Business Process Modeling Notation (BPMN)), and can dynamically adjust the model call order. The multimodal model library integrates large cloud-based models, lightweight edge models, and local algorithm models (such as table detection models), supporting hot-swappable replacement. The intelligent scheduler automatically selects the optimal text parsing strategy based on document type (such as PDF format, image format) and content characteristics (such as whether it contains tables). The path decision rule library has a built-in set of defining rules used to evaluate document complexity (such as calculating the CS score according to Table 1) and decide which parsing loop to call. The hybrid programming engine architecture allows developers to customize complex interface calls through code (such as calling blockchain APIs for data notarization), compensating for the functional limitations of pure workflows.

[0043] By using the quantified complex document definition rules and workflow scheduling rules shown in Table 1, the processing path selection is automated and optimized, reducing the reliance on expert experience and enabling the system to adaptively handle massive and varied document types, thus reducing operation and maintenance decision-making costs by 60%.

[0044] For example, when the target document is defined as a simple document (such as plain text, single column, clear printing, no stamps, or other complex elements) by document complexity, the fast parsing loop is determined as the document parsing strategy; when the target document is defined as a complex document (such as containing at least one high-weight specific document feature in Table 1) by document complexity, the deep parsing strategy is determined as the document parsing strategy; when the target document is defined as a hybrid document (such as a clear multi-column text and image document that is between a simple document and a complex document) by document complexity, the hybrid parsing loop is determined as the document parsing strategy.

[0045] Furthermore, when the target document includes both low-weight and high-weight specific document features as shown in Table 1, the document parsing strategy includes a lightweight model parsing strategy and a large model parsing strategy. The lightweight model parsing strategy and the large model parsing strategy can be dynamically switched. In this way, through the model hot-plug mechanism, a model container interface specification (ModelContainerInterface) is designed to support dynamic updates of the model library at runtime, thereby achieving the goal of seamless model replacement. Here, a container interface specification (ModelContainer Interface) can be designed to support dynamic updates of the model library at runtime.

[0046] like Figure 2 As shown, when the target document includes page 1 and page 2, and page 1 contains only low-weight specific document features, while page 2 contains both low-weight and high-weight specific document features, the text parsing strategies adapted to the document complexity of the target document include a lightweight model parsing strategy for parsing page 1 and a lightweight collaborative large model parsing strategy for parsing page 2.

[0047] Step 102: Parse the target document according to the text parsing strategy to obtain the document recognition result of the target document.

[0048] Specifically, when the target document is a simple document and the text parsing strategy is a lightweight model parsing strategy for simple documents, the lightweight model parsing strategy can be used to parse the target document. That is, the simple document is input into the edge lightweight model (such as the MobileOCR model) for parsing, and the parsing result output by the edge lightweight model is determined as the document recognition result of the target document; thereby saving cloud resources and reducing latency.

[0049] When the target document is a complex document and the text parsing strategy is a large model parsing strategy for complex documents, the large model parsing strategy can be used to parse the complex document. That is, the complex document is input into a large cloud model (such as the Gemma3 model or the SmolDocling model), and the large cloud model parses at least one high-weight specific document feature in the complex document and fuses the output multimodal parsing results to obtain the document recognition result. In this way, the powerful understanding ability of the large model ensures the accuracy of document recognition.

[0050] When the target document is a hybrid document and the text parsing strategy is a lightweight collaborative large model parsing strategy for hybrid documents, the lightweight collaborative large model parsing strategy can be used to parse the hybrid document. That is, the edge lightweight model first preprocesses (such as denoising, enhancement, etc.), then the cloud large model parses at least one high-weight specific document feature, and finally the local algorithm performs post-processing or local verification. Post-processing can be understood as using the edge lightweight model to continue parsing at least one low-weight specific document feature in the target document, and local verification can be understood as verifying the accuracy of the model output results; thus balancing efficiency and accuracy.

[0051] For example, when the local algorithm model includes a handwriting recognition model and / or a table structure recognition model, and the target document contains both tables and handwriting, the cloud-based large model can be used to parse the target document. If merged cells are detected in the table during parsing, the table structure recognition model is switched to parse the merged cells. Then, the cloud-based large model is switched back to parse the other cells in the table that do not contain merged cells, until the table parsing is complete. Next, the cloud-based large model is used to parse other document features in the target document besides tables. If handwriting is detected during parsing, the handwriting recognition model is switched to recognize the handwriting. Then, the cloud-based large model is switched back to parse the target document, until the target document parsing is complete, and the document recognition result of the target document is obtained.

[0052] The document recognition method provided in this application first identifies the document complexity of the target document and determines a text parsing strategy that matches the document complexity. Since the text parsing strategy includes at least one of a lightweight model parsing strategy, a large model parsing strategy, and a lightweight collaborative large model parsing strategy, parsing the target document according to the text parsing strategy not only effectively solves the problem of insufficient recognition accuracy of mixed elements in complex documents, but also enables low-complexity content to be processed quickly locally, high-complexity content to obtain sufficient computing resources, and medium-complexity content to balance efficiency and accuracy through a collaborative mechanism by establishing a complexity evaluation and strategy matching mechanism. This significantly improves the accuracy and real-time performance of recognizing complex documents, and also significantly improves the resource utilization of edge devices and the cloud.

[0053] Based on the above Figure 1 The method shown, in one example embodiment, involves identifying the document complexity of the target document in step 101 and determining a text parsing strategy adapted to the document complexity. The specific process in this embodiment can be achieved through… Figure 3 Steps 201 to 203 shown are implemented.

[0054] Step 201: Identify one or more target content elements in the target document; the target content elements are content elements related to the document complexity.

[0055] Step 202: Determine the content complexity of one or more types of target content elements in the target document, and determine the text parsing strategy corresponding to each content complexity.

[0056] Step 203: Determine the text parsing strategy that is compatible with the document complexity based on the text parsing strategy corresponding to each content complexity.

[0057] Among them, target content elements can be understood as content elements that affect the difficulty of parsing the document structure in the target document, and will directly affect the document complexity of the target document. Such as tables, pictures, formulas and other content elements with specific format or semantic features, can be identified by using image segmentation algorithms combined with semantic analysis models, or by locating regions with significant layout features or content density in the target document to classify elements and identify different categories of target content elements.

[0058] Content complexity can be understood as the degree of structuring and information density within a single target content element. Specifically, it can be calculated by using layout analysis algorithms to determine the number of rows and columns in a table and the merging of cells, or by using image feature extraction algorithms to assess the number of objects and hierarchical relationships in an image, thereby quantifying the parsing difficulty of the corresponding target content element.

[0059] Dynamic adaptation of text parsing strategies can be understood as matching parsing models with different computing resource configurations based on the content complexity level of each type of target content element. For example, assigning a type of target content element with high content complexity to a large cloud model for parsing, and assigning a type of target content element with low content complexity to a lightweight edge model for parsing.

[0060] Specifically, for at least one target content element identified from the target document, it can first be classified, such as into image content elements and / or table content elements. Then, the content complexity of each category of target content elements is evaluated; that is, when a category of target content elements includes at least one target content element, the number of content complexity values ​​evaluated can also be at least one. For example, if a category of target content elements includes three target content elements, one, two, or three content complexity values ​​can be evaluated. Next, the text parsing strategy corresponding to each content complexity value is determined, and based on the text parsing strategy corresponding to each content complexity value, a text parsing strategy adapted to the document complexity is determined. For edge-side lightweight models, they can be used to parse simple documents, as well as complex or hybrid documents, for each low-content-complexity target content element (such as at least one low-weight specific document feature in Table 1 above). When there are at least two low-content-complexity elements, the corresponding text parsing strategies are different. For cloud-based large models, they can be used to parse complex or hybrid documents, for each high-content-complexity target content element (such as at least one high-weight specific document feature in Table 1 above). When there are at least two high-content-complexity elements, the corresponding text parsing strategies are different. For collaboration between local lightweight models and cloud-based large models, they can be used to parse low-content-complexity and high-content-complexity target content elements in hybrid documents. They can also be used for preprocessing (denoising, binarization) by the edge-side lightweight model, followed by parsing of content elements (titles, tables, images, etc.) by the cloud-based large model (such as the LayoutM model) through the AI ​​workflow engine, and then table parsing by calling the cloud-based large model (such as Hehe Information's "Large Model Accelerator 2.0") through the AI ​​workflow engine, or switching to the edge-side lightweight model to parse low-resolution images through the AI ​​workflow engine. This allows us to determine the text parsing strategy corresponding to each content complexity, and to define the text parsing strategy corresponding to each content complexity as a text parsing strategy that is adapted to the document complexity.

[0061] For example, refer to Figure 4 The diagram shown illustrates the code-first workflow, as follows: Figure 4 As shown, when extracting document features from the input target document and determining its document type accordingly, if the target document is determined to be a PDF document containing tables, it is classified as a complex document by calculating its CS score (i.e., document complexity). In this case, the appropriate document parsing strategy is to use the Hehe Information Big Model Accelerator 2.0 cloud-based big model for parsing. Alternatively, if the target document is determined to be an image document containing low-resolution images, it is classified as a simple document by calculating its CS score (i.e., document complexity). In this case, the appropriate edge-based lightweight model for parsing is selected. The parsed structured data is then output in a specified format for export to enterprise systems (such as ERP or AI knowledge bases) to complete the knowledge entry.

[0062] The document recognition method provided in this application establishes a hierarchical processing mechanism for content elements, enabling differentiated parsing processing based on the characteristics of different content elements within a document. This significantly improves the overall recognition accuracy of complex documents while optimizing the collaborative utilization of local and cloud computing resources. It avoids performance bottlenecks or resource mismatches that occur when a single model processes complex documents, effectively balancing the investment in computing resources with the requirements for parsing quality.

[0063] Based on the above Figure 3 In one example embodiment of the method shown, step 202 involves determining the content complexity of one or more target content elements in the target document. The specific process of this embodiment can be implemented through the following steps.

[0064] If the target content element includes a table, the content complexity of the table is determined based on the table's layout complexity and content density.

[0065] Among them, layout complexity can be understood as the complexity of the table structure layout. Specifically, it can be achieved by methods such as row and column number detection, merged cell recognition, and nested table analysis, in order to quantify the degree of interference caused by the table structure to the parsing.

[0066] Content density can be understood as the amount of information carried per unit area. It can be achieved through character statistics and data item distribution detection to assess the intensity of the table content's demand for computing resources.

[0067] Specifically, for tables, a common high-complexity content element in complex documents, differentiated analysis can be performed based on the table's layout complexity and content density to determine its corresponding content complexity. That is, when a table is detected as a target content element in the document, the table's structure is first analyzed. This involves analyzing row and column distribution characteristics to identify whether there are merged cells across rows and columns, detecting the number of nested table levels, and counting the total number of cells to form a quantitative indicator of the table's layout complexity. Simultaneously, the table's content is scanned, counting the number of characters per unit area and detecting the density of relationships between data items to generate an evaluation parameter for the table's content density. The table's content complexity is then obtained by weighting layout complexity and content density.

[0068] For example, refer to Figure 5 The diagram shown illustrates the table content parsing process, as follows: Figure 5As shown, when three table areas are identified in the target document, the complexity of each table is first determined. If one table's complexity indicates the presence of complex cross-page / merged cells, the TableTransformer sub-model in the cloud-based large model can be used to reconstruct the table structure and parse the text content. If another table's complexity indicates it is a document / receipt, the document sub-model in the cloud-based large model can be used for parsing and the rule engine can be used for verification to extract key fields and verify logical compliance. If the last table's complexity indicates it is a simple rule table, the Tesseract-OCR sub-model in the cloud-based large model can be used in conjunction with the rule engine to parse the text content and directly output structured data. Then, the parsing results of the three tables are fused, and the fused result is used as part of the final document recognition result. In this way, multimodal fusion technology (image + text joint analysis) solves the recognition errors of traditional OCR in complex layouts (such as cross-page tables), achieving a parsing stability rate of 99.99%.

[0069] like Figure 6 As shown, when the target document contains Table 1, Table 2 and Table 3, and Table 1 contains merged cells, Table 2 is a ticket, and Table 3 is a simple rule table, the Table Transformer sub-model is used to reconstruct the table structure of Table 1 and parse the text content, the ticket sub-model is used to parse Table 2, and the Tesseract-OCR sub-model is used in conjunction with the rule engine to parse Table 3.

[0070] The document recognition method provided in this application establishes a dual evaluation system of structural features and content features, which can accurately identify table types with complex merged cells but sparse content. This effectively solves the problem of inappropriate parsing strategy selection caused by the complexity of table layout and the difference in content density, significantly improves the parsing accuracy of documents containing tables, and avoids excessive consumption of computing resources.

[0071] Based on the above Figure 3 In one example embodiment of the method shown, step 202 involves determining the content complexity of one or more target content elements in the target document. The specific process of this embodiment can be implemented through the following steps.

[0072] If the target content element includes an image, then the content complexity corresponding to the image is determined based on the element complexity of the image.

[0073] The element complexity of an image refers to the complexity of the combination of visual elements such as graphics, colors, and textures in the image. Specifically, it can be achieved by using image segmentation algorithms to extract the number of graphic outlines, using color histograms to count the dispersion of color distribution, and using convolutional neural networks to extract the length of texture feature vectors. This can be used to quantify the diversity of the internal structure of an image and provide a data foundation for subsequent content complexity classification.

[0074] The content complexity of an image can be understood as a document processing difficulty level based on the complexity of the image's elements. Specifically, a preset complexity threshold range can be used to map element complexity into three levels: low, medium, and high. This allows for dynamic matching of image content with different levels of complexity and appropriate parsing strategies.

[0075] Specifically, for images, a common content element in a document, the image type can be determined based on the complexity of the image elements. Image types can include, but are not limited to, simple icons / simple logos, complex icons / complex diagrams, natural images / natural photographs, etc. Then, based on the pre-set mapping relationship between different types of images and image content complexity, the content complexity of the image can be determined, so as to determine the text parsing strategy corresponding to the content complexity of the image.

[0076] For example, refer to Figure 7 The diagram shown illustrates the image content parsing process, as follows: Figure 7 As shown, when three image regions are identified in the target document, the image type of each image is first determined. If the content complexity representation of one image type corresponds to a simple icon / logo, a lightweight edge model can be called for classification and description to generate Alt text. If the content complexity representation of another image type corresponds to a complex chart / schematic diagram, a cloud-based VLM model can be called for image-text joint understanding to extract data and generate descriptive text. If the content complexity representation of the last image type corresponds to a natural image / photo, an object detection and scene classification model can be used to identify key objects and scene information. Afterward, the parsing results of the three images are fused and the fused result is used as part of the final document recognition result.

[0077] like Figure 8 As shown, when the target document contains Image 1, Image 2 and Image 3, and Image 1 is a simple icon, Image 2 is a complex chart, and Image 3 is a natural image, the edge-based lightweight model is used to classify and describe Image 1, the cloud-based VLM model is used to parse Image 2, and the object detection and scene classification model is used to identify Image 3.

[0078] It is understandable that, through the aforementioned... Figure 5 and Figure 7As described, a fine-grained internal judgment and processing flow was designed for key complex nodes such as images and tables. Based on their subtypes (e.g., images are categorized as icons, charts, and natural images), a dedicated model is invoked to achieve accurate identification of local complex elements within the document, greatly improving the parsing accuracy of local content. Thus, the document recognition method provided in this application supports switching to a larger model at any time (e.g., switching from Tesseract to Gemma3), improving the accuracy of processing different types of documents (e.g., corporate promotional PDFs, process drawings) by 20% to 50%.

[0079] The document recognition method provided in this application, by quantitatively analyzing the element complexity of graphics, colors, textures and other elements within an image, can not only accurately distinguish the complexity differences of different types of images and achieve precise matching of parsing strategies, but also effectively identify the true processing difficulty of image documents. This avoids the resource consumption caused by overusing large models for simple images, solves the problem of fluctuating recognition accuracy caused by ignoring differences in image elements in existing technologies, and achieves stable processing of complex image documents.

[0080] Based on the above Figure 1 In one example embodiment of the method shown, step 101 determines a text parsing strategy adapted to the document complexity. The specific process of this step in this embodiment can be achieved through… Figure 9 Steps 301 to 303 shown are implemented.

[0081] Step 301: If the document complexity is lower than the preset minimum complexity, determine a lightweight model parsing strategy that matches the document complexity.

[0082] Step 302: If the document complexity is higher than the preset maximum complexity, then determine the large model parsing strategy that is compatible with the document complexity.

[0083] Step 303: If the document complexity is between the preset minimum complexity value and the preset maximum complexity value, then determine the lightweight collaborative large model parsing strategy that is adapted to the document complexity.

[0084] The preset minimum complexity can be understood as a pre-set lower limit of the document complexity quantification threshold. Specifically, it can be implemented by weighting indicators such as the number of rows and columns in a table, image resolution, or text paragraph density. It can be used to trigger the edge lightweight model processing with the lowest resource consumption.

[0085] The preset maximum complexity can be understood as a pre-set upper limit for the quantification threshold of document complexity. Specifically, it can be achieved through comprehensive evaluation of parameters such as the complexity of multimodal element combination or semantic relevance. It can be used to start high-precision cloud-based large model processing.

[0086] Specifically, referring to Table 1 above, the CS score of the target document can be calculated, and this CS score can be used as the document complexity of the target document. The document complexity can then be compared with a preset minimum complexity value and / or a preset maximum complexity value to determine the relationship between the document complexity and these preset minimum and / or maximum complexity values. Subsequently, based on the mapping relationship between different document complexities and text parsing strategies, a text parsing strategy suitable for the document complexity of the target document can be determined.

[0087] In this way, when the document complexity is detected to be below the preset minimum complexity value, the locally deployed lightweight model is prioritized for parsing, avoiding the consumption of cloud computing resources. When the document complexity exceeds the preset maximum complexity value, the target document is uploaded to the cloud-based large model for full parsing, ensuring accurate identification of complex structures. For medium-complexity documents between the preset minimum and maximum complexity values, the local lightweight model completes the basic content parsing of low-complexity target content elements, while high-complexity target content elements are submitted to the cloud-based large model for processing. This two-level collaborative mechanism achieves optimal allocation of computing resources. This dynamic strategy selection mechanism effectively resolves the contradiction between limited local computing power and excessive cloud resource consumption.

[0088] For example, when the preset minimum complexity is 2 and the preset maximum complexity is 4, if CS≤2, the target document is determined to be a simple document with plain text, single column, clear printing, and no complex elements. In this case, a lightweight parsing strategy suitable for CS≤2 is determined. If CS≥4, the target document is determined to be a complex document containing at least one high-weight specific document feature from Table 1. In this case, a large model parsing model suitable for CS≥4 is determined. If 2<CS<4, the target document is determined to be a hybrid document between simple and complex documents (such as a clear multi-column text and image document). In this case, a lightweight collaborative large model parsing strategy suitable for 2<CS<4 is determined. Based on this, workflow loop calling rules can be set, and the intelligent scheduler can call the AI ​​workflow according to the following rules: If the target document is defined as a "simple document", then the "fast parsing loop" is invoked. Reason: To save cloud resources and reduce latency.

[0089] Path: Lightweight edge model (such as MobileOCR) -> Direct output.

[0090] If the target document is defined as a "complex document," then the "deep parsing loop" is invoked. Reason: A strong understanding of large models is required to ensure accuracy.

[0091] Path: Large cloud model (such as Gemma3, SmolDocling) -> Multimodal fusion -> Output.

[0092] If the target document is defined as a "hybrid document", then the "hybrid parsing loop" is invoked. Reason: To balance efficiency and accuracy.

[0093] Path: Edge preprocessing (denoising, enhancement) -> Cloud-based large model core analysis -> Local algorithm post-processing / verification.

[0094] During the IF recognition process, handwritten text was detected. The THEN dynamic subprocess switched to the handwriting recognition model.

[0095] During the IF recognition process, the merged cell table THEN dynamic subprocess switches to the table structure recognition model. Thus, using the multi-dimensional weighted scoring rules (complexity score CS) based on layout, content, quality, and professionalism as shown in Table 1, the document type is objectively defined, and a clear IF-THEN scheduling rule base is established accordingly. This enables the system to adaptively select the optimal solution, achieving automated and intelligent decision-making for the parsing loop.

[0096] Therefore, it can be understood that this application integrates a large cloud model with a lightweight edge model, and designs three processing loops: a fast parsing loop, a deep parsing loop, and a hybrid parsing loop. These loops are intelligently scheduled according to document definition rules, reducing cloud load by an average of 30% while ensuring accuracy. In this way, through the model library and the edge-cloud multi-loop collaborative mechanism, the ability to recognize complex documents, real-time performance, and resource utilization are improved.

[0097] The document recognition method provided in this application divides complexity ranges by preset thresholds and automatically switches text parsing strategies for different scenarios. While ensuring recognition accuracy, it significantly reduces resource consumption and achieves dynamic adaptation of document processing strategies. In low-complexity scenarios, it saves cloud computing resources; in high-complexity scenarios, it ensures accurate extraction of key information; and in medium-complexity scenarios, it balances processing speed and recognition accuracy through a collaborative processing mechanism. This hierarchical processing method effectively overcomes the technical defect of traditional methods where resource utilization and recognition accuracy cannot be simultaneously achieved, enabling accurate recognition of local complex elements within a document and greatly improving the parsing accuracy of local content.

[0098] Based on the above Figure 1 In one example embodiment of the method shown, to ensure higher accuracy of the document recognition results, the reliability of the document recognition results can be quantified, and anomaly handling can be performed when the reliability is insufficient. Based on this, the document recognition method provided in this application embodiment may further include a post-processing stage for the recognition results. The specific process of this post-processing stage in this embodiment can be achieved through… Figure 10 Steps 401 and 402 shown are implemented.

[0099] Step 401: Determine the confidence level of the document recognition result. The confidence level is used to characterize the accuracy of the document recognition result.

[0100] Step 402: If the confidence level is less than the preset confidence threshold, perform anomaly processing on the document recognition results.

[0101] Confidence level can be understood as a numerical indicator obtained through probability calculation or statistical methods. Specifically, it can be achieved by using the probability distribution of the model output, result consistency verification, or historical data matching degree, and is used to quantify the credibility of document recognition results.

[0102] The preset reliability threshold can be immediately set as a pre-defined critical value to trigger anomaly handling. This can be achieved through empirical setting, dynamic adjustment algorithms, or methods to determine the optimal value based on test data, and is used to filter out identification results with insufficient reliability. For example, the preset reliability threshold could be 90%.

[0103] Anomaly handling refers to the correction or review of low-confidence identification results. Specifically, it can be achieved through automatic correction by rule engines, manual review intervention, or multi-model collaborative verification, in order to improve the accuracy of the final output.

[0104] Specifically, for the document recognition result of the target document, the accuracy of the document recognition result can be quantified first, that is, the confidence level of the document recognition result can be determined. Here, multi-model voting mechanism and semantic consistency check can be used to calculate the confidence level of the document recognition result.

[0105] For example, the confidence level of the document recognition result can be calculated using the following formula (1).

[0106] Confidence = α×Model_Agreement + β×Semantic_Consistency + γ×Layout_Reasonableness (1) In Equation (1), Confidence represents the confidence level of the document recognition result, Model_Agreement represents the model consistency parameter, Semantic_Consistency represents the semantic consistency parameter, Layout_Reasonableness represents the layout reasonableness parameter, α represents the weight coefficient of the model consistency parameter Model_Agreement, β represents the weight coefficient of the semantic consistency parameter Semantic_Consistency, and γ represents the weight coefficient of the layout reasonableness parameter Layout_Reasonableness.

[0107] Regarding equation (1), it should be noted that the weight coefficient α ranges from 0.5 to 0.7, the weight coefficient β ranges from 0.2 to 0.4, and the weight coefficient γ ranges from 0.1 to 0.2. Furthermore, these three weight coefficients, α, β, and γ, reflect the differences in the importance of different dimensions to the final confidence score, and their relationship is: α > β > γ.

[0108] The three weighting coefficients mentioned above can be configured according to the document type. For example, when processing common documents in the enterprise knowledge base that have relatively standardized formats but diverse content (such as business contracts and technical reports), the three weighting coefficients can be configured as follows: α = 0.6, β = 0.3, γ = 0.1. In this way, while ensuring a high recall rate, model divergences and semantic anomalies can be effectively identified.

[0109] To further enhance adaptability, the three weighting coefficients mentioned above can be dynamically fine-tuned based on document type and business scenario. For example, when processing highly structured documents (such as invoices and forms), the weighting coefficient γ should be appropriately increased, i.e., adjusted to 0.15~0.25, as disruption of the layout structure is a significant indicator of recognition errors. When processing documents with strong semantic relationships (such as legal clauses and technical papers), the weighting coefficient β should be appropriately increased, i.e., adjusted to 0.35~0.45, as contextual semantic verification is more effective. When processing documents with complex layouts and poor image quality, the weighting coefficient α should be maintained or increased, relying on the multi-model voting results to determine reliability.

[0110] Continuing, for the confidence level of the determined document recognition result (such as calculated by formula (1)), it can be compared with the preset confidence threshold to determine whether it is sufficient to trigger the abnormal handling process; and, if the confidence level of the document recognition result is less than the preset confidence threshold, the document recognition result can be abnormally handled.

[0111] It should be noted that, referring to Figure 11 The diagram shown illustrates the priority processing of AI workflows. Figure 11As shown, the hybrid programming engine of the document recognition system can determine the document complexity of the target document through semantic analysis. When this document complexity characterizes the target document as a complex document, specifically a medical report containing handwritten annotations, a lightweight collaborative large-model parsing strategy adapted to the document complexity can be determined. This strategy includes edge-end preprocessing (such as contrast enhancement), cloud-based large-model (such as the SmolDocling model) deep parsing, and local algorithm verification. Edge-end preprocessing (such as contrast enhancement) can reduce cloud data transmission volume and shorten response time by 50%. After using this lightweight collaborative large-model parsing strategy to parse the target document, the document recognition result is obtained, and its confidence level is determined. Further, if the confidence level of the document recognition result is lower than a preset confidence threshold, an anomaly handling mechanism can be triggered. This involves determining the corresponding anomaly handling process based on the confidence level of the document recognition result, i.e., manual intervention for abnormal recognition results, or system correction combined with manual intervention. The specific anomaly handling mechanism can be found below. Figure 12 and with Figure 12 Related descriptions.

[0112] The document recognition method provided in this application constructs a quality monitoring layer through confidence calculation, which can proactively identify potential errors and select differentiated correction strategies for different confidence levels. It can dynamically identify low-quality results in the document recognition process, thereby significantly reducing the risk of systematic errors caused by limitations such as model recognition capabilities while maintaining document recognition efficiency.

[0113] Based on the above Figure 10 In one example embodiment of the method shown, step 402 involves anomaly handling of the document recognition result. The specific process of this embodiment can be implemented through the following steps.

[0114] If the confidence level is within the preset abnormal confidence level range, the document recognition results are corrected according to the rule engine; the preset confidence threshold is greater than the maximum value of the preset abnormal confidence level range.

[0115] The preset anomaly confidence range can be understood as a confidence interval between a preset confidence threshold and a manual review trigger threshold. Specifically, it can be implemented using a numerical interval division method. For example, the confidence level can be divided into 70%-89% as the anomaly range, the preset confidence threshold can be set to 90%, and the manual review trigger threshold can be set to 69%. This preset anomaly confidence range can be used to identify medium-risk identification results in document recognition results that may be programmatically correctable. For instance, when the preset confidence threshold is 90%, the preset anomaly confidence range can be 70%-89%.

[0116] A rule engine can be understood as an automated processing module with built-in domain knowledge verification rules. Specifically, it can be implemented using regular expression matching, semantic template mapping, or knowledge graph querying. For example, it can configure rules for verifying ticket numbers and amounts in a ticket recognition scenario. Through this rule engine, abnormal fields in the document recognition results can be logically verified and their formats corrected using structured rules.

[0117] Specifically, when the confidence level of a document recognition result falls within a preset abnormal confidence level range, it indicates that there are suspicious errors in some fields of the document recognition result, but the errors have not yet reached a severity level requiring manual intervention. At this point, the rule engine correction process can be automatically triggered: First, the key fields in the document recognition result are checked for format compliance, such as verifying whether the date field conforms to the YYYY-MM-DD format; second, logical contradictions are detected through semantic correlation analysis, such as verifying the consistency between contract amount and invoice amount; finally, the standard terminology library in the domain knowledge base is called for terminology correction, such as replacing incorrectly identified professional terms with standardized expressions in the knowledge base. The entire correction process is completed without manual intervention. Only when the rule engine cannot resolve the anomaly will the corrected result be recalculated for confidence and proceed to the next processing stage.

[0118] For example, refer to Figure 12 The diagram shown illustrates the exception handling process, as follows: Figure 12 As shown, when the confidence level of the document recognition result falls within a preset abnormal confidence level range, an anomaly handling process can be automatically triggered. An automatic annotation algorithm automatically marks suspicious areas in the document recognition result. Specifically, a heatmap of the document recognition result can be generated based on attention-based heatmap technology, accurately locating and marking suspicious areas with different colored borders. For example, red borders mark low-confidence text areas, and yellow borders mark medium-confidence image areas, each accompanied by a confidence score and suggested corrections. The annotation results are then temporarily stored in a queue awaiting review. Subsequently, each suspicious area in the queue undergoes an anomaly type determination. This workflow automatically marks suspicious areas, allowing the expert system to be quickly integrated through a low-code platform, improving document recognition efficiency by 80%.

[0119] When the anomaly type is determined to be a simple error that can be corrected using rules, a rule engine can be used to automatically correct simple errors in the document recognition results. Specifically, when the rule engine can perform character-level, lexical-level, and semantic-level corrections, it can correct common OCR error mappings at the character level, such as "0"→"O", "1"→"I"; and / or perform spell checks using a domain dictionary at the lexical level; and / or verify the reasonableness of contextual semantics through semantic-level corrections. The corrected result is then determined, and its validity is assessed. If the correction passes verification, it is output; otherwise, it is assigned to a manual review platform for intervention. This process aims to improve the confidence level of the document recognition results until it exceeds a pre-set confidence threshold.

[0120] The document recognition method provided in this application, by combining a confidence-based hierarchical processing mechanism with a rule engine, not only achieves accurate and automated correction of medium-risk recognition results, avoiding unnecessary resource consumption, but also enables rapid and targeted automated correction of document recognition results with potential errors that have not yet reached the threshold for manual processing, significantly reducing the workload of manual review. At the same time, it improves the response speed and accuracy of anomaly handling, making it suitable for scenarios with high processing timeliness requirements, such as financial bill recognition and medical report parsing, and optimizing the allocation of computing resources while ensuring recognition accuracy.

[0121] Based on the above Figure 10 In one example embodiment of the method shown, step 402 involves anomaly handling of the document recognition result. The specific process of this embodiment can be implemented through the following steps.

[0122] If the confidence level is less than the minimum value of the preset abnormal confidence level range, the document recognition result is sent to the manual review platform for manual correction to obtain the manual correction result; the manual correction result is used as training data to participate in at least one of the following: rule engine optimization, lightweight model incremental training, and large model incremental training.

[0123] The minimum value of the preset anomaly confidence range can be understood as the lower limit threshold of the pre-set confidence interval, which is used to accurately identify the identification result with the lowest accuracy.

[0124] The manual review platform can be understood as a data processing system that deploys manual annotation interfaces. Its role is to transform low-confidence identification results into traceable correction data.

[0125] Rule engine optimization can be understood as the process of updating the logical judgment rule base based on the results of manual correction. Specifically, it can be implemented using dynamic compilation technology of conditional expressions, which can improve the coverage of structured data processing rules.

[0126] Lightweight model incremental training can be understood as updating the parameters of a lightweight model deployed at the edge using the results of manual correction, which enhances the real-time recognition capability of localized scenarios.

[0127] Incremental training of large models refers to using manually corrected results as the training dataset for large models in the cloud. This can be achieved using parameter fine-tuning techniques, and its role is to improve the generalization learning ability of complex document features.

[0128] Specifically, when the confidence level of a document recognition result is lower than the minimum of the preset anomaly confidence level range (e.g., confidence level < 70%), it can be determined that there are complex errors in the document recognition result that require manual judgment and handling. At this time, a manual intervention mechanism can be automatically triggered, that is, the low-confidence document recognition result is pushed to the manual review platform's anomaly handling process. The low-confidence recognition result is pushed to the manual review platform for correction, and the manually corrected result is used as training data to participate in subsequent rule engine optimization, lightweight model incremental training, and / or large model incremental training. Through this anomaly handling mechanism of manual review + algorithm verification, the recognition error rate in key scenarios such as financial instruments can be reduced from 15% to below 2%.

[0129] For example, continue to refer to Figure 12 When the confidence level of the document recognition result is less than the minimum value of the preset anomaly confidence level range, and the anomaly type of the suspicious area in the review queue is a complex error requiring manual judgment, the document recognition result can be assigned to the manual review platform. This manual review platform includes task queue management, collaborative annotation tools, manual review templates, and a manual review process. The task queue management function can assign tasks according to priority and domain expertise. The collaborative annotation tool supports simultaneous review by multiple people, conflict detection, and resolution. The manual review templates preset review points for different document types (contracts, invoices, reports). The manual review process includes a first-level review based on simple corrections by annotation personnel, a second-level review by domain experts handling complex semantic issues, and an arbitration mechanism where disputed cases are ultimately decided by senior experts.

[0130] pass Figure 12 It is understandable that this application, through intelligent anomaly handling process and manual intervention process, can not only pre-trigger manual review based on confidence level, but also achieve traceability of document recognition results by combining blockchain technology, meeting the compliance requirements of industries such as finance and healthcare. Moreover, the workflow can automatically mark suspicious areas, and the expert system can quickly intervene through the low-code platform, improving efficiency by 80%, thereby ensuring the accuracy of document recognition in critical scenarios.

[0131] Based on this, complex errors in document recognition results are subject to review quality monitoring and optimization. Review quality monitoring includes cross-validation, sampling review, and performance evaluation. Cross-validation can be understood as randomly assigning the same task to multiple reviewers to check the consistency of results. Sampling review can be understood as conducting quality checks on 10% of the reviewed tasks. Performance evaluation can be understood as evaluating reviewers based on indicators such as review accuracy, review efficiency, and review consistency. Review quality optimization includes intelligent pre-filling, batch operations, and templated responses. Intelligent pre-filling can be understood as the document recognition system providing multiple correction suggestions, which reviewers only need to select or fine-tune. Batch operations can be understood as batch correction of similar errors. Templated responses can be understood as commonly used correction instruction templates, reducing repetitive input.

[0132] Reviewers verify and correct document recognition results through a visual interface. The manual review results are then fed back to the document recognition system, which updates its results based on the corrections. This triggers a feedback learning mechanism, where manual corrections are simultaneously labeled as training data. This training data is used to optimize the rule engine, lightweight model training, and / or large model incremental training modules. The rule engine automatically generates new validation rules by analyzing logical error patterns in the manual corrections. Lightweight model incremental training uses an incremental learning algorithm to quickly update weight parameters on a local device. Large model incremental training fine-tunes the system by fusing new samples in a cloud-based training cluster. This forms a closed-loop feedback learning mechanism from anomaly detection to model optimization, continuously feeding manual corrections back into the system for self-evolution, thereby continuously improving the system's ability to handle extremely complex documents.

[0133] In this way, through a closed-loop feedback learning mechanism, the document recognition system can continuously learn from manual corrections, improving the overall recognition accuracy by 0.5%-1% per month, and shortening the adaptation time for new document types from 2-3 weeks to 3-5 days.

[0134] For example, refer to Figure 13 The diagram shown illustrates the complete closed-loop feedback learning process. Figure 13 As shown, the manual correction results obtained through the manual review platform can be collected in real time and placed in the amendment example database. Then, each collected manual correction result is preprocessed such as data cleaning and labeling. A training dataset is then constructed based on the preprocessed manual correction results, and a learning strategy is selected based on the training dataset. The selectable learning strategies include online fine-tuning of the model using incremental learning, construction of a difficult example sample library using difficult example mining, and optimization of the rule engine using rule updates.

[0135] Continue, as Figure 13 As shown, the model learning results obtained through online model fine-tuning can be used for model version management. Model versions can be stored in a model version repository or used for A / B testing. The constructed difficult example sample library can be stored there or used for targeted model retraining, with the training results also used for A / B testing. The optimized rule engine can be updated with rule library versions, and the updated results can be used to verify rule effectiveness. Both the model A / B test results and the rule effectiveness verification results can be evaluated to determine if the performance has improved. If the performance has improved, a new lightweight model, large model, and / or rule engine can be deployed. If the performance has decreased, the previous version of the lightweight model, large model, and / or rule engine can be rolled back. Afterward, the document recognition system performance continues to be monitored, and the next round of feedback learning loops begins. Through this multi-level anomaly handling mechanism, the final accuracy of low-confidence cases can be increased from 65% to 99.5%. The rule engine automatically corrects and resolves approximately 40% of abnormal cases, allowing manual review resources to focus on truly complex cases, reducing labor costs by 35%.

[0136] It should be noted that, for Figure 13 The data collection scope for manually corrected results includes the text content and coordinates of the manually corrected text, changes in confidence level before and after correction, correction type classification (such as character errors, semantic errors, layout errors, etc.), reviewer information, and review time. For each manually reviewed result in preprocessing, the data processing workflow includes data cleaning and labeling, data anonymization, quality filtering, and standardization. Data anonymization involves removing sensitive personal information of the corresponding reviewer; quality filtering removes low-quality correction examples (such as excessively short review time or unreasonable correction content); and standardization unifies the data format and adds metadata tags. For incremental learning, the applicable scenario is daily small-batch manual correction results. Its technical implementation methods include online learning algorithms, catastrophic forgetting prevention algorithms, learning rate control methods, and update frequency methods. The online learning algorithm is based on stochastic gradient descent (SGD) for model parameter fine-tuning; the catastrophic forgetting prevention algorithm uses elastic weight consolidation (EWC) technology; the learning rate control method dynamically adjusts the learning rate to balance new and old knowledge; and the update frequency method is daily scheduled incremental updates.

[0137] for Figure 13Difficult cases are identified through mining, and can be defined as cases with the same pattern of errors identified multiple times, cases where multiple reviewers have inconsistent revision opinions, and / or cases of systematic identification problems in a specific type of document. Methods for difficult case mining include cluster analysis and association rule mining. Cluster analysis identifies clusters of error patterns, while association rule mining discovers the correlations between errors. For targeted model retraining, a dedicated training set of difficult cases can be constructed, and training can be performed by adjusting the loss function, increasing the weight of difficult cases, and using specific model optimization and problem-solving techniques.

[0138] for Figure 13 The optimization of the rule engine can include rule discovery, rule validation, and rule management. Rule discovery involves automatically extracting new rules from successful amendment cases, analyzing correction patterns based on decision trees, and generating candidate rules. Rule validation involves testing the effectiveness of new rules on a validation set and evaluating rule coverage, accuracy, and conflict status. Rule management includes version control, conflict resolution, and effect monitoring. Version control refers to the versioning management of the rule engine, conflict resolution involves the detection and handling of conflicts between new rules and existing rules, and effect monitoring involves the continuous tracking of the effectiveness of the rules.

[0139] for Figure 13 The model version management system includes version naming conventions, a version rollback mechanism, and version performance tracking. The version naming conventions include the major version, this version, and revision number-date. The version rollback mechanism retains historical versions and supports rapid rollback. Version performance tracking fully records the performance metrics for each version. For deploying new lightweight models, large models, and / or rule engines, a gradual deployment strategy can be adopted, including shadow mode, A / B testing, and full deployment. Shadow mode allows new models to run in parallel without affecting actual performance, and the results are compared. A / B testing involves small-scale testing to statistically verify the significance of the results. Full deployment involves a complete replacement after successful verification.

[0140] for Figure 13 The effectiveness evaluation involves an evaluation index system that includes accuracy indicators, efficiency indicators, and business indicators. Accuracy indicators include character-level accuracy, vocabulary-level accuracy, and semantic-level accuracy; efficiency indicators include processing time, human intervention rate, and automatic correction success rate; business indicators include recognition performance for different document types and the accuracy of key information extraction. For Figure 13The closed-loop optimization involved involves continuous monitoring and optimization through real-time monitoring, regular evaluation, and quarterly reviews. Real-time monitoring provides 24 / 7 system performance monitoring, regular evaluation provides weekly performance evaluation reports, and quarterly reviews involve in-depth analysis of the system's evolution and adjustments to the learning strategy. By establishing a complete feedback loop from manual correction to model updates, and employing multiple learning strategies such as incremental learning, difficult example mining, and rule optimization, the system's monthly accuracy improves by 0.5%-1%. Model version management and gradual deployment ensure the learning process is safe and controllable, achieving continuous system optimization and adaptive capabilities.

[0141] The document recognition method provided in this application establishes a three-level collaborative optimization mechanism involving manual correction and multi-module collaborative training. This effectively solves the problem of insufficient accuracy in recognizing complex documents due to a single parsing model. A manual review mechanism triggered by a confidence threshold ensures the reliability of recognition results for extremely complex documents. Simultaneously, the manual correction results are used for rule engine optimization and incremental training of models of different scales, overcoming the technical deficiency of low utilization of error samples in traditional methods and achieving continuous autonomous improvement of the system's recognition capabilities. Furthermore, by differentiating the training methods of lightweight and large models, the method balances the real-time requirements of edge computing scenarios with the learning capabilities of complex features in the cloud, forming a multi-level optimization system.

[0142] Based on the above Figure 10 In one example embodiment of the method shown, for the confidence level determined in step 401, considering that when the confidence level reaches the maximum confidence threshold, in addition to outputting the document recognition result, further corrections can be made to the regions where the confidence level does not reach 100%. Based on this, the document recognition method provided in this application embodiment may further include a high-confidence recognition result post-processing flow, which in this embodiment can be implemented through the following steps.

[0143] If the confidence level is greater than or equal to the preset confidence threshold, the document recognition result is converted into structured data with coordinates and stored in the AI ​​knowledge base; or, the target image region in the target document is parsed in a secondary manner; the target image region is the image region in the target document that is related to the confidence level being greater than or equal to the preset confidence threshold.

[0144] Among them, structured data with coordinates can be understood as a data form that associates and encodes the text content of the document recognition result with its spatial location information in the target document. Specifically, it can be implemented using XML or JSON format combined with two-dimensional coordinate system annotation, so that subsequent retrieval can quickly locate the content based on spatial relationships.

[0145] The target image region refers to a high-confidence region in the target document that has been confirmed by confidence assessment, and where there are anomalies in the high-confidence region that cause the confidence score to fall below the percentage. Specifically, it can be defined by bounding box coordinates or semantic segmentation results, allowing the document recognition system to perform fine-grained processing on the verified region and avoid the resource consumption caused by repeated parsing of the entire document.

[0146] Specifically, when the confidence level of the document recognition result is greater than or equal to a preset confidence threshold (e.g., confidence level ≥ 90%), a data storage mode or a secondary parsing mode can be executed. In the data storage mode, the document recognition result is converted into structured data with coordinates. For example, a mapping relationship is established between table content and its row and column positions on the page, enabling the AI ​​knowledge base to build an index by spatial dimensions and improve the efficiency of subsequent intelligent retrieval. In this way, the structured data output and multi-system integration can enhance the compatibility of industry applications. Specifically, when the document recognition result is output as JSON data with coordinate information, it can support seamless integration with ERP and BI tools, shortening the reimbursement cycle by 60% in enterprise financial and tax automation scenarios. Furthermore, AI knowledge component libraries (such as Hehe Information open-source tools) help industry developers build personalized document processing solutions, thereby enhancing the compatibility of industry applications.

[0147] Alternatively, in secondary parsing mode, the code script in the document recognition system calls the SigLIP visual editor to perform secondary analysis on the target image region in the target document. For example, when the target image region is a medical report containing chief editor annotations, image super-resolution reconstruction can be performed on the text region in the medical report that is obscured by coffee stains, and the impact of edge blurring on the document recognition result can be eliminated through local enhancement operations. The two modes are implemented through a dynamic selection mechanism: when the AI ​​knowledge base needs to quickly accumulate standardized data, the data storage mode is executed first; when critical areas require redundant verification, the secondary parsing mode is activated. Figure 14 As shown, when the confidence level of the document recognition result is greater than or equal to the preset confidence threshold, the document recognition result is converted into structured data with coordinates and stored in the AI ​​knowledge base; or, the target image region in the target document is subjected to secondary analysis.

[0148] The document recognition method provided in this application achieves the association management of document content and spatial information through coordinate structured storage, enabling the AI ​​knowledge base to have retrieval capabilities based on layout features. Simultaneously, the local secondary parsing mechanism avoids the computational redundancy of traditional full-document repetitive processing, allowing computational resources to be concentrated on quality enhancement of key areas. Structured data storage significantly improves the retrieval speed and content relevance of the knowledge base, while local secondary parsing reduces computational resource consumption while ensuring accuracy in key areas. The dynamic selection mechanism of two processing modes allows the system to flexibly adjust processing strategies according to business needs, optimizing resource utilization while ensuring processing quality.

[0149] For example, refer to Figure 15 The diagram shown illustrates the complete workflow of the document recognition method, starting from... Figure 15 Understandably, by establishing a complete, feedback-enabled closed-loop automated chain from feature extraction, intelligent decision-making, multi-loop processing to quality verification and knowledge output of target documents, dynamic model scheduling and complex process orchestration are achieved. The entire process can decouple computation and data dependencies through a hybrid approach of code scripts and AI workflows, and layered API design (such as the HybridFlow framework), supporting flexible adjustments to processing logic. It should be noted that... Figure 15 The specific processes involved can be referred to in the aforementioned embodiments. They will not be repeated here.

[0150] It should be noted that although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. On the contrary, the steps depicted in the flowchart can be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0151] Figure 16 A schematic diagram of a hardware architecture suitable for implementing embodiments of this application is shown, such as... Figure 16 As shown, the electronic device 1600 specifically includes a processor 1601, a communication interface 1602, a memory 1603, and a communication bus 1604. The processor 1601, communication interface 1602, and memory 1603 communicate with each other via the communication bus 1604. The memory 1603 stores programs that can be executed by the processor 1601. The processor 1601 executes the programs stored in the memory 1603 to implement… Figure 1 The steps of the method shown are as follows.

[0152] The communication bus 1604 mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 1604 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 16The bus is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. Communication interface 1602 is used for communication between the aforementioned electronic device and other devices. Memory 1603 may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor 1501. The aforementioned processor 1601 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc., or it may be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0153] In another embodiment of this application, a computer-readable storage medium is provided, which stores a computer program that, when run on a computer, causes the computer to perform the actions described in the above embodiments. Figure 1 The steps of the method shown are as follows.

[0154] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape, etc.), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0155] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A document recognition method, characterized in that, The method includes: Identify the document complexity of the target document and determine a text parsing strategy that is suitable for the document complexity; the text parsing strategy includes at least one of a lightweight model parsing strategy, a large model parsing strategy, and a lightweight collaborative large model parsing strategy. The target document is parsed according to the text parsing strategy to obtain the document recognition result of the target document.

2. The method according to claim 1, characterized in that, The process of identifying the document complexity of the target document and determining a text parsing strategy adapted to the document complexity includes: Identify one or more target content elements in the target document; the target content elements are content elements related to the complexity of the document. Determine the content complexity of one or more types of target content elements in the target document, and determine the text parsing strategy corresponding to each content complexity. Based on the text parsing strategy corresponding to each of the aforementioned content complexities, a text parsing strategy adapted to the document complexity is determined.

3. The method according to claim 2, characterized in that, Determining the content complexity of one or more target content elements in the target document includes: If the target content element includes a table, then the content complexity of the table is determined based on the table's layout complexity and content density.

4. The method according to claim 2, characterized in that, Determining the content complexity of one or more target content elements in the target document includes: If the target content element includes an image, then the content complexity corresponding to the image is determined based on the element complexity of the image.

5. The method according to any one of claims 1 to 4, characterized in that, The determination of a text parsing strategy adapted to the document complexity includes: If the document complexity is lower than the preset minimum complexity, then determine the lightweight model parsing strategy that is compatible with the document complexity. If the document complexity is higher than the preset maximum complexity, then a large model parsing strategy that is compatible with the document complexity is determined. If the document complexity is between the preset minimum complexity value and the preset maximum complexity value, then the lightweight collaborative large model parsing strategy adapted to the document complexity is determined.

6. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Determine the confidence level of the document recognition result, wherein the confidence level is used to characterize the accuracy of the document recognition result; If the confidence level is less than a preset confidence threshold, the document recognition result will be processed as an anomaly.

7. The method according to claim 6, characterized in that, The anomaly handling of the document recognition results includes: If the confidence level is within a preset abnormal confidence level range, the document recognition result is corrected according to the rule engine; the preset confidence threshold is greater than the maximum value of the preset abnormal confidence level range.

8. The method according to claim 6, characterized in that, The anomaly handling of the document recognition results includes: If the confidence level is less than the minimum value of the preset abnormal confidence level range, the document recognition result is sent to the manual review platform for manual correction to obtain the manual correction result; the manual correction result is used as training data to participate in at least one of rule engine optimization, lightweight model incremental training and large model incremental training.

9. The method according to claim 6, characterized in that, The method further includes: If the confidence level is greater than or equal to the preset confidence threshold, the document recognition result is converted into structured data with coordinates and stored in the AI ​​knowledge base; or, the target image region in the target document is subjected to secondary analysis; the target image region is the image region in the target document that is related to the confidence level being greater than or equal to the preset confidence threshold.

10. A computer program product, characterized in that, The computer program product includes instructions that, when executed, cause the method as described in any one of claims 1-9 to be implemented.

Images