A confidence distribution based multi-engine optical character recognition adaptive switching and bottom-up method

Abstract

The application discloses a multi-engine optical character recognition adaptive switching and bottom-up method and system based on confidence distribution. A local or end-side first OCR engine is used as a main path to recognize an image and output a first result containing confidence of each text block. Global confidence statistics representing overall quality and local low-confidence distribution statistics representing low-confidence text block distribution are calculated based on the result. A variable switching strategy in the running period is read. When any index exceeds a threshold, a second OCR engine based on a visual language model is called to bottom up, instead of fixed multi-engine parallel or character-level fusion. When the second engine fails, times out or does not meet the acceptance condition, the first result is returned and the source is recorded. For multi-image documents, concurrent control and request throttling are performed on triggered remote calls, and the remote calls are merged in sequence. The application considers speed and cost, improves recognition accuracy on demand, and has failure bottom-up and upstream friendliness.

Description

Technical Field

[0001] This invention relates to the fields of optical character recognition and intelligent document processing technology, and in particular to a method, system, device, and medium for adaptive switching and failure fallback between a local or edge-side optical character recognition engine and an optical character recognition engine based on a visual language model, based on the confidence distribution of the recognition results of the first engine. Background Technology

[0002] Document digitization, invoice and contract recognition, and other scenarios widely rely on optical character recognition. Existing solutions have the following limitations.

[0003] Firstly, a single-engine fixed approach: using only local or edge-side optical character recognition engines results in fast response and low cost, but for low-quality images, such as blurry, complex layouts, stamp obscuration, handwritten characters, and tables, the recognition accuracy fluctuates greatly; using only high-precision remote models results in high latency, high pay-as-you-go costs, and concurrency is constrained by upstream service rate limiting.

[0004] Secondly, fixed multi-engine parallelism and result fusion: multiple optical character recognition engines are always executed for the same image, and then compared, voted or fused at the character level. Such schemes always produce multiple recognition results, causing unnecessary secondary recognition and cost waste for high-quality images.

[0005] Third, low-confidence local re-identification: After cropping the low-confidence characters or text segments in the first round of identification, optical character recognition is performed again. This still remains at the level of traditional optical character recognition reprocessing, without introducing heterogeneous high-capability models, and also lacks process continuity guarantees when remote calls fail.

[0006] Fourth, parallel supplementation of optical character recognition and generative artificial intelligence: The source information is simultaneously fed into the optical character recognition model and the generative artificial intelligence model, and the output is determined according to the consistency or acceptance threshold. If the condition is not met, it is transferred to manual processing. This is a parallel supplement and manual backup. It is not based on local optical character recognition as the main path, but rather on calling remote models as needed according to the quality distribution of its results, and reverting to local results if remote models fail.

[0007] In summary, existing technologies lack an integrated solution that primarily uses local optical character recognition, triggers heterogeneous visual language model optical character recognition as a fallback based on the confidence distribution of the results, gracefully rolls back local results and marks the source when remote calls fail, and performs request management for batch remote calls. Summary of the Invention

[0008] The technical problem to be solved by this invention is: in the optical character recognition processing of multi-image or multi-page documents, how to use low-latency local optical character recognition as the main recognition path, accurately identify images with degraded local optical character recognition based on the overall quality and local quality distribution of its output results, trigger high-complexity heterogeneous visual language model optical character recognition on demand, and maintain the continuity of the recognition process and the stability of requests in scenarios such as remote service failure, timeout, or batch high concurrency.

[0009] To address the aforementioned technical problems, the technical solution adopted by this invention is summarized as follows: A local or edge-side first optical character recognition engine is used as the main path to recognize image units, outputting a first recognition result containing confidence scores for each text block; based on the first recognition result itself, two complementary statistics are calculated: a global confidence statistic (denoted as G in this paper) characterizing the overall recognition quality of the entire image, and a local low-confidence distribution statistic (denoted as L in this paper) characterizing the distribution degree of low-confidence text blocks; the engine switching strategy readable during runtime is read; when either of the two types of statistics exceeds the corresponding threshold, a trigger is determined, and only when triggered is a second optical character recognition engine based on a visual language model or a multimodal generation model called as a fallback, rather than fixed multi-engine parallelism or character-level fusion; if the second engine call is successful, its recognition result is adopted; if the call fails, times out, or the acceptance conditions are not met, the first recognition result is rolled back and the source is recorded with structured metadata; for objects to be recognized containing multiple image units, only the triggered remote calls are subject to concurrency control and request throttling, and the final recognition results of each image unit are merged in sequence.

[0010] The beneficial effects of this invention are as follows: remote visual language models are invoked only for degraded images that are truly necessary, significantly reducing unnecessary remote recognition requests, lowering average latency and network and billing overhead, while maintaining overall recognition accuracy; two complementary indices, global confidence statistic G and local low confidence distribution statistic L, can simultaneously detect overall degradation and local concentrated degradation; graceful rollback when remote calls fail ensures that the recognition process is not interrupted and the source audit is retained; concurrency control and request throttling ensure that batch processing is friendly and stable to upstream services. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the overall structure of the system of the present invention; Figure 2 This is a flowchart illustrating the switching decision and failure rollback process based on global and local dual indicators in this invention. Figure 3 This is a schematic diagram illustrating the concurrency control and request throttling of multiple image documents according to the present invention; Figure 4 This is a flowchart illustrating the overall process of applying the present invention to PDF document processing. Detailed Implementation

[0012] The present invention will be further described below with reference to embodiments. In this embodiment, the first optical character recognition engine is a visual recognition engine deployed locally or on the edge, and the second optical character recognition engine is a cloud-based visual language model invoked via an artificial intelligence gateway.

[0013] After recognizing a single image unit, the first optical character recognition engine outputs a first recognition result, which includes at least the recognized text and multiple text blocks. Each text block contains text content, confidence level, and optional text box coordinates.

[0014] The global confidence statistic G is used to characterize the overall recognition quality of the entire image. In one embodiment, the global confidence statistic G is the weighted average confidence score of the entire image, and its weight is determined according to the text length or text box area of ​​each text block, or it can be determined according to the number of characters, text box width, or text box height; in other embodiments, the global confidence statistic G can also be the average confidence score, median confidence score, confidence score quantile, confidence score variance, or confidence score entropy.

[0015] The local low-confidence distribution statistic L is used to characterize the distribution degree of low-confidence text blocks in image units. In one embodiment, the local low-confidence distribution statistic L is the proportion of low-confidence blocks, that is, the proportion of text blocks with a confidence level lower than the text block confidence threshold Tb to the total number of text blocks; in other embodiments, it may also be the proportion of the number of characters of low-confidence text blocks, the proportion of text box area, the proportion of weight, spatial clustering, or the low-confidence proportion of key regions.

[0016] In one embodiment, a key region is determined based on text box coordinates, template area, field label, table area, seal area, or signature area. A different text block confidence threshold or local low confidence distribution threshold is applied to text blocks within the key region than to other regions. Triggering is determined when the local low confidence distribution statistic of the key region exceeds the region threshold.

[0017] In one embodiment, the quality determination employs short-circuit logic: first, the global confidence statistic G is compared with a first threshold T1; if G is less than T1, a trigger is determined. If no trigger is found, the local low-confidence distribution statistic L is compared with a second threshold T2; if L is greater than T2, a trigger is determined. In other words, a trigger is determined if either G is less than T1 or L is greater than T2. Figure 2 As shown, when the first recognition result does not contain a text block, it is not triggered based on the local low-confidence distribution statistic L. In other embodiments, the global confidence statistic G and the local low-confidence distribution statistic L are jointly input into a weighted scoring function or a learned classification function to obtain the quality judgment result.

[0018] The enable switch, first threshold T1, text block confidence threshold Tb and second threshold T2 in the engine switching strategy are obtained through a unified configuration reading interface and read at the entry point of each recognition process, so that runtime modifications take effect immediately in subsequent recognition without restarting or recompiling; the values ​​are rolled back according to the priority of configuration storage, program startup parameters or environment variables and preset default values.

[0019] The second optical character recognition engine is invoked only when a trigger is determined. The second optical character recognition engine can be an optical character recognition service based on a visual language model or a multimodal generation model, deployed in the cloud, privately, or locally. Its invocation configuration is determined by a model invocation strategy, which can be determined by a preset record in preset storage pointed to by the currently active preset identifier. This preset record includes at least three of the following: model identifier, prompt word, maximum output length, post-processing rules, and activation status. Alternatively, it can be determined by a model registry, a strategy engine, or document and image type routing rules. By modifying the currently active preset identifier, the visual language model or prompt word strategy used can be switched without modifying the program code. In one embodiment, image units are converted into image data unified resource identifiers or image encoded data, and sent along with the prompt word to the model corresponding to the model identifier via an AI gateway. The temperature parameter is set to zero, and the returned raw text is cleaned, extracted, or formatted according to post-processing rules to serve as the recognition result of the second optical character recognition engine.

[0020] When the second optical character recognition engine is successfully invoked, its recognition result is used as the final recognition result for the image unit, and the source is recorded as the path of the second optical character recognition engine. When the second optical character recognition engine call fails, times out, returns an empty result, returns a result that does not meet the requirements, or is rate-limited, the first recognition result is used as the final recognition result for the image unit, and the source is recorded as the path of the first optical character recognition engine or the first optical character recognition engine rollback path after the second optical character recognition engine fails, using structured metadata. The triggering or rollback reason can also be recorded, and the request identifier, response status, usage information, and time consumption information of the second optical character recognition engine call are associated and saved with the final recognition record, such as... Figure 2 As shown.

[0021] When the object to be identified includes multiple image units, the recognition tasks for each image unit are distributed and executed concurrently. The number of concurrently executed tasks is limited by a configurable concurrency limit N. The concurrency control can be implemented using semaphores, work queues, thread pools, coroutine pools, or task queues. Request throttling is only applied to calls that trigger the second optical character recognition engine. This request throttling can be implemented using at least one of the following: minimum interval between adjacent initiations (dt), token bucket, leaky bucket, sliding window rate limiting, fixed window rate limiting, or exponential backoff. No waiting time is set for the first triggered call, and subsequent triggered calls are throttled according to the minimum initiation interval (dt). Figure 3 As shown; the concurrency limit N and the throttling parameter are configurable during runtime.

[0022] The final recognition results of each image unit are merged into the document text according to their order in the object to be recognized. When the object to be recognized is a PDF document, the original text layer of the PDF document is extracted first, then image units are extracted from the PDF document, the above recognition and switching processing is performed on the extracted image units, and their final recognition results are merged with the original text layer to obtain the text recognition result of the PDF document, such as... Figure 4 As shown.

[0023] The system implementing the above method includes a first optical character recognition engine module, a confidence evaluation module, a configuration strategy module, a switching decision module, a second optical character recognition engine module, a fallback processing module, a concurrent scheduling module, and a result merging module. The functions of each module correspond to the steps of the aforementioned method, and their connections are as follows: Figure 1 As shown. The present invention can also be implemented as an electronic device including a processor and a memory, and as a computer-readable storage medium storing a computer program.

[0024] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-engine optical character recognition adaptive switching and fallback method based on confidence distribution, characterized in that, include: S1. Call the local or terminal-side first OCR engine to perform text recognition on the image unit in the object to be recognized, and obtain the first recognition result. The first recognition result includes the recognized text and multiple text blocks, each text block having a corresponding confidence level. S2. Calculate two complementary quality statistics based on the confidence levels of the multiple text blocks, including: a global confidence statistic characterizing the overall recognition quality of the image unit, and a local low-confidence distribution statistic characterizing the distribution degree of low-confidence text blocks in the image unit; S3. Read the engine switching strategy readable during runtime, the engine switching strategy including a switch to enable switching, a first threshold corresponding to the global confidence statistic, a text block confidence threshold, and a second threshold corresponding to the local low-confidence distribution statistic; S4. When the global confidence statistic is lower than the first threshold, or the local low-confidence distribution statistic is higher than the second threshold, a trigger is determined, and only when a trigger is determined, a second OCR engine based on a visual language model or a multimodal generation model is called to perform text recognition on the image unit; when a trigger is not determined, the image unit is determined according to the first recognition result. The final recognition result of the image unit; wherein, the second OCR engine is not executed on the image units that have not been triggered, and character-level fusion or voting is not performed between the first recognition result and the recognition result of the second OCR engine; S5, when the second OCR engine is successfully called, the final recognition result of the image unit is determined according to the recognition result of the second OCR engine; when the second OCR engine call fails, times out, or does not meet the acceptance conditions, the first recognition result is used as the final recognition result of the image unit, and the source of the final recognition result is recorded as the first OCR engine path or the second OCR engine path using structured metadata; S6, when the object to be recognized includes multiple image units, only the call that triggers the second OCR engine is subject to concurrency control and request throttling, and the final recognition results of multiple image units are merged according to the order of the image units in the object to be recognized.

2. The method according to claim 1, characterized in that, The global confidence statistic includes at least one of the following: average confidence level, weighted average confidence level, median confidence level, confidence level quantile, confidence level variance, and confidence level entropy.

3. The method according to claim 1, characterized in that, The local low-confidence distribution statistics include at least one of the following: the proportion of low-confidence text blocks, the proportion of character count, the proportion of text length, the proportion of text box area, the proportion of weight, spatial clustering, and the proportion of low-confidence key regions; the low-confidence text block is a text block with a confidence level lower than the text block confidence threshold.

4. The method according to claim 3, characterized in that, The key region is defined by text box coordinates, template region, field label, table region, seal region, or signature region; different text block confidence thresholds or local low confidence distribution thresholds are applied to text blocks within the key region compared to other regions.

5. The method according to claim 1, characterized in that, The determination trigger includes: inputting the global confidence statistic and the local low confidence distribution statistic into a quality determination function to obtain a quality determination result, wherein the quality determination function is a short-circuit logic determination function, a weighted scoring function, or a learned classification function.

6. The method according to claim 1, characterized in that, The second OCR engine is an OCR service based on a visual language model or a multimodal generation model, deployed in the cloud, privately, or locally.

7. The method according to claim 1, characterized in that, Calling the second OCR engine includes determining at least two of the following based on the model calling strategy: model identifier, prompt words, maximum output length, output format constraints, and post-processing rules; the model calling strategy is determined by the currently active preset identifier, the model registration center, the strategy engine, or document and image type routing rules.

8. The method according to claim 1, characterized in that, The concurrency control includes using semaphores, work queues, thread pools, coroutine pools, or task queues to limit the number of concurrent second OCR engine calls; the request throttling includes at least one of the following: minimum interval between adjacent initiations, token bucket, leaky bucket, sliding window rate limiting, fixed window rate limiting, or exponential backoff.

9. The method according to claim 2, characterized in that, The weights of the weighted average confidence score are determined based on the text length, number of characters, or area of ​​the text box.

10. The method according to claim 3, characterized in that, The local low-confidence distribution statistic is the proportion of the number of low-confidence text blocks to the total number of text blocks; the determination trigger is a short-circuit logic: first compare the global confidence statistic with the first threshold, and if no trigger is found, then compare the local low-confidence distribution statistic with the second threshold.

11. The method according to claim 1, characterized in that, The engine switching strategy is read at each identification and processing entry point, and changes made during runtime take effect immediately in subsequent identifications; and values ​​are selected in a priority hierarchy according to configuration storage, program startup parameters or environment variables, and preset default values.

12. The method according to claim 7, characterized in that, The currently active preset identifier points to a preset record in the preset storage. The preset record includes at least three of the following: model identifier, prompt word, maximum output length, post-processing rule, and enabled status. By modifying the currently active preset identifier, the model or prompt word strategy used can be switched without modifying the program code.

13. The method according to claim 1, characterized in that, It also includes recording the trigger or rollback reason, which includes at least one of the following: global confidence statistic is lower than the first threshold, local low confidence distribution statistic is higher than the second threshold, second OCR engine call failure, timeout or rate limiting; and the request identifier, response status, usage information, and time consumption information of the second OCR engine call are associated and saved with the final recognition result.

14. The method according to claim 8, characterized in that, The request throttling does not set a waiting time for the first triggered call, and throttling is performed on subsequent triggered calls according to the minimum initiation interval; the concurrency limit and the throttling parameter runtime are configurable.

15. The method according to claim 1, characterized in that, When the object to be identified is a PDF document, the original text layer of the PDF document is extracted first, then image units are extracted from the PDF document, and the steps S1 to S6 are performed on the extracted image units, and the final recognition result is merged with the original text layer.

16. A multi-engine optical character recognition adaptive switching and fallback system based on confidence distribution, characterized in that, include: The first OCR engine module is used to obtain a first recognition result containing multiple text blocks with confidence scores from local or edge OCR. The confidence evaluation module is used to calculate the global confidence statistic and the local low confidence distribution statistic; the configuration strategy module is used to read the runtime variable engine switching strategy; the switching decision module is used to determine the trigger when the global confidence statistic is lower than the first threshold or the local low confidence distribution statistic is higher than the second threshold, and only call the second OCR engine when triggered. The second OCR engine module provides OCR services based on visual language models or multimodal generation models. The fallback processing module is used to roll back the first recognition result and record the source with structured metadata when the second OCR engine call fails, times out, or does not meet the acceptance conditions; The concurrent scheduling module is used to perform concurrency control and request throttling only on the triggered second OCR engine calls; The result merging module is used to merge the final recognition results of multiple image units in sequence.

17. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the method of any one of claims 1 to 15.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 15.

Images