Dynamic fault-tolerant invoice duplicate checking system and method based on multi-modal OCR and hybrid indexing

Abstract

The application relates to the technical field of data processing, and discloses a dynamic fault-tolerant invoice duplicate checking system and method based on multi-modal OCR and mixed indexes. The original invoice image is subjected to invoice image quality evaluation, a corresponding OCR model is selected to perform invoice information recognition on the enhanced invoice image, index features are used to preliminarily screen candidate invoice data, and then the invoice data dynamic fault-tolerant duplicate checking is performed on the candidate invoice data by using duplicate checking features and the invoice image quality evaluation result, so that an invoice duplicate checking report is generated. Therefore, the application provides a dynamic fault-tolerant invoice duplicate checking scheme driven by the invoice image quality evaluation result. The multi-dimensional quality evaluation, differentiated OCR adaptation, dynamic fault-tolerant matching and manual auditing are combined to improve the repeated invoice recognition accuracy. Through the time-consuming and load negatively related design of the OCR model, the load correction of the character interval and the model adjustment driven by the manual workload, the system can stably operate under high and low loads.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a dynamic fault-tolerant invoice duplicate checking system and method based on multi-modal OCR and hybrid indexing. BACKGROUND

[0002] In enterprise financial management, invoices are the key vouchers for expense reimbursement and cost accounting. Repeatedly reimbursing invoices will lead to virtual increase of enterprise costs and distortion of profits, and seriously affect the accuracy of financial data. In addition, in government procurement, financial institution credit audit and other scenarios, the authenticity and uniqueness of invoices are also important basis for judging the legality of business. Invoice duplicate checking can effectively reduce business risks.

[0003] The existing invoice duplicate checking system has the following limitations: (1) The existing scheme mostly uses single OCR technology to process invoices images of different sources and different qualities. For low-quality invoice images such as blur, wrinkle and stain, the recognition accuracy is low, which leads to many errors in the extracted invoice information, affecting the reliability of the subsequent duplicate checking results; (2) The existing technology usually adopts a unified duplicate checking matching requirement without considering the quality difference of invoice images. For the error information extracted from low-quality images, if the conventional matching standard is still used, it is easy to miss or mischeck; and for high-quality images, too strict matching standard will increase unnecessary computing cost; (3) During the peak of invoice processing, a large number of invoices enter the duplicate checking system at the same time. The existing scheme does not dynamically adjust the duplicate checking strategy according to the system data processing load, which may lead to slow system response, processing delay, and even system crash and other problems.

[0004] Therefore, how to improve the duplicate checking accuracy of the invoice duplicate checking system during the peak of invoice processing, and balance the real-time and efficiency of duplicate checking, is a technical problem to be solved. SUMMARY

[0005] The present application provides a dynamic fault-tolerant invoice duplicate checking system and method based on multi-modal OCR and hybrid indexing, which aims to solve at least one of the above technical problems.

[0006] To achieve the above purpose, the present application provides a dynamic fault-tolerant invoice duplicate checking method based on multi-modal OCR and hybrid indexing, which comprises the following steps:

[0007] The received original invoice image is preprocessed and invoice image quality is evaluated to obtain an enhanced invoice image and invoice associated information; wherein the invoice associated information includes the invoice image number and invoice image quality evaluation result of each enhanced invoice image;

[0008] According to the invoice image quality evaluation result, a corresponding OCR model is selected to perform invoice information recognition on the enhanced invoice image, and invoice information text is generated and stored in an invoice information database;

[0009] Index features and duplicate checking features in the invoice information text are extracted, the index features are used to query candidate invoice data meeting the conditions in the invoice data record library, and the duplicate checking features are used to perform invoice data dynamic fault-tolerant duplicate checking in the candidate invoice data according to the invoice image quality evaluation result;

[0010] Based on the invoice data dynamic fault-tolerant duplicate checking result and the invoice image number, an invoice duplicate checking report is generated.

[0011] Optionally, the pre-processing and invoice image quality evaluation are performed on the received original invoice image to obtain the enhanced invoice image and the invoice association information, and the steps specifically include:

[0012] The original invoice images of different sources are received, a unique image identification number is assigned to each original invoice image as an invoice image code, and the invoice image and the invoice image code are stored in an image database;

[0013] The quality of each original invoice image in the image database is evaluated from four dimensions of clarity, contrast, noise level and integrity, a quality score of each original invoice image is generated, the fault-tolerant coefficient is converted from the quality score according to a preset relationship negatively correlated with the quality score, and the fault-tolerant coefficient and the quality score are taken as the invoice image quality evaluation result and the invoice image number to form the invoice association information;

[0014] According to the quality score in the invoice image quality evaluation result, a corresponding image enhancement algorithm is used to process the original invoice image to generate an enhanced invoice image.

[0015] Optionally, the quality of each original invoice image is evaluated from four dimensions of clarity, contrast, noise level and integrity, and a quality score of each original invoice image is generated, and the fault-tolerant coefficient is converted from the quality score according to a preset relationship negatively correlated with the quality score, and the fault-tolerant coefficient and the quality score are taken as the invoice image quality evaluation result and the invoice image number to form the invoice association information.

[0016] Based on the edge intensity mean value, the gray value dynamic range, the Gaussian noise estimation result and the invoice frame integrity, the clarity evaluation value, the contrast evaluation value, the noise level evaluation value and the integrity evaluation value of each original invoice image are measured;

[0017] The clarity evaluation value, the contrast evaluation value, the noise level evaluation value and the integrity evaluation value are normalized, and the quality score of each original invoice image is calculated by using the weighted summation method for the normalized result, and the fault-tolerant coefficient is converted from the quality score according to a preset relationship negatively correlated with the quality score;

[0018] wherein the expression of the quality score is specifically:

[0019] ;

[0020] wherein, denotes the quality score, , , , denote the normalized sharpness evaluation value, the contrast evaluation value, the noise degree evaluation value and the integrity evaluation value respectively, , , , denote the weight of the sharpness evaluation value, the contrast evaluation value, the noise degree evaluation value and the integrity evaluation value respectively;

[0021] wherein the expression of the fault tolerance coefficient is specifically:

[0022] ;

[0023] wherein, denotes the fault tolerance coefficient.

[0024] Optionally, for each original invoice image in the image database, the original invoice image is processed by using a corresponding image enhancement algorithm according to the quality score in the invoice image quality evaluation result to generate an enhanced invoice image, and the step specifically includes:

[0025] for the original invoice image with a quality score not less than the first preset value, the original invoice image is processed by using a gray scale normalization method for image enhancement;

[0026] for the original invoice image with a quality score less than the first preset value but higher than the second preset value, the original invoice image is processed by using a histogram equalization and Gaussian filtering method for image enhancement;

[0027] for the original invoice image with a quality score not higher than the second preset value, the original invoice image is processed by using a multi-scale Gaussian denoising, local contrast enhancement and guided filter sharpening method for image enhancement.

[0028] Optionally, according to the invoice image quality evaluation result, a corresponding OCR model is selected to recognize the enhanced invoice image for invoice information, generate invoice information text and store it in the invoice information database, and the step specifically includes:

[0029] According to the quality score in the invoice image quality evaluation result, assign a quality score interval level corresponding type OCR model to each enhanced invoice image according to the OCR model type library, and generate an invoice image processing list of each type of OCR model to store the assignment result of the enhanced invoice image;

[0030] Among them, the OCR model type library stores a plurality of different types of OCR models whose feature extraction time consumption and model running load are negatively correlated under different score interval levels; when assigning a corresponding type of OCR model to each enhanced invoice image, according to the quality score of the corresponding original invoice image of each enhanced invoice image, select the OCR model with the shortest feature extraction time consumption as the optimal OCR model in the corresponding score interval level and the enhanced invoice image corresponding to the score interval;

[0031] Call the OCR interface of the corresponding type of OCR model, send each enhanced invoice image in the invoice image processing list to the corresponding type of OCR model for invoice information recognition of the enhanced invoice image, extract a plurality of invoice key information to form an invoice information text, and store it in the invoice information database;

[0032] Among them, the invoice information text includes a combination of multiple invoice key information such as invoice code, invoice number, invoice date, invoice amount, seller information and buyer information.

[0033] Optionally, the index feature and the duplicate feature in the invoice information text are extracted, the index feature is used to query the candidate invoice data meeting the conditions in the invoice data record library, and the duplicate feature is used to perform the invoice data dynamic fault tolerance duplicate checking step in the candidate invoice data according to the invoice image quality evaluation result, which specifically includes:

[0034] The index feature and the duplicate feature in the invoice information text are extracted; wherein the index feature includes a combination of at least one feature in the invoice code, the invoice date, the invoice year, or the seller area in the seller information, and the duplicate feature includes a combination of at least one feature in the invoice number, the invoice date, the invoice amount, and the seller taxpayer identification number in the seller information;

[0035] The index feature is used to perform invoice data matching in the invoice data record library to determine whether there is candidate invoice data with complete feature matching, if so, generate a candidate invoice data set, if not, convert the corresponding invoice information text into invoice data and store it in the invoice data record library;

[0036] The duplicate feature and the fault tolerance coefficient in the invoice image quality evaluation result are used to perform dynamic fault tolerance duplicate checking in the candidate invoice data set considering the running load of the invoice duplicate checking server.

[0037] Optionally, by using the duplicate checking feature and the fault tolerance coefficient in the invoice image quality evaluation result, a dynamic fault tolerance duplicate checking step considering the operation load of the invoice duplicate checking server is performed in the candidate invoice data set, specifically including:

[0038] Real-time collection of hardware operation load parameters of the invoice duplicate checking server, and determination of the load state level of the invoice duplicate checking server based on the hardware operation load parameters;

[0039] According to the fault tolerance coefficient in the invoice image quality evaluation result, the character interval basic value when the candidate invoice data set and the duplicate checking feature use continuous character interval for target length feature character matching is determined, and the character interval basic value is corrected by using the load state level to generate a character interval actual value;

[0040] According to the character interval actual value, the target length character sampling comparison of the character interval actual value is performed on the feature string formed by each candidate invoice data in the candidate invoice data set and the duplicate checking feature;

[0041] When the sampling comparison is matched successfully, the candidate invoice data matched successfully and the invoice information text corresponding to the duplicate checking feature are output as a temporary invoice duplicate checking result to the artificial audit end, and the invoice duplicate checking result is output after the original invoice image is audited by the artificial audit. When the sampling comparison is not matched successfully, the corresponding invoice information text is converted into invoice data and stored in the invoice data record library.

[0042] Optionally, the step of determining the character interval basic value when the candidate invoice data set and the duplicate checking feature use continuous character interval for target length feature character matching according to the fault tolerance coefficient in the invoice image quality evaluation result, and correcting the character interval basic value by using the load state level to generate a character interval actual value, specifically includes:

[0043] According to the fault tolerance coefficient in the invoice image quality evaluation result, the character interval basic value when the candidate invoice data set and the duplicate checking feature use continuous character interval for target length feature character matching is determined in the relationship table of different fault tolerance coefficients and fault tolerance duplicate checking continuous character interval basic values;

[0044] The product of the character interval correction ratio corresponding to the load state level and the character interval basic value is used as the correction of the character interval basic value to generate the character interval actual value.

[0045] Optionally, the method further includes:

[0046] In each monitoring period, the unit time receiving quantity of the temporary invoice duplicate checking result of the artificial audit end is obtained, and the unit time receiving quantity is compared with the upper and lower limits of the artificial audit workload;

[0047] When the number of received per unit time exceeds the upper limit of the workload of manual review, the optimal OCR model of each score interval level is adjusted to an OCR model with higher feature extraction time consumption but lower model running load;

[0048] When the number of received per unit time exceeds the upper limit of the workload of manual review, the optimal OCR model of each score interval level is adjusted to an OCR model with lower feature extraction time consumption but higher model running load.

[0049] In addition, in order to achieve the above-mentioned purpose, the application also provides a dynamic fault-tolerant invoice duplicate checking system based on multi-modal OCR and hybrid index, comprising:

[0050] An evaluation module is configured to pre-process and evaluate the quality of the received original invoice images, to obtain enhanced invoice images and invoice association information; wherein the invoice association information comprises the invoice image number and the quality evaluation result of each enhanced invoice image;

[0051] An identification module is configured to select a corresponding OCR model to identify the invoice information of the enhanced invoice images according to the quality evaluation result of the invoice images, to generate invoice information text and store it in an invoice information database;

[0052] A duplicate checking module is configured to extract index features and duplicate checking features from the invoice information text, to query the candidate invoice data meeting the conditions in the invoice data record library by using the index features, and to perform dynamic fault-tolerant duplicate checking of the invoice data in the candidate invoice data by using the duplicate checking features and the quality evaluation result of the invoice images;

[0053] A generation module is configured to generate an invoice duplicate checking report based on the dynamic fault-tolerant duplicate checking result of the invoice data and the invoice image number.

[0054] The beneficial effects of this invention are as follows: It proposes a dynamic fault-tolerant invoice deduplication system and method based on multimodal OCR and hybrid indexing. By preprocessing and evaluating the quality of the received original invoice image, an enhanced invoice image and invoice association information containing the invoice image number and the invoice image quality evaluation result are obtained. Using the invoice image quality evaluation result, a corresponding OCR model is selected to recognize the invoice information in the enhanced invoice image and generate invoice information text. By extracting index features and deduplication features, the index features are used for preliminary screening of candidate invoice data. Then, the deduplication features and the invoice image quality evaluation result are used to perform dynamic fault-tolerant deduplication invoice data in the candidate invoice data. Finally, based on the dynamic fault-tolerant deduplication result and the invoice image number, an invoice deduplication report is generated. Therefore, this invention proposes a dynamic fault-tolerant invoice deduplication scheme based on multimodal OCR selection and hybrid indexing driven by invoice image quality assessment results. By combining multi-dimensional quality assessment, differentiated OCR adaptation, dynamic fault-tolerant matching with manual review, it can significantly improve the accuracy of duplicate invoice recognition during peak invoice processing periods. At the same time, through the design of OCR model time consumption and load negative correlation, character interval load correction, and model adjustment driven by manual workload, the system can run stably under both high and low loads, balancing the real-time performance and efficiency of deduplication detection. Attached Figure Description

[0055] Figure 1 This is a flowchart illustrating the dynamic fault-tolerant invoice deduplication method based on multimodal OCR and hybrid indexing according to an embodiment of the present invention.

[0056] Figure 2 This is a schematic diagram of the structure of the dynamic fault-tolerant invoice deduplication system based on multimodal OCR and hybrid indexing according to an embodiment of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0058] This invention provides a dynamic, fault-tolerant invoice deduplication method based on multimodal OCR and hybrid indexing, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the dynamic fault-tolerant invoice deduplication method based on multimodal OCR and hybrid indexing according to an embodiment of the present invention.

[0059] In this embodiment, a dynamic fault-tolerant invoice deduplication method based on multimodal OCR and hybrid indexing is described, the method comprising the following steps:

[0060] S1: pre-processing and invoice image quality assessment are performed on the received original invoice image to obtain an enhanced invoice image and invoice association information; wherein the invoice association information includes invoice image number and invoice image quality assessment result of each enhanced invoice image;

[0061] S2: according to the invoice image quality assessment result, a corresponding OCR model is selected to perform invoice information recognition on the enhanced invoice image, to generate invoice information text and store it in an invoice information database;

[0062] S3: index features and duplicate checking features in the invoice information text are extracted, the index features are used to query the candidate invoice data meeting the conditions in the invoice data record library, and the duplicate checking features and the invoice image quality assessment result are used to perform invoice data dynamic fault-tolerant duplicate checking in the candidate invoice data;

[0063] S4: based on the invoice data dynamic fault-tolerant duplicate checking result and the invoice image number, an invoice duplicate checking report is generated.

[0064] It should be noted that the existing invoice duplicate checking system has the following limitations: (1) the existing scheme mostly uses a single OCR technology to process invoice images of different sources and different qualities, and the recognition accuracy is low for low-quality invoice images such as blurred, wrinkled, and stained images, resulting in many errors in the extracted invoice information, which affects the reliability of the subsequent duplicate checking result; (2) the existing technology usually uses a unified duplicate checking matching requirement without considering the quality difference of the invoice images, and for the error information extracted from low-quality images, if the conventional matching standard is still used, it is easy to miss or mischeck; and for high-quality images, too strict matching standard will increase unnecessary computing cost; (3) during the peak of invoice processing, a large number of invoices enter the duplicate checking system at the same time, and the existing scheme does not dynamically adjust the duplicate checking strategy according to the system data processing load, which may cause slow system response, processing delay, and even system crash.

[0065] To solve the above problems, the embodiment performs invoice image quality assessment on the original invoice image, selects a corresponding OCR model to perform invoice information recognition on the enhanced invoice image, uses index features to preliminarily screen the candidate invoice data, and then uses duplicate checking features and invoice image quality assessment results to perform invoice data dynamic fault-tolerant duplicate checking in the candidate invoice data to generate an invoice duplicate checking report. Thus, the present application proposes a dynamic fault-tolerant invoice duplicate checking scheme driven by the invoice image quality assessment result, which improves the repeated invoice recognition accuracy by combining multi-dimensional quality assessment, differentiated OCR adaptation, dynamic fault-tolerant matching with manual review, and makes the system stable under high and low loads through time-consuming and load negatively related design of the OCR model, load correction of character spacing, and model adjustment driven by manual workload.

[0066] In the preferred embodiment, the preprocessing and invoice image quality assessment are performed on the received original invoice image, and the enhanced invoice image and invoice association information are obtained, specifically including:

[0067] S11: receiving original invoice images of different sources, assigning a unique image identification number to each original invoice image as an invoice image code, and storing the invoice image and the invoice image code in an image database;

[0068] S12: For each original invoice image in the image database, quality assessment is performed on the original invoice image from four dimensions of clarity, contrast, noise level and integrity, and a quality score of each original invoice image is generated. The quality score is converted into a fault tolerance coefficient according to a preset relationship negatively correlated with the quality score, and the fault tolerance coefficient and the quality score are used as the invoice image quality assessment result and the invoice image code to form the invoice association information;

[0069] S13: For each original invoice image in the image database, the original invoice image is processed by using a corresponding image enhancement algorithm according to the quality score in the invoice image quality assessment result, and an enhanced invoice image is generated.

[0070] In this embodiment, first, multiple-source original invoice images are received, a unique identification number is assigned to each image and stored in an image database, and traceability of the image is realized. Secondly, the image quality is evaluated from four dimensions of clarity (edge strength), contrast (gray scale dynamic range), noise level (Gaussian noise estimation), and integrity (frame detection), a quality score is generated and converted into a fault tolerance coefficient negatively correlated with the score (the worse the quality, the higher the fault tolerance requirement), and the association information of the image number, the quality score and the fault tolerance coefficient is formed. Finally, a differential image enhancement algorithm is matched according to the quality score, and an enhanced image is generated.

[0071] Thus, the present application realizes the whole life cycle traceability of the invoice image through the unique identification number, facilitates problem positioning, converts the subjective image quality into objective quantitative indicators by using multi-dimensional quality assessment and fault tolerance coefficient conversion, and provides data support for the dynamic adjustment of subsequent OCR selection and duplicate matching.

[0072] In the preferred embodiment, the quality of the original invoice image is evaluated from four dimensions of clarity, contrast, noise level and integrity, and a quality score of each original invoice image is generated. The quality score is converted into a fault tolerance coefficient according to a preset relationship negatively correlated with the quality score, specifically including:

[0073] S121: Based on the edge strength mean value, the gray scale dynamic range, the Gaussian noise estimation result and the invoice frame integrity, the clarity evaluation value, the contrast evaluation value, the noise level evaluation value and the integrity evaluation value of each original invoice image are measured respectively;

[0074] S122: normalize the clarity evaluation value, the contrast evaluation value, the noise degree evaluation value and the integrity evaluation value, and calculate the quality score of each original invoice image by using weighted summation for the normalized results, and convert the quality score into a fault tolerance coefficient according to a preset relationship in negative correlation with the quality score;

[0075] wherein the expression of the quality score is specifically:

[0076] ;

[0077] In the formula, denotes the quality score, , , , denote the normalized clarity evaluation value, the contrast evaluation value, the noise degree evaluation value and the integrity evaluation value respectively, , , , denote the weights of the clarity evaluation value, the contrast evaluation value, the noise degree evaluation value and the integrity evaluation value respectively;

[0078] Specifically, the clarity can be measured by calculating the edge intensity mean value of the image; the contrast can be determined by calculating the dynamic range of the image grayscale value; the noise degree can be evaluated by using the Gaussian noise estimation method; and the integrity can be judged by detecting whether the invoice frame and the key region (such as the invoice code and invoice number region) are complete.

[0079] wherein the expression of the fault tolerance coefficient is specifically:

[0080] ;

[0081] In the formula, denotes the fault tolerance coefficient.

[0082] In the embodiment, the four-dimensional original evaluation values (clarity using edge intensity mean value, contrast using grayscale dynamic range, noise degree using Gaussian noise estimation, and integrity using frame detection result) are obtained by professional algorithms, the original evaluation values are normalized to the interval [0, 1] to eliminate the dimensional differences of different dimensions; then the quality score is calculated by weighted summation, wherein the weights are set according to the influence degree of each dimension on the OCR recognition, and finally the fault tolerance coefficient is generated by negative correlation design, realizing the direct correlation of quality and fault tolerance demand, ensuring the objectivity of quality evaluation and the rationality of fault tolerance demand.

[0083] In a preferred embodiment, for each original invoice image in the image database, the original invoice image is processed according to the quality score in the invoice image quality evaluation result to generate an enhanced invoice image using a corresponding image enhancement algorithm, specifically including:

[0084] S131: For the quality score The original invoice image with a quality score not lower than a first preset value is subjected to image enhancement processing in a gray scale normalization manner;

[0085] S132: For the quality score The original invoice image with a quality score lower than the first preset value but higher than a second preset value is subjected to image enhancement processing in a histogram equalization and Gaussian filtering manner;

[0086] S133: For the quality score The original invoice image with a quality score not higher than the second preset value is subjected to image enhancement processing in a multi-scale Gaussian denoising, local contrast enhancement and guided filter sharpening manner.

[0087] In this embodiment, the quality score is divided into three intervals and matched with corresponding enhancement strategies: when the score is not lower than the first preset value (e.g. 0.7, high-quality image), only gray scale normalization processing is used to avoid over-processing and destroying original features; when the score is lower than the first preset value but higher than the second preset value (e.g. 0.3-0.7, medium-quality image), a combination strategy of "histogram equalization (to improve contrast) and Gaussian filtering (to remove noise)" is used to repair moderate defects; when the score is not higher than the second preset value (e.g. ≤0.3, low-quality image), a deep repair strategy of "multi-scale Gaussian denoising (Gaussian kernel distribution filtering based on different standard deviations), local contrast enhancement (to calculate the gray mean and variance of each block and perform contrast stretching on blocks where characters may exist), and guided filter sharpening (to control the sharpening intensity through the edge information of the guided image of multi-scale Gaussian denoising, e.g. to enhance contrast only in the character edge area with a gray gradient greater than 20)" is used to select the minimum necessary enhancement means according to the image defect degree.

[0088] Thus, this embodiment reduces the consumption of computing resources for the light processing of high-quality images while retaining original clear features; accurately repairs core defects such as insufficient contrast and a small amount of noise for the combined strategy of medium-quality images; and maximally restores character outlines and details for the deep repair of low-quality images, significantly improving the OCR adaptability of images, and achieving a balance between enhancement effect and resource consumption as a whole.

[0089] In a preferred embodiment, according to the invoice image quality evaluation result, the step of selecting a corresponding OCR model to perform invoice information recognition on the enhanced invoice image, generating invoice information text and storing it to the invoice information database, specifically includes:

[0090] S21: According to the quality score in the invoice image quality evaluation result, assign a quality score interval level corresponding type OCR model to each enhanced invoice image according to the OCR model type library, and store the invoice image processing list of each type of OCR model to store the assignment result of the enhanced invoice image;

[0091] Among them, the OCR model type library stores a plurality of OCR models of different types whose feature extraction time and model running load are negatively correlated under different score interval levels; when assigning a corresponding type of OCR model to each enhanced invoice image, according to the quality score of the corresponding original invoice image of each enhanced invoice image, select the OCR model with the shortest feature extraction time as the optimal OCR model in the corresponding score interval level and the enhanced invoice image corresponding to the score interval.

[0092] S22: Call the OCR interface of the corresponding type of OCR model, send each enhanced invoice image in the invoice image processing list to the corresponding type of OCR model for invoice information recognition of the enhanced invoice image, extract a plurality of invoice key information to form an invoice information text, and store it to the invoice information database.

[0093] Among them, the invoice information text includes a combination of multiple invoice key information such as invoice code, invoice number, invoice date, invoice amount, seller information and buyer information.

[0094] It should be noted that different types of OCR models include a general OCR model based on Tesseract Basic Edition applied to a high score interval (e.g. ≥0.7) (the principle is to extract the stroke width, contour and other collective features of the character), a character recognition model based on SVM applied to a medium score interval (e.g. 0.3-0.7) (the principle is to extract HOG features and LBP features of the character, and then use SVM classification), and a deep learning OCR model based on CNN-LSTM applied to a low score interval (e.g. ≤0.3) (the principle is to use CNN to extract spatial features of the image, such as local texture of the character; use LSTM to capture sequence dependency of the character, such as character order). And the OCR model whose feature extraction time and model running load are negatively correlated under each score interval refers to the difference between the lightweight model and the optimized model. The lightweight model has fewer parameters and lower computational complexity, and occupies less CPU during running. However, due to the weak feature extraction capability, it needs longer inference time to ensure accuracy, and the optimized model is the opposite.

[0095] In this embodiment, by constructing an OCR model type library, each quality score interval in the library corresponds to several OCR models with negative correlation between feature extraction time consumption and running load (such as a lightweight model with low load but long time consumption, and an optimized model with high load but short time consumption); for each enhanced image, according to the interval in which the quality score is located, the model with the shortest “feature extraction time consumption” in the interval is selected as the optimal model, and an image processing list is generated; finally, the interface of the optimal model is called to extract key information such as invoice code, number, and amount, to form an information text and store it in a database. The negative correlation design of the OCR model library provides flexible space for system load adjustment, the optimal selection standard with the shortest time consumption improves the processing efficiency of a single invoice, the standardized information extraction and storage provide structured data support for subsequent feature extraction and duplication checking, avoiding processing errors caused by unstructured text, balancing efficiency and resources on the premise of ensuring recognition accuracy.

[0096] In a preferred embodiment, index features and duplication checking features in the invoice information text are extracted, the index features are used to query the candidate invoice data meeting the conditions in the invoice data record library, and the duplication checking features and the invoice image quality evaluation results are used to perform the invoice data dynamic fault-tolerant duplication checking step in the candidate invoice data, specifically including:

[0097] S31: Extracting index features and duplication checking features in the invoice information text; wherein the index features include a combination of at least one feature of invoice code, invoice date, invoice year, or seller region in seller information, and the duplication checking features include a combination of at least one feature of invoice number, invoice date, invoice amount, and seller taxpayer identification number in seller information;

[0098] S32: Using the index features to perform invoice data matching in the invoice data record library, and determining whether there is candidate invoice data with complete feature matching, if yes, generating a candidate invoice data set, and if no, converting the corresponding invoice information text into invoice data and storing it in the invoice data record library;

[0099] S33: Using the duplication checking features and the fault-tolerant coefficient in the invoice image quality evaluation results to perform dynamic fault-tolerant duplication checking considering the running load of the invoice duplication checking server in the candidate invoice data set.

[0100] On this basis, the duplication checking features and the fault-tolerant coefficient in the invoice image quality evaluation results are used to perform the dynamic fault-tolerant duplication checking step considering the running load of the invoice duplication checking server in the candidate invoice data set, specifically including:

[0101] S331: Real-time acquisition of hardware running load parameters of the invoice duplication checking server, and determination of the load state level of the invoice duplication checking server based on the hardware running load parameters;

[0102] S332: According to the fault tolerance coefficient in the invoice image quality evaluation result, determine the character interval basic value when the duplicate checking feature and the candidate invoice data set use continuous character interval for target length feature character matching, correct the character interval basic value using the load state level, and generate the character interval actual value;

[0103] S333: According to the character interval actual value, sample and compare the target length characters of the character interval actual value for the feature string formed by the duplicate checking feature and each candidate invoice data in the candidate invoice data set;

[0104] S334: When the sample comparison is successful, output the candidate invoice data matched successfully and the invoice information text corresponding to the duplicate checking feature to the artificial audit end as a temporary invoice duplicate checking result, and output the invoice duplicate checking result after manually auditing the original invoice image. When the sample comparison is unsuccessful, convert the corresponding invoice information text into invoice data and store it in the invoice data record library.

[0105] In this embodiment, two types of features are first separated from the invoice information text: index features (invoice code, invoice year, seller region, etc., used for fast screening) and duplicate checking features (invoice number, amount, seller taxpayer identification number, etc., used for accurate matching), then the candidate invoices are matched in the invoice data record library through the index features, if there are, a candidate set is generated, if not, the current invoice is directly stored in the database, and finally the dynamic fault-tolerant duplicate checking considering server load is performed in the candidate set combined with the duplicate checking features and the fault tolerance coefficient in the quality evaluation result, the range is narrowed through the index features, and accurate duplicate checking is realized through dynamic matching.

[0106] Further, when performing dynamic fault-tolerant duplicate checking considering server load, first, real-time collect hardware parameters such as CPU usage and memory occupation of the server to determine the load state level (low / medium / high); second, according to the fault tolerance coefficient, query the fault tolerance coefficient and character interval basic value table to obtain the basic interval of continuous character matching (the higher the fault tolerance coefficient, the smaller the basic interval), then adjust the basic value with the correction proportion corresponding to the load level to obtain the actual interval (the higher the load level, the larger the interval, and the lower the load level, the smaller the interval); then sample and compare the feature string of the duplicate checking feature and the candidate invoice according to the actual interval, and finally if the matching is successful, it is pushed to artificial audit, and if it fails, the current invoice is stored in the database.

[0107] For example, the server collects the CPU usage rate of 85% (high load) in real time, the fault tolerance coefficient of a certain invoice is 0.1, and the character interval basic value is 3 obtained by querying the control table; the correction ratio corresponding to the high load is 1.2, the actual interval is 3*1.2=3.6 (rounded to 4), the candidate invoice data "00X23456" is compared with the duplicate feature "00123456" according to the interval 4, the core character is matched, and the candidate invoice data is pushed to the manual review, and finally it is determined that it is a duplicate invoice.

[0108] Therefore, the embodiment ensures that the duplicate checking strategy adapts to the system state by collecting the real-time load, avoids calculation overload when the load is high, and realizes the dynamic adjustment of the character interval by reducing the interval when the quality is poor (to avoid missing checks) and increasing the interval when the load is high (to balance efficiency). The sample comparison reduces the calculation amount of full character matching, and the manual review makes up for the limitations of machine matching, thereby improving the reliability of duplicate checking.

[0109] In the preferred embodiment, according to the fault tolerance coefficient in the invoice image quality evaluation result, the character interval basic value when the duplicate feature and the candidate invoice data set use continuous character interval for target length feature character matching is determined, the load state level is used to correct the character interval basic value, and the character interval actual value is generated. The step specifically includes:

[0110] S3321: According to the fault tolerance coefficient in the invoice image quality evaluation result, in the relationship control table of different fault tolerance coefficients and continuous character interval basic values, the character interval basic value when the duplicate feature and the candidate invoice data set use continuous character interval for target length feature character matching is queried;

[0111] S3322: The product of the character interval correction ratio corresponding to the load state level and the character interval basic value is used as the correction of the character interval basic value, and the character interval actual value is generated.

[0112] In the embodiment, first, a preset control table of fault tolerance coefficients and continuous character interval basic values is established (for example, the fault tolerance coefficient 0.8 corresponds to the basic value 1, and the fault tolerance coefficient 0.1 corresponds to the basic value 3), and the basic value is obtained by querying according to the fault tolerance coefficient of the current invoice; second, the corresponding character interval correction ratio is set for different load state levels (for example, the low load is 0.9, the medium load is 1.0, and the high load is 1.2); and finally, the final interval is calculated by the formula actual interval= basic value*correction ratio, which is used for sample comparison. The preset of the control table in the embodiment realizes the standardized mapping of the fault tolerance coefficient and the interval, avoids subjective setting, the load correction ratio associates the system state with the matching strictness, ensures that the calculation amount is reduced by expanding the interval when the load is high, and the precision is improved by reducing the interval when the load is low, and the quantitative calculation of the final interval makes the dynamic duplicate checking strategy practical and reproducible.

[0113] In a preferred embodiment, the method further comprises:

[0114] S335: In each monitoring period, the number of temporary invoice duplicate checking results received per unit time at the artificial audit end is obtained, and the number is compared with the upper and lower limits of the workload of artificial audit;

[0115] S336: When the number of temporary invoice duplicate checking results received per unit time exceeds the upper limit of the workload of artificial audit, the optimal OCR model of each scoring interval level is adjusted to an OCR model with higher feature extraction time consumption but lower model running load;

[0116] S337: When the number of temporary invoice duplicate checking results received per unit time exceeds the upper limit of the workload of artificial audit, the optimal OCR model of each scoring interval level is adjusted to an OCR model with lower feature extraction time consumption but higher model running load.

[0117] In this embodiment, the number of temporary duplicate checking results received per unit time at the artificial audit end is counted in a fixed monitoring period, and the number is compared with the preset upper and lower limits of the workload: if the upper limit is exceeded (the artificial is busy), the optimal OCR model of each quality interval is switched to a model with lower load but longer time consumption (to provide more system running hardware resources for dynamic fault-tolerant duplicate checking of the duplicate checking features, reduce the misjudgment rate, and indirectly reduce the artificial audit amount); if the lower limit is lower (the artificial is idle), the model with higher load but shorter time consumption is switched (to reduce the time consumption of OCR recognition, to increase the cost of artificial audit to obtain faster duplicate checking efficiency), the machine processing strategy is adjusted through the artificial load, to avoid the audit delay caused by the artificial overload or the resource waste caused by the artificial idleness; the generation speed of the temporary duplicate checking result is indirectly adjusted through the model switching, to maintain the artificial workload in a reasonable interval, and to improve the collaborative efficiency of the overall process.

[0118] Referring to Figure 2 , Figure 2 FIG. 1 is a structural schematic diagram of a dynamic fault-tolerant invoice duplicate checking system based on multi-modal OCR and hybrid index according to an embodiment of the present application.

[0119] As shown in Figure 2 , the dynamic fault-tolerant invoice duplicate checking system based on multi-modal OCR and hybrid index according to an embodiment of the present application comprises:

[0120] An evaluation module 10 is configured to perform preprocessing and invoice image quality evaluation on the received original invoice image, to obtain an enhanced invoice image and invoice associated information; wherein the invoice associated information comprises the invoice image number and the invoice image quality evaluation result of each enhanced invoice image;

[0121] An identification module 20 is configured to select a corresponding OCR model to recognize invoice information of the enhanced invoice image according to the invoice image quality evaluation result, generate invoice information text, and store the invoice information text into an invoice information database;

[0122] A duplicate checking module 30 is configured to extract index features and duplicate checking features from the invoice information text, query candidate invoice data meeting a condition in the invoice data record library by using the index features, and perform dynamic fault-tolerant duplicate checking of invoice data in the candidate invoice data by using the duplicate checking features and the invoice image quality evaluation result.

[0123] A generation module 40 is configured to generate an invoice duplicate checking report based on the dynamic fault-tolerant duplicate checking result of the invoice data and the invoice image number.

[0124] Other embodiments or specific implementations of the dynamic fault-tolerant invoice duplicate checking system based on multi-modal OCR and hybrid indexing can refer to the above method embodiments, which will not be described here.

[0125] It can be understood that, in the description of the specification, the description of the terms "an embodiment", "another embodiment", "other embodiments", or "first embodiment to Nth embodiment" means that the specific features, structures, materials or characteristics described in combination with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0126] It should be noted that, in this paper, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or system. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0127] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the specification and drawings of the present application, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A dynamic fault-tolerant invoice duplicate checking method based on multi-modal OCR and hybrid indexing, characterized in that, The method includes the following steps: The received original invoice images are preprocessed and their quality is assessed to obtain enhanced invoice images and invoice association information; wherein, the invoice association information includes the invoice image number and the invoice image quality assessment result for each enhanced invoice image; Based on the invoice image quality assessment results, the corresponding OCR model is selected to recognize invoice information in the enhanced invoice image, generate invoice information text, and store it in the invoice information database. Extract the index features and deduplication features from the invoice information text, use the index features to query candidate invoice data that meet the conditions in the invoice data record database, and use the deduplication features and the invoice image quality assessment results to perform dynamic fault-tolerant deduplication of invoice data in the candidate invoice data; The index features include the invoice code, the invoice year in the invoice date, or the seller region in the seller information; the deduplication features include the invoice number, invoice date, invoice amount, and the seller taxpayer identification number in the seller information. Based on the dynamic error-tolerant deduplication results of invoice data and the invoice image number, an invoice deduplication report is generated.

2. The multi-modal OCR and hybrid indexing based dynamic fault-tolerant invoice duplicate detection method as claimed in claim 1, wherein, The steps of preprocessing and evaluating the quality of the received raw invoice image to obtain an enhanced invoice image and invoice association information specifically include: Receive original invoice images from different sources, assign a unique image identifier to each original invoice image as an invoice image code, and store the invoice image and the invoice image code in the image database; For each original invoice image in the image database, the quality of the original invoice image is evaluated from four dimensions: sharpness, contrast, noise level and integrity. A quality score is generated for each original invoice image. The score is converted into a tolerance coefficient according to a preset relationship that is negatively correlated with the quality score. The tolerance coefficient and the quality score are used as the quality evaluation result of the invoice image and together with the invoice image number to form the invoice association information. For each original invoice image in the image database, the corresponding image enhancement algorithm is used to process the original invoice image based on the quality score in the invoice image quality assessment results, and an enhanced invoice image is generated.

3. The multi-modal OCR and hybrid indexing based dynamic fault-tolerant invoice duplicate detection method as claimed in claim 2, wherein, The quality of the original invoice images is evaluated from four dimensions: sharpness, contrast, noise level, and integrity. A quality score is generated for each original invoice image, and a tolerance coefficient is converted according to a preset relationship that is negatively correlated with the quality score. The specific steps include: Based on the mean edge intensity, dynamic range of gray values, Gaussian noise estimation results, and integrity of the invoice border, the sharpness evaluation value, contrast evaluation value, noise level evaluation value, and integrity evaluation value of each original invoice image are measured. The sharpness assessment value, contrast assessment value, noise level assessment value and integrity assessment value are normalized, and the quality score of each original invoice image is calculated by weighted summation of the normalization results. The score is then converted into a fault tolerance coefficient according to a preset relationship that is negatively correlated with the quality score. The expression for the quality score is specifically as follows: ； In the formula, denotes the quality score, , , , respectively denote the normalized sharpness evaluation value, the contrast evaluation value, the noise degree evaluation value and the integrity evaluation value, , , , respectively denote the weight of the sharpness evaluation value, the contrast evaluation value, the noise degree evaluation value and the integrity evaluation value. The expression for the fault tolerance coefficient is as follows: ； In the formula, represents a fault tolerance coefficient.

4. The multi-modal OCR and hybrid indexing based dynamic fault-tolerant invoice duplicate detection method as claimed in claim 3, wherein, For each original invoice image in the image database, the original invoice image is processed according to the quality score in the invoice image quality evaluation result to generate an enhanced invoice image using a corresponding image enhancement algorithm, specifically including: For quality score The original invoice image not less than the first preset value is processed by image enhancement in a gray scale normalization manner. For quality score If the original invoice image is lower than the first preset value but higher than the second preset value, the original invoice image is subjected to image enhancement processing in a manner of histogram equalization and Gaussian filtering. For quality score If the original invoice image is not higher than the second preset value, the original invoice image is subjected to image enhancement processing in a manner of multi-scale Gaussian denoising, local contrast enhancement and guided filter sharpening.

5. The multi-modal OCR and hybrid indexing based dynamic fault-tolerant invoice duplicate detection method as claimed in claim 1, wherein, According to the invoice image quality evaluation result, an OCR model corresponding to the quality score is selected to identify the enhanced invoice image, and the invoice information text is generated and stored in the invoice information database, specifically including: According to the quality score in the invoice image quality evaluation result, the enhanced invoice image is assigned to a quality score interval corresponding to the type of OCR model, and the invoice image processing list of each type of OCR model is generated to store the assignment result of the enhanced invoice image; Wherein, the OCR model type library stores a plurality of OCR models of different types under different score interval levels, and the feature extraction time and model running load are negatively correlated; when assigning a corresponding type of OCR model to each enhanced invoice image, according to the quality score of the corresponding original invoice image of each enhanced invoice image, the shortest feature extraction time is selected from the plurality of OCR models under the corresponding score interval level. The optimal OCR model of the score interval and the enhanced invoice image corresponding to it; Call the OCR interface of the corresponding type of OCR model, send each enhanced invoice image in the invoice image processing list to the corresponding type of OCR model for invoice information identification of the enhanced invoice image, extract the invoice information text composed of a plurality of invoice key information, and store it in the invoice information database; Wherein, the invoice information text includes a combination of multiple invoice key information such as invoice code, invoice number, invoice date, invoice amount, seller information and buyer information.

6. The multi-modal OCR and hybrid indexing based dynamic fault-tolerant invoice duplicate detection method as claimed in claim 5, wherein, Extract the index feature and the duplicate feature in the invoice information text, use the index feature to query the candidate invoice data that meets the conditions in the invoice data record library, and use the duplicate feature and the invoice image quality evaluation result to perform the invoice data dynamic fault tolerance duplicate checking step in the candidate invoice data, specifically including: Extract the index feature and the duplicate feature in the invoice information text; Use the index feature to perform invoice data matching in the invoice data record library, determine whether there is a candidate invoice data that completely matches the feature, if yes, generate a candidate invoice data set, if not, convert the corresponding invoice information text into invoice data and store it in the invoice data record library; Use the duplicate feature and the fault tolerance coefficient in the invoice image quality evaluation result to perform dynamic fault tolerance duplicate checking in the candidate invoice data set considering the running load of the invoice duplicate checking server.

7. The multi-modal OCR and hybrid indexing based dynamic fault-tolerant invoice duplicate detection method as claimed in claim 6, wherein, Use the duplicate feature and the fault tolerance coefficient in the invoice image quality evaluation result to perform the dynamic fault tolerance duplicate checking step in the candidate invoice data set considering the running load of the invoice duplicate checking server, specifically including: Real-time acquisition of hardware running load parameters of the invoice duplicate checking server, based on the hardware running load parameters, determine the load state level of the invoice duplicate checking server; According to the fault tolerance coefficient in the invoice image quality evaluation result, a character interval basic value when the duplicate checking feature and the candidate invoice data set use continuous character interval for target length feature character matching is determined, the character interval basic value is corrected by using the load state level, and a character interval actual value is generated; According to the character interval actual value, target length character sampling comparison of the character interval actual value is performed on a feature string formed by each candidate invoice data in the candidate invoice data set and the duplicate checking feature; When the sampling comparison is successful, the candidate invoice data matched successfully and invoice information text corresponding to the duplicate checking feature are output as a temporary invoice duplicate checking result to an artificial auditing end, and the invoice duplicate checking result is output after the original invoice image is audited by the artificial auditing end. When the sampling comparison is unsuccessful, the corresponding invoice information text is converted into invoice data and stored in an invoice data record library.

8. The multi-modal OCR and hybrid indexing based dynamic fault-tolerant invoice duplicate detection method as claimed in claim 7, wherein, According to the fault tolerance coefficient in the invoice image quality evaluation result, a character interval basic value when the duplicate checking feature and the candidate invoice data set use continuous character interval for target length feature character matching is determined, the character interval basic value is corrected by using the load state level, and a character interval actual value is generated According to the fault tolerance coefficient in the invoice image quality evaluation result, a character interval basic value when the duplicate checking feature and the candidate invoice data set use continuous character interval for target length feature character matching is determined, the character interval basic value is corrected by using the load state level, and a character interval actual value is generated The method further includes:

9. The multi-modal OCR and hybrid indexing based dynamic fault-tolerant invoice duplicate detection method as claimed in claim 7, wherein, In each monitoring period, a unit time receiving quantity of the temporary invoice duplicate checking result of the artificial auditing end is acquired, and the unit time receiving quantity is compared with an upper limit and a lower limit of an artificial auditing workload; When the unit time receiving quantity exceeds the upper limit of the artificial auditing workload, the optimal OCR model of each scoring interval level is adjusted to an OCR model that has higher feature extraction time consumption but lower model running load; When the unit time receiving quantity exceeds the upper limit of the artificial auditing workload, the optimal OCR model of each scoring interval level is adjusted to an OCR model that has lower feature extraction time consumption but higher model running load. It includes:

10. A dynamic fault-tolerant invoice deduplication system based on multi-modal OCR and hybrid indexing, characterized in that, An evaluation module is configured to perform preprocessing and invoice image quality evaluation on the received original invoice image, to obtain an enhanced invoice image and invoice associated information; wherein the invoice associated information includes an invoice image number and an invoice image quality evaluation result of each enhanced invoice image; An identification module is configured to select a corresponding OCR model to perform invoice information identification on the enhanced invoice image according to the invoice image quality evaluation result, to generate an invoice information text and store it in an invoice information database; A duplicate checking module is configured to extract an index feature and a duplicate checking feature from the invoice information text, to query a candidate invoice data meeting a condition in an invoice data record library by using the index feature, and to perform invoice data dynamic fault tolerance duplicate checking in the candidate invoice data by using the duplicate checking feature and the invoice image quality evaluation result. ​ The index features include invoice code, invoice year in invoice date, or seller region in seller information, and the duplicate checking features include invoice number, invoice date, invoice amount, and seller taxpayer identification number in seller information. The generating module is configured to generate an invoice duplicate checking report based on the invoice data, the dynamic fault-tolerant duplicate checking result, and the invoice image number.

Images