Bill image quality evaluation method and system based on multi-scale ROI weighted entropy

Abstract

The application discloses a kind of based on multi-scale ROI weighted entropy bill image quality evaluation method and system, belong to bill image quality evaluation technical field.The bill image quality evaluation method of the present application, comprising: obtaining bill image, and the basic quality inspection is carried out to the bill image, and the bill image after basic quality inspection is preprocessed, and target bill image is generated;The target bill image is segmented into structured area and unstructured area, and multi-scale ROI weighted entropy calculation is carried out to segmented area, and multi-scale ROI weighted entropy value is obtained;The dynamic threshold value generated in advance is compared with the multi-scale ROI weighted entropy value, and comparison result is generated, and the image quality of the target bill image is evaluated based on the comparison result.The implementation of the present application can realize the quantitative evaluation, efficient correction and accurate screening of financial and tax bill image test data.

Description

Technical Field

[0001] This invention relates to the field of invoice image quality assessment technology, and more specifically, to an invoice image quality assessment method and system based on multi-scale ROI weighted entropy. Background Technology

[0002] Current deep learning-based document image information extraction heavily relies on high-quality document image test datasets, but constructing existing test sets faces three major challenges:

[0003] 1. Data quality defects are hidden. Invoice images generally have problems such as blurry printing, glare interference, wrinkles and deformation. Traditional manual review relies on visual inspection, which is inefficient and subject to subjective bias.

[0004] 2. General image quality indicators (sharpness / brightness detection) cannot reflect the effectiveness of key areas of the invoice (such as invoice code and amount column), and do not take into account the differentiated quality requirements of structured (table) and unstructured (handwritten) areas.

[0005] 3. Low-quality test samples lead to distorted model test results and misjudgments of model performance. The lack of quantitative standards renders cross-model comparisons meaningless. Test data quality assessment is strongly coupled with the model's document information extraction task, necessitating the establishment of a dedicated quality control system for downstream tasks. Summary of the Invention

[0006] To address the above problems, this invention proposes a method for evaluating the quality of invoice images based on multi-scale ROI weighted entropy, comprising:

[0007] Acquire a ticket image, perform basic quality inspection on the ticket image, and preprocess the ticket image after basic quality inspection to generate a target ticket image;

[0008] The target ticket image is segmented into structured and unstructured regions, and multi-scale ROI weighted entropy is calculated for the segmented regions to obtain multi-scale ROI weighted entropy values.

[0009] The pre-generated dynamic threshold is compared with the multi-scale ROI weighted entropy value to generate a comparison result. Based on the comparison result, the image quality of the target ticket image is evaluated.

[0010] Optionally, perform basic quality checks on the ticket images, including:

[0011] Verify the resolution of the ticket images and discard ticket images with a resolution lower than 180 dpi;

[0012] Calculate the RGB mean of the ticket image and remove ticket images with an RGB mean ≥ 240 or an RGB mean ≤ 20;

[0013] Identify physical damage to the ticket image and discard ticket images with tears or missing corners greater than 5%.

[0014] Optionally, preprocessing may be performed on the ticket images after basic quality inspection, including:

[0015] The data source of the ticket image is determined, and based on the data source, the reflective area of ​​the ticket image is detected, and the reflective area of ​​the ticket image from different data sources is subjected to differentiated reflection elimination processing.

[0016] For the ticket images after anti-reflection processing, the ticket type is identified, and non-rigid geometric deformation correction is performed on the ticket images of different ticket types;

[0017] For the ticket image after non-rigid geometric deformation correction, the color deviation is quantified, and the color deviation of the ticket image is compensated based on the quantization result.

[0018] For the ticket image after color deviation compensation, the handwritten content in the ticket image is detected, and the handwritten content is subjected to stroke enhancement processing.

[0019] Optionally, before preprocessing the ticket images after basic quality inspection, the following steps are included:

[0020] Set the threshold for the dynamically adjusted parameters;

[0021] Within the threshold, the invoice images after basic quality inspection are preprocessed.

[0022] Optionally, the target ticket image is segmented into structured and unstructured regions, including:

[0023] The target ticket image is segmented into three categories using a lightweight U-Net network to obtain probability masks for the table region, printed text region, and handwritten content region.

[0024] Based on the probability mask, the percentage of handwritten pixels p_hand, the percentage of printed text p_print, and the percentage of table lines p_line within each ROI are calculated.

[0025] Based on the handwriting pixel ratio p_hand, the printed text ratio p_print, and the table line ratio p_line, the global handwriting ratio handwriting_ratio is calculated using the following formula:

[0026] handwriting_ratio = Total number of global handwriting pixels / Total number of pixels in the global valid information area;

[0027] Based on the global handwriting ratio handwriting_ratio, the structured and unstructured regions in the target bill image are segmented;

[0028] And refined processing and dynamic weight allocation are performed on the structured and unstructured regions.

[0029] Optionally, multi-scale ROI weighted entropy calculation is performed on the segmented regions to obtain the multi-scale ROI weighted entropy value, including:

[0030] Local entropy value calculation is performed on the ROI region of the target bill image;

[0031] Multi-modal entropy features are extracted from the segmented regions of the target bill image;

[0032] Based on the local entropy value obtained from the local entropy value calculation, the extracted multi-modal entropy features, and the weights of the segmented regions, hybrid entropy fusion calculation is performed to obtain the weighted entropy value of the multi-scale ROI;

[0033] Among them, based on the handwriting pixel ratio p_hand, the printed text ratio p_print, and the table line ratio p_line, the weights of the segmented regions are calculated respectively, including: the weights of the handwriting region, the printed text region, and the table region.

[0034] Optionally, the grading of image quality includes:

[0035] Pass, indicating inclusion in the available test set, criterion: Hfinal ≤ 0.9T_dynamic and w_base < 0.5 or Hgradient < 0.8T_dynamic;

[0036] Acceptable defect, indicating allowed to be used but marked criterion: 0.9T_dynamic < Hfinal ≤ 1.3T_dynamic and w_base ≤ 0.3;

[0037] Boundary case, indicating that review is required, criterion: 1.0T_dynamic < Hfinal ≤ 1.2T_dynamic;

[0038] Reject, indicating that it is not available for rejection processing criterion: Hfinal > 1.2T_dynamic for key area or non-key area > 1.5T_dynamic and the number of exceeded standard points ≥ 3 or w_base > 0.8 and ΔE > 12;

[0039] Among them, Hfinal is the multi-scale ROI weighted entropy value, and Tdynamic is the dynamic threshold.

[0040] Furthermore, this invention also proposes a ticket image quality assessment system based on multi-scale ROI weighted entropy, comprising:

[0041] The data preprocessing unit is used to acquire the ticket image, perform basic quality inspection on the ticket image, and preprocess the ticket image after basic quality inspection to generate the target ticket image.

[0042] The weighted calculation unit is used to segment the target ticket image into structured and unstructured regions, and to perform multi-scale ROI weighted entropy calculation on the segmented regions to obtain multi-scale ROI weighted entropy values.

[0043] The quality assessment unit is used to compare the pre-generated dynamic threshold with the multi-scale ROI weighted entropy value, generate a comparison result, and evaluate the image quality of the target ticket image based on the comparison result.

[0044] Optionally, perform basic quality checks on the ticket images, including:

[0045] Verify the resolution of the ticket images and discard ticket images with a resolution lower than 180 dpi;

[0046] Calculate the RGB mean of the ticket image and remove ticket images with an RGB mean ≥ 240 or an RGB mean ≤ 20;

[0047] Identify physical damage to the ticket image and discard ticket images with tears or missing corners greater than 5%.

[0048] Optionally, preprocessing may be performed on the ticket images after basic quality inspection, including:

[0049] The data source of the ticket image is determined, and based on the data source, the reflective area of ​​the ticket image is detected, and the reflective area of ​​the ticket image from different data sources is subjected to differentiated reflection elimination processing.

[0050] For the ticket images after anti-reflection processing, the ticket type is identified, and non-rigid geometric deformation correction is performed on the ticket images of different ticket types;

[0051] For the ticket image after non-rigid geometric deformation correction, the color deviation is quantified, and the color deviation of the ticket image is compensated based on the quantization result.

[0052] For the ticket image after color deviation compensation, the handwritten content in the ticket image is detected, and the handwritten content is subjected to stroke enhancement processing.

[0053] Optionally, before preprocessing the ticket images after basic quality inspection, the following steps are included:

[0054] Set the threshold for the dynamically adjusted parameters;

[0055] Within the threshold, the invoice images after basic quality inspection are preprocessed.

[0056] Optionally, the target ticket image is segmented into structured and unstructured regions, including:

[0057] The target ticket image is segmented into three categories using a lightweight U-Net network to obtain probability masks for the table region, printed text region, and handwritten content region.

[0058] Based on the probability mask, the percentage of handwritten pixels p_hand, the percentage of printed text p_print, and the percentage of table lines p_line within each ROI are calculated.

[0059] Based on the handwriting pixel ratio p_hand, the printed text ratio p_print, and the table line ratio p_line, the global handwriting ratio handwriting_ratio is calculated using the following formula:

[0060] handwriting_ratio = Total number of global handwriting pixels / Total number of pixels in the global valid information area;

[0061] Based on the global handwriting ratio, the structured and unstructured regions in the target document image are segmented.

[0062] The structured and unstructured regions are then subjected to refined processing and dynamic weight allocation.

[0063] Optionally, multi-scale ROI weighted entropy calculation is performed on the segmented region to obtain the multi-scale ROI weighted entropy value, including:

[0064] Calculate the local entropy value for the ROI region of the target ticket image;

[0065] Multimodal entropy features are extracted from the segmented regions of the target ticket image;

[0066] Based on the local entropy value calculated from the local entropy value, the extracted multimodal entropy features, and the weights of the segmented regions, a fusion calculation of the hybrid entropy is performed to obtain the weighted entropy value of the multi-scale ROI;

[0067] Specifically, the weights of the segmented regions are calculated based on the proportion of handwritten pixels (p_hand), the proportion of printed text (p_print), and the proportion of table lines (p_line), including the weights of the handwritten region, the printed text region, and the table region.

[0068] Optionally, the grading of image quality includes:

[0069] Pass, indicating inclusion in the available test set, with the criterion: Hfinal ≤ 0.9T_dynamic and w_base < 0.5 or Hgradient < 0.8T_dynamic;

[0070] Acceptable defect, indicating that it is allowed to be used but the marking criterion is: 0.9T_dynamic < Hfinal ≤ 1.3T_dynamic and w_base ≤ 0.3;

[0071] Boundary case, indicating that a review is required, with the criterion: 1.0T_dynamic < Hfinal ≤ 1.2T_dynamic;

[0072] Rejection, indicating that it is not available for rejection processing, with the criterion: Hfinal > 1.2T_dynamic in the key area or non - key area > 1.5T_dynamic and the number of exceeded standard points ≥ 3 or w_base > 0.8 and ΔE > 12;

[0073] Among them, Hfinal is the multi - scale ROI weighted entropy value, and Tdynamic is the dynamic threshold.

[0074] On the other hand, the present invention also provides a computing device, including: one or more processors; <s

[0075] The processor is used to execute one or more programs;

[0076] When the one or more programs are executed by the one or more processors, the method as described above is implemented.

[0077] On the other hand, the present invention also provides a computer - readable storage medium, on which a computer program is stored. When the computer program is executed, the method as described above is implemented.

[0078] Compared with the prior art, the beneficial effects of the present invention are: <o

[0079] The present invention provides a method for evaluating the quality of bill images based on multi - scale ROI weighted entropy, including: obtaining a bill image, performing basic quality inspection on the bill image, and pre - processing the bill image after basic quality inspection to generate a target bill image; segmenting the target bill image into structured areas and unstructured areas, calculating the multi - scale ROI weighted entropy for the segmented areas to obtain the multi - scale ROI weighted entropy value; comparing the pre - generated dynamic threshold with the multi - scale ROI weighted entropy value to generate a comparison result, and based on the comparison result, evaluating the image quality of the target bill image. The implementation of the present invention can achieve quantitative evaluation, efficient correction and accurate screening of the test data of fiscal and tax bill images. Attached Figure Description

[0080] Figure 1 This is a flowchart of the method of the present invention;

[0081] Figure 2 This is a flowchart of an embodiment of the method of the present invention;

[0082] Figure 3 This is a structural diagram of the system of the present invention. Detailed Implementation

[0083] Exemplary embodiments of the invention will now be described with reference to the accompanying drawings. However, the invention may be embodied in many different forms and is not limited to the embodiments described herein. These embodiments are provided to fully and completely disclose the invention and to fully convey its scope to those skilled in the art. The terminology used in the exemplary embodiments illustrated in the drawings is not intended to limit the invention. In the drawings, the same units / elements are referred to by the same reference numerals.

[0084] Unless otherwise stated, the terms used herein (including technical terms) have their common meaning as understood by one of ordinary skill in the art. Furthermore, it is understood that terms defined in commonly used dictionaries should be understood to have a meaning consistent with the context of their relevant field, and not to be interpreted as having an idealized or overly formal meaning.

[0085] Example 1:

[0086] This invention proposes a document image quality assessment method S100 based on multi-scale ROI weighted entropy, such as... Figure 1 As shown, it includes:

[0087] S101, acquire the ticket image, perform basic quality inspection on the ticket image, and preprocess the ticket image after basic quality inspection to generate the target ticket image;

[0088] S102, the target ticket image is segmented into structured and unstructured regions, and the segmented regions are subjected to multi-scale ROI weighted entropy calculation to obtain multi-scale ROI weighted entropy values.

[0089] S103, compare the pre-generated dynamic threshold with the multi-scale ROI weighted entropy value to generate a comparison result, and evaluate the image quality of the target ticket image based on the comparison result.

[0090] The basic quality inspection of the ticket images includes:

[0091] Verify the resolution of the ticket images and discard ticket images with a resolution lower than 180 dpi;

[0092] Calculate the RGB mean of the ticket image and remove ticket images with an RGB mean ≥ 240 or an RGB mean ≤ 20;

[0093] Identify physical damage to the ticket image and discard ticket images with tears or missing corners greater than 5%.

[0094] The preprocessing of the invoice images after basic quality inspection includes:

[0095] The data source of the ticket image is determined, and based on the data source, the reflective area of ​​the ticket image is detected, and the reflective area of ​​the ticket image from different data sources is subjected to differentiated reflection elimination processing.

[0096] For the ticket images after anti-reflection processing, the ticket type is identified, and non-rigid geometric deformation correction is performed on the ticket images of different ticket types;

[0097] For the ticket image after non-rigid geometric deformation correction, the color deviation is quantified, and the color deviation of the ticket image is compensated based on the quantization result.

[0098] For the ticket image after color deviation compensation, the handwritten content in the ticket image is detected, and the handwritten content is subjected to stroke enhancement processing.

[0099] The preprocessing of the invoice images after basic quality inspection includes:

[0100] Set the threshold for the dynamically adjusted parameters;

[0101] Within the threshold, the invoice images after basic quality inspection are preprocessed.

[0102] The segmentation of the target ticket image into structured and unstructured regions includes:

[0103] The target ticket image is segmented into three categories using a lightweight U-Net network to obtain probability masks for the table region, printed text region, and handwritten content region.

[0104] Based on the probability mask, the percentage of handwritten pixels p_hand, the percentage of printed text p_print, and the percentage of table lines p_line within each ROI are calculated.

[0105] Based on the handwriting pixel ratio p_hand, the printed text ratio p_print, and the table line ratio p_line, the global handwriting ratio handwriting_ratio is calculated using the following formula:

[0106] handwriting_ratio = Total number of global handwritten pixels / Total number of pixels in the global effective information area;

[0107] Based on the global handwriting ratio handwriting_ratio, the structured area and the unstructured area in the target bill image are segmented;

[0108] And refined processing and dynamic weight assignment are performed on the structured area and the unstructured area.

[0109] Among them, calculating the multi-scale ROI weighted entropy of the segmented area to obtain the multi-scale ROI weighted entropy value includes:

[0110] Calculating the local entropy value for the ROI area of the target bill image;

[0111] Extracting multi-modal entropy features for the segmented area of the target bill image;

[0112] Based on the local entropy value calculated from the local entropy value, the extracted multi-modal entropy features, and the weight of the segmented area, a hybrid entropy fusion calculation is performed to obtain the weighted entropy value of the multi-scale ROI;

[0113] Among them, based on the handwritten pixel ratio p_hand, the printed text ratio p_print, and the table line ratio p_line, the weights of the segmented areas are calculated respectively, including: the weights of the handwritten area, the printed text area, and the table area.

[0114] Among them, the classification of image quality includes:

[0115] Pass, indicating inclusion in the available test set, criterion: Hfinal ≤ 0.9T_dynamic and w_base < 0.5 or Hgradient < 0.8T_dynamic;

[0116] Acceptable defect, indicating allowed to be used but marked criterion: 0.9T_dynamic < Hfinal ≤ 1.3T_dynamic and w_base ≤ 0.3;

[0117] Boundary case, indicating that a review is required, criterion: 1.0T_dynamic < Hfinal ≤ 1.2T_dynamic;

[0118] Rejection, indicating that it is not available for rejection processing, criterion: Hfinal > 1.2T_dynamic for the key area or non-key area > 1.5T_dynamic and the number of exceeded standard points ≥ 3 or w_base > 0.8 and ΔE > 12;

[0119] Where Hfinal is the multi-scale ROI weighted entropy value, and Tdynamic is the dynamic threshold.

[0120] The invention will be further explained below with reference to specific implementation examples:

[0121] In the specific implementation process, firstly, basic quality inspection is carried out on the ticket images, and severely damaged or invalid input samples are quickly eliminated through resolution screening, sensor saturation detection and physical damage analysis.

[0122] Subsequently, multi-stage adaptive collaborative optimization is performed on the images that pass the initial inspection: polarization compensation is used to dynamically adjust the intensity of reflection elimination, and non-rigid geometric correction based on image size is combined to achieve wrinkle repair. Simultaneously, color shift dynamic tolerance correction and stroke enhancement processing based on classification confidence are implemented to output standardized high-quality images.

[0123] Furthermore, an improved U-Net network and Hough transform collaborative algorithm are adopted to achieve accurate segmentation and differentiated processing of table regions and handwritten content; a region-adaptive multi-scale ROI weighted entropy model is designed, and the texture complexity of different regions is accurately quantified through dynamic window adjustment and hybrid entropy fusion strategy.

[0124] Finally, the detection threshold is automatically adapted based on the ticket type and regional weight; a hierarchical quality judgment benchmark is constructed to automatically remove samples with irreparable defects, ensuring high reliability of test data.

[0125] Its specific implementation process, such as Figure 2 As shown, it includes:

[0126] Step 1. Preliminary basic quality inspection, including:

[0127] Before entering the formal image preprocessing process, an initial screening is performed to quickly detect and intercept irreparable input images, improving overall processing efficiency. First, image resolution is checked; images below 180 dpi are directly marked and discarded. Next, sensor saturation is checked; images with an overall RGB mean ≥240 or ≤20 are marked and discarded. Physical damage is identified through edge analysis; images with tears or missing corners exceeding 5% are marked and discarded. This step is strictly limited to objectively quantifiable hardware defects, quickly identifying substandard images and ensuring that subsequent fine-tuning processes only accept images with basic quality.

[0128] Step 2. Joint preprocessing of the invoice images, including:

[0129] For images that pass the initial screening, multi-stage adaptive collaborative optimization is performed to address the unique issues of reflection, wrinkles, color cast, and low contrast in invoice images, thereby achieving standardized image output.

[0130] (1) Reflection detection and elimination:

[0131] Image Source-Based Reflective Area Detection and Removal: Automated source identification of input images is performed, achieving accurate classification by examining image metadata features: when a scanner manufacturer's EXIF ​​label (e.g., "EPSON Scan") is detected, it is identified as an industrial scan image; if a mobile phone brand mark (e.g., "iPhone Camera") is present and light source direction data is included, it is identified as a mobile-captured image. For cases without valid EXIF ​​data, pixel feature analysis is used for auxiliary judgment: images with a resolution exceeding 300 DPI and a uniform background are classified as scanned images, while those with lens distortion or a dynamic range greater than 10 EV are classified as captured images. The EXIF ​​light source direction is recorded as EXIF_l ight. Differentiated processing is applied to images from different sources: industrial scanned images are processed by calculating the saturation channel variance (threshold σ). 2 <15) Detect reflective areas and repair them using polarization compensation (θ = 15°, λ = 0.7); for images captured by mobile devices, combine EXIF ​​light source direction data and highlight region clustering (k-means) to jointly locate reflective areas, and apply an improved inpainting algorithm in conjunction with an edge protection strategy for repair. When the confidence level of the image source determination is below 80%, the two detection algorithms are executed in parallel, and the intersection area of ​​the results is used.

[0132] (2) Non-rigid geometric deformation correction:

[0133] For the image after reflection removal, layout matching is performed first. Similar templates are retrieved from the preset template library, and sub-pixel matching (error < 0.5 pixels) is achieved based on SIFT feature points to identify the ticket type (denoted as p_type). Then, deformation correction is performed. Control point pairs are established through thin plate spline transformation (TPS) to minimize the bending energy function and obtain a standard layout image that is geometrically aligned with the template.

[0134] (3) Color deviation quantification and compensation:

[0135] For the geometrically corrected image, reference colors are extracted, and the reference colors are dynamically selected for different document types: for VAT invoices, the red stamp (HSV: H∈[0,15]∪[160,180], S>50) is the core correction target; for train tickets and other documents with blue backgrounds, the blue area needs to be jointly detected (Lab: b∈[-20,-5]); for grayscale or black-and-white documents (such as small receipts), only brightness equalization is performed. Color standardization is ensured by using the CIE2000 color difference formula (ΔE<3) and 3D LUT nonlinear mapping, while skipping irrelevant channel processing to improve efficiency.

[0136] (4) Stroke enhancement:

[0137] For the color-corrected image, a lightweight MobiNetV3-0.5x classifier is first used for handwriting detection. After detecting handwriting, a 1×3 horizontal bar kernel is used for morphological dilation to enhance continuous strokes for horizontal writing, while a 3×1 vertical bar kernel is used for Top-Hat transformation to highlight vertical stroke features for vertical writing. Subsequently, ink restoration processing is performed. For faded text on thermal paper, a simulated image in the 365nm ultraviolet band is fused with the original image with a weight of 0.6. The oxidation features of the ultraviolet channel are used to enhance residual strokes. At the same time, the CLAHE algorithm is used to enhance the contrast of ordinary faded text. The final output is an enhanced image with clear strokes and intact directional features.

[0138] (5) Parameter dynamic adjustment strategy:

[0139] The system is initialized using preset parameters (e.g., λ = 0.7, control point density 20 pts / cm). 2 ΔE<3), and then dynamic optimization is triggered by real-time analysis of input image features. Reflection elimination parameters: Based on the actual detected polarization reflection intensity, the compensation coefficient is linearly adjusted within the range of λ=0.5~0.9 to eliminate reflection. Geometric correction parameters: The control point density is automatically adapted according to the document size (A4 paper 20pts / cm²). 2 →Receipt 30pts / cm 2 If the matching error continuously exceeds the threshold, the tolerance is relaxed to 1.5px. Color shift correction parameter: When ΔE is continuously >5, the standard is gradually relaxed to ΔE <8 and local contrast is enhanced. Stroke enhancement parameter: Based on the classification confidence of MobileNetV3, the threshold is dynamically adjusted (0.75~0.9), and ultraviolet fusion and CLAHE hybrid processing are superimposed on low confidence areas.

[0140] This step automatically adapts to input images of different qualities through dynamic parameter adjustment, and outputs a standardized image.

[0141] Step 3. Structured / unstructured region segmentation, including:

[0142] To address the mixed characteristics of fixed-format content (such as table lines, printed text boxes, and fixed fields such as "invoice code" and "amount") and free-form content (such as handwriting / signature) in invoice images, a hybrid segmentation strategy combining deep learning and traditional algorithms is adopted.

[0143] A lightweight U-Net network (1.8M parameters) is used to perform tri-class semantic segmentation on the input image, obtaining probability masks for table regions, printed text, and handwritten regions. For the probability masks output by U-Net, the percentage of handwritten pixels (p_hand), printed text (p_print), and table lines (p_line) within each ROI are calculated (e.g., if handwritten pixels account for 60% of an ROI, then p_hand = 0.6). The global handwriting ratio is then calculated: handwriting_ratio = total number of handwritten pixels in the entire image / total number of pixels in the entire image's effective information region.

[0144] Refined processing of structured regions: For table regions, non-table line segments are filtered out by Hough transform with angular constraints (0° / 90°±5° tolerance) to ensure the continuity of table lines.

[0145] Refined processing of unstructured areas: For handwritten content, the Stroke Width Transform (SWT) algorithm is applied to merge broken strokes, preserving the writing details completely.

[0146] Implement a dynamic weighting strategy: Based on business needs, differentiated weighting can be applied to different regions to determine the criticality of invoice information extraction (recorded as w_base). For example, key identification regions (such as invoice numbers and codes) can be assigned a high weight of 0.8 to 1.0, as they directly impact core data extraction; table structure regions can be assigned a medium weight of 0.2 to 0.8 to balance their structured characteristics and secondary importance; while background elements can be assigned a low weight of ≤0.2. Through this dynamic adjustment mechanism, considering regional confidence levels, area proportions, and invoice type characteristics, the weighting allocation ensures both business logic and adaptability, achieving optimal quality assessment indicators.

[0147] This step uses a hybrid segmentation strategy to divide the ticket image into a fixed-format region (table) and a free-form region (handwriting), providing a foundation for subsequent differential entropy calculation.

[0148] Step 4. Calculate the weighted entropy of the multi-scale ROI, including:

[0149] Based on the previous region segmentation results, differentiated texture entropy calculations are performed on different regions of the ticket.

[0150] (1) Calculation of local entropy:

[0151] The gray-level co-occurrence matrix entropy is calculated within the ROI region using the sliding window method:

[0152] Here, Wxy represents a dynamic window centered at pixel (x,y), and p(i|Wxy) is the probability of grayscale value i appearing within the window. Texture entropy quantifies the "complexity of pixel brightness variations," directly reflecting the region's quality: an abnormally high entropy value indicates blurriness / noise, while an abnormally low value indicates missing information.

[0153] Implement a dynamic windowing strategy: Adjust the detection window size dynamically based on the ROI weight level. Higher weights (stronger business importance) result in smaller window sizes and higher detection accuracy. For example, a 3×3 micro-scale window is used for critical areas such as invoice codes (weight > 0.8), achieving a precise detection of 0.1mm. 2 Printing defects; weighted areas such as the amount column (0.2 < weight ≤ 0.8) use a 7×7 window to balance sensitivity; background areas (weight ≤ 0.2) are quickly filtered using a 15×15 window to improve calculation efficiency.

[0154] Calculate local statistics for each sliding window:

[0155] μ_local=mean(Hlocal),σ_local=std(Hlocal)

[0156] Implement standardization: H'_local = (Hlocal - μ_local) / σ_local

[0157] Truncation: Values ​​exceeding ±3σ are forcibly set as boundary values.

[0158] (2) Multimodal entropy feature extraction:

[0159] For the framed area of ​​the table, the image gradient is extracted using the 4-direction Sobel operator and the uncertainty of its direction distribution is calculated to obtain the gradient entropy Hgradient.

[0160] For handwritten text regions, a 3-layer Haar wavelet decomposition is first performed, and the wavelet entropy Hwavelet is calculated based on the decomposition results.

[0161] Entropy normalization:

[0162] H'_gradient = Hgradient / (2 * log2(4)) # Maps to [0,1]

[0163] H'_wavelet = Hwavelet / (3 * log2(8)) # Maps to [0,1]

[0164] (3) Hybrid entropy fusion calculation:

[0165] For key areas (e.g., window sizes ranging from 3x3 to 7x7, adjusted according to weight), the mixed entropy value is dynamically calculated based on the proportion of table lines, printed text, and handwritten text.

[0166] Hfinal=w_gradient*H'_gradient+w_local*H'_local

[0167] +w_wavelet*H'_wavelet,

[0168] For non-critical areas (e.g., window size 15×15), use H'_local directly.

[0169] Hfinal = H'_local.

[0170] Wherein, the table line weight is: w_gradient = p_line * w_base_line / (p_print * w_base_print + p_hand * w_base_hand + p_line * w_base_line);

[0171] Printed text weight w_local = p_print * w_base_print / (p_print * w_base_print + p_hand * w_base_hand + p_line * w_base_line);

[0172] Handwritten weight: w_wavelet=p_hand*w_base_hand / (p_print*w_base_print+p_hand*w_base_hand+p_l ine*w_base_l ine);

[0173] w_base_line, w_base_print, and w_base_hand are the table lines, printed text, and handwritten content recorded in the third step. The w_base values ​​are recorded according to the importance of the area and can be dynamically adjusted based on the type of document. For example:

[0174]

[0175] Hfinal, as the final quality indicator, achieves targeted quality assessment of different regional characteristics through differentiated feature extraction and dynamic weight allocation.

[0176] Step 5. Adaptive Dynamic Threshold Generation

[0177] The threshold Tdynamic of Hfinal is calculated based on the dynamically loaded threshold parameters (T0,k,σ) to determine whether Hfinal exceeds the standard, and thus whether the image quality is acceptable.

[0178] Load preset parameters:

[0179] Based on the ticket type p_type identified in step 2, the system dynamically loads preset threshold parameters: for example:

[0180]

[0181] Calculate dynamic thresholds based on parameter adjustment:

[0182] Tdynamic=max(T0-k·σ,0.5T0),

[0183] Key region: Tdynamic = max(T0 - (k + α·w_base)·σ, 0.4T0),

[0184] Non-critical region: Tdynamic = max(T0 - k·σ, 0.6T0),

[0185] Where α is the weight sensitivity coefficient (0.5 is recommended by default), and w_base is the region weight (0 to 1.0) recorded in step 3.

[0186] Tdynamic is dynamically adjusted according to different regions (critical and non-critical regions), while different lower limits are set (0.4T0 for critical regions and 0.6T0 for non-critical regions) to ensure that the dynamic time is within a reasonable range.

[0187] Overall threshold adaptability is achieved through T0 incremental optimization, k-scenario adaptation, and α and σ condition calculation. A hierarchical control mechanism is implemented for the three core parameters.

[0188] The baseline threshold T0 is calibrated using the Hfinal entropy distribution of qualified samples; for example, the 99th percentile T0 = 0.85 is used for the VAT invoice code area. Incremental optimization is implemented: when the model's accuracy in extracting data from three consecutive batches of test data exceeds 95%, the threshold is gradually tightened by T0 × 0.98 to continuously improve quality standards, while a lower limit constraint of 0.5T0 prevents over-adjustment.

[0189] The sensitivity coefficient k adopts a dynamic strategy driven by the testing phase: k = 2.5 (extremely stringent screening) in the smoke test phase, k = 1.8 (balancing quality and diversity) in the model evaluation phase, and k = 3.0 (forced exposure of defects) in the adversarial test phase. When the model accuracy fluctuates abnormally, the system automatically fine-tunes the k value in ±0.2 steps to ensure that the detection sensitivity matches the requirements of the scenario.

[0190] The noise standard deviation σ is calculated using historical qualified samples (excluding interference from production data) and employs a conditional statistical method: only entropy values ​​lower than the current T0 are collected to calculate the segmented standard deviation, avoiding contamination of noise estimation by defective samples. When the testing equipment or environment changes, the σ calculation window is automatically reset (by default, the most recent 1000 sets of data are retained) to maintain the cross-cycle stability of quality judgment.

[0191] Through triple dynamic control, a balance between accuracy and robustness in threshold adaptation is achieved. The T0 baseline can be recalibrated quarterly using newly added qualified samples, and an automated mapping table between k-values ​​and testing phases can be established.

[0192] Step 6. Final quality judgment, including:

[0193] A four-level adjudication mechanism is implemented based on the comparison of multi-scale ROI weighted entropy values ​​(Hfinal) and dynamic thresholds (Tdynamic):

[0194]

[0195]

[0196] This invention achieves efficient quality inspection of invoice images through multi-stage collaboration: preliminary screening intercepts hardware-level defects; preprocessing pipeline repairs reversible defects; hybrid region segmentation and multi-scale entropy assessment accurately locate local quality problems; dynamic threshold adjudication balances business needs and technical requirements; and through hierarchical quality judgment, while ensuring the quality of core data, it retains usable images to the maximum extent, achieving the intelligent quality inspection goal of "strictly controlling critical data and tolerating non-critical data".

[0197] This invention enables quantitative evaluation, efficient correction, and accurate screening of tax and financial invoice image test data, specifically as follows:

[0198] Precise quantitative assessment: A triple mechanism of multi-scale hybrid entropy + adaptive weight allocation + dynamic threshold arbitration ensures that handwritten receipts receive the same analytical precision as printed receipts, quantifies defect detection, and improves assessment accuracy;

[0199] Adaptive multi-parameter collaborative optimization: Based on real-time feedback, dynamic parameter adjustment and intelligent repair are linked to improve the effective sample rate;

[0200] Improve assessment efficiency: Automated assessment reduces human intervention;

[0201] Improve data quality: Quickly assess and filter the quality of invoice image data to improve test data quality and enhance the reliability of test evaluation.

[0202] Example 2:

[0203] Furthermore, this invention also proposes a ticket image quality assessment system 200 based on multi-scale ROI weighted entropy, such as... Figure 3 As shown, it includes:

[0204] The data preprocessing unit 201 is used to acquire the ticket image, perform basic quality inspection on the ticket image, and preprocess the ticket image after basic quality inspection to generate the target ticket image.

[0205] The weighted calculation unit 202 is used to segment the target ticket image into structured and unstructured regions, and to perform multi-scale ROI weighted entropy calculation on the segmented regions to obtain multi-scale ROI weighted entropy values.

[0206] The quality assessment unit 203 is used to compare the pre-generated dynamic threshold with the multi-scale ROI weighted entropy value, generate a comparison result, and evaluate the image quality of the target ticket image based on the comparison result.

[0207] The basic quality inspection of the ticket images includes:

[0208] Verify the resolution of the ticket images and discard ticket images with a resolution lower than 180 dpi;

[0209] Calculate the RGB mean of the ticket image and remove ticket images with an RGB mean ≥ 240 or an RGB mean ≤ 20;

[0210] Identify physical damage to the ticket image and discard ticket images with tears or missing corners greater than 5%.

[0211] The preprocessing of the invoice images after basic quality inspection includes:

[0212] The data source of the ticket image is determined, and based on the data source, the reflective area of ​​the ticket image is detected, and the reflective area of ​​the ticket image from different data sources is subjected to differentiated reflection elimination processing.

[0213] For the ticket images after anti-reflection processing, the ticket type is identified, and non-rigid geometric deformation correction is performed on the ticket images of different ticket types;

[0214] For the ticket image after non-rigid geometric deformation correction, the color deviation is quantified, and the color deviation of the ticket image is compensated based on the quantization result.

[0215] For the ticket image after color deviation compensation, the handwritten content in the ticket image is detected, and the handwritten content is subjected to stroke enhancement processing.

[0216] The preprocessing of the invoice images after basic quality inspection includes:

[0217] Set the threshold for the dynamically adjusted parameters;

[0218] Within the threshold, the invoice images after basic quality inspection are preprocessed.

[0219] The segmentation of the target ticket image into structured and unstructured regions includes:

[0220] The target ticket image is segmented into three categories using a lightweight U-Net network to obtain probability masks for the table region, printed text region, and handwritten content region.

[0221] Based on the probability mask, the percentage of handwritten pixels p_hand, the percentage of printed text p_print, and the percentage of table lines p_line within each ROI are calculated.

[0222] Based on the handwriting pixel ratio p_hand, the printed text ratio p_print, and the table line ratio p_line, the global handwriting ratio handwriting_ratio is calculated using the following formula:

[0223] handwriting_ratio = Total number of global handwriting pixels / Total number of pixels in the global valid information area;

[0224] Based on the global handwriting ratio, the structured and unstructured regions in the target document image are segmented.

[0225] The structured and unstructured regions are then subjected to refined processing and dynamic weight allocation.

[0226] This includes calculating the multi-scale ROI weighted entropy of the segmented regions to obtain the multi-scale ROI weighted entropy values, including:

[0227] Calculate the local entropy value for the ROI region of the target ticket image;

[0228] Multimodal entropy features are extracted from the segmented regions of the target ticket image;

[0229] Based on the local entropy value calculated from the local entropy value, the extracted multimodal entropy features, and the weights of the segmented regions, a fusion calculation of the hybrid entropy is performed to obtain the weighted entropy value of the multi-scale ROI;

[0230] Specifically, the weights of the segmented regions are calculated based on the proportion of handwritten pixels (p_hand), the proportion of printed text (p_print), and the proportion of table lines (p_line), including the weights of the handwritten region, the printed text region, and the table region.

[0231] Among them, the grading of image quality includes:

[0232] Pass, indicating inclusion in the available test set, with the criterion: Hfinal ≤ 0.9T_dynamic and w_base < 0.5 or Hgradient < 0.8T_dynamic;

[0233] Acceptable defect, indicating allowed for use but with the marking criterion: 0.9T_dynamic < Hfinal ≤ 1.3T_dynamic and w_base ≤ 0.3;

[0234] Boundary case, indicating that a review is required, with the criterion: 1.0T_dynamic < Hfinal ≤ 1.2T_dynamic;

[0235] Rejection, indicating not available for rejection processing, with the criterion: Hfinal > 1.2T_dynamic in the key area or non - key area > 1.5T_dynamic and the number of exceeded standard points ≥ 3 or w_base > 0.8 and ΔE > 12;

[0236] Among them, Hfinal is the multi - scale ROI weighted entropy value, and Tdynamic is the dynamic threshold.

[0237] The implementation of the present invention can achieve quantitative evaluation, efficient correction, and precise screening of the test data of fiscal and tax bill images.

[0238] Embodiment 3:

[0239] Based on the same inventive concept, the present invention also provides a computer device, which includes a processor and a memory. The memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or may also be other general - purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field - Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function, so as to implement the steps of the method in the above - mentioned embodiment.

[0240] Example 4:

[0241] Based on the same inventive concept, this invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the steps of the method in the above embodiments.

[0242] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0243] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0244] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0245] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0246] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0247] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for evaluating the image quality of invoices based on multi-scale ROI weighted entropy, characterized in that, include: Acquire a ticket image, perform basic quality inspection on the ticket image, and preprocess the ticket image after basic quality inspection to generate a target ticket image; The target ticket image is segmented into structured and unstructured regions, and multi-scale ROI weighted entropy is calculated for the segmented regions to obtain multi-scale ROI weighted entropy values. The pre-generated dynamic threshold is compared with the multi-scale ROI weighted entropy value to generate a comparison result. Based on the comparison result, the image quality of the target ticket image is evaluated. The step of performing multi-scale ROI weighted entropy calculation on the segmented region to obtain the multi-scale ROI weighted entropy value includes: Calculate the local entropy value for the ROI region of the target ticket image; Multimodal entropy features are extracted from the segmented regions of the target ticket image; Based on the local entropy value calculated from the local entropy value, the extracted multimodal entropy features, and the weights of the segmented regions, a fusion calculation of the hybrid entropy is performed to obtain the weighted entropy value of the multi-scale ROI; Among them, the weights of the segmented regions are calculated based on the proportion of handwritten pixels p_hand, the proportion of printed text p_print, and the proportion of table lines p_line, including the weights of the handwritten region, the printed text region, and the table region. The formula for calculating the dynamic threshold is as follows: Tdynamic= max(T0-k·σ, 0.5T0); Key region: Tdynamic=max(T0-(k+α·w_base)·σ, 0.4T0); Non-critical regions: Tdynamic = max(T0 - k·σ, 0.6T0); Where α is the weight sensitivity coefficient, w_base is the region weight, T0 is the baseline threshold, k is the sensitivity coefficient, and σ is the noise standard deviation.

2. The method for evaluating the image quality of invoices according to claim 1, characterized in that, Basic quality checks are performed on the document images, including: Verify the resolution of the ticket images and discard ticket images with a resolution lower than 180 dpi; Calculate the RGB mean of the ticket image and remove ticket images with an RGB mean ≥ 240 or an RGB mean ≤ 20; Identify physical damage to the ticket image and discard ticket images with tears or missing corners greater than 5%.

3. The method for evaluating the image quality of invoices according to claim 1, characterized in that, The preprocessing of the invoice images after basic quality inspection includes: The data source of the ticket image is determined, and based on the data source, the reflective area of ​​the ticket image is detected, and the reflective area of ​​the ticket image from different data sources is subjected to differentiated reflection elimination processing. For the ticket images after anti-reflection processing, the ticket type is identified, and non-rigid geometric deformation correction is performed on the ticket images of different ticket types; For the ticket image after non-rigid geometric deformation correction, the color deviation is quantified, and the color deviation of the ticket image is compensated based on the quantization result. For the ticket image after color deviation compensation, the handwritten content in the ticket image is detected, and the handwritten content is subjected to stroke enhancement processing.

4. The method for evaluating the image quality of invoices according to claim 3, characterized in that, Before preprocessing the ticket images after basic quality inspection, the process includes: Set the threshold for the dynamically adjusted parameters; Within the threshold range of the dynamically adjusted parameters, the invoice images after basic quality inspection are preprocessed.

5. The method for evaluating the image quality of invoices according to claim 1, characterized in that, The target ticket image is segmented into structured and unstructured regions, including: The target ticket image is segmented into three categories using a lightweight U-Net network to obtain probability masks for the table region, printed text region, and handwritten content region. Based on the probability mask, the percentage of handwritten pixels p_hand, the percentage of printed text p_print, and the percentage of table lines p_line within each ROI are calculated. Based on the handwriting pixel ratio p_hand, the printed text ratio p_print, and the table line ratio p_line, the global handwriting ratio handwriting_ratio is calculated using the following formula: handwriting_ratio = Total number of global handwriting pixels / Total number of pixels in the global valid information area; Based on the global handwriting ratio, the structured and unstructured regions in the target document image are segmented. The structured and unstructured regions are then subjected to refined processing and dynamic weight allocation.

6. The method for evaluating the image quality of invoices according to claim 1, characterized in that, The image quality grading includes: Passing indicates inclusion in the available test set, with the criteria being: Hfinal ≤ 0.9T_dynamic and w_base < 0.5 or Hgradient < 0.8T_dynamic; An acceptable defect means that use is permitted, but the criteria are: 0.9T_dynamic < Hfinal ≤ 1.3T_dynamic and w_base ≤ 0.3; Boundary cases indicate that the criterion for review is: 1.0T_dynamic < Hfinal ≤ 1.2T_dynamic; Removal indicates that the removal process cannot be performed if: Hfinal > 1.2T_dynamic critical region or non-critical region > 1.5T_dynamic and the number of out-of-specification points is ≥ 3 or w_base > 0.8 and ΔE > 12; Where Hfinal is the multi-scale ROI weighted entropy value, Tdynamic is the dynamic threshold, w_base is the region weight, and Hgradient is the gradient entropy.

7. A ticket image quality assessment system based on multi-scale ROI weighted entropy, characterized in that, include: The data preprocessing unit is used to acquire the ticket image, perform basic quality inspection on the ticket image, and preprocess the ticket image after basic quality inspection to generate the target ticket image. The weighted calculation unit is used to segment the target ticket image into structured and unstructured regions, and to perform multi-scale ROI weighted entropy calculation on the segmented regions to obtain multi-scale ROI weighted entropy values. A quality assessment unit is used to compare a pre-generated dynamic threshold with the multi-scale ROI weighted entropy value, generate a comparison result, and evaluate the image quality of the target ticket image based on the comparison result. The step of performing multi-scale ROI weighted entropy calculation on the segmented region to obtain the multi-scale ROI weighted entropy value includes: Calculate the local entropy value for the ROI region of the target ticket image; Multimodal entropy features are extracted from the segmented regions of the target ticket image; Based on the local entropy value calculated from the local entropy value, the extracted multimodal entropy features, and the weights of the segmented regions, a fusion calculation of the hybrid entropy is performed to obtain the weighted entropy value of the multi-scale ROI; Among them, the weights of the segmented regions are calculated based on the proportion of handwritten pixels p_hand, the proportion of printed text p_print, and the proportion of table lines p_line, including the weights of the handwritten region, the printed text region, and the table region. The formula for calculating the dynamic threshold is as follows: Tdynamic= max(T0-k·σ, 0.5T0); Key region: Tdynamic=max(T0-(k+α·w_base)·σ, 0.4T0); Non-critical regions: Tdynamic = max(T0 - k·σ, 0.6T0); Where α is the weight sensitivity coefficient, w_base is the region weight, T0 is the baseline threshold, k is the sensitivity coefficient, and σ is the noise standard deviation.

8. A computer device, characterized in that, include: One or more processors; A processor is used to execute one or more programs; When the one or more programs are executed by the one or more processors, the method described in any one of claims 1-6 is implemented.

9. A computer-readable storage medium, characterized in that, It contains a computer program, which, when executed, implements the method as described in any one of claims 1-6.

Images