Intelligent auditing system for financial reimbursement documents based on visual character recognition

Abstract

The application relates to the technical field of image recognition, and discloses a financial reimbursement document intelligent auditing system based on visual character recognition, which comprises the following steps: generating a page grayscale image; generating an illumination normalization image; clustering to obtain a basic correction image; generating a forward perturbation image and a reverse perturbation image; calculating a topological coefficient; determining a topological abnormal character box; and generating an auditing marked image. The dense geometric correction link can adapt to local bending and creases of paper, and the output curvature parameter can provide a physical deformation reference for subsequent processing; the constructed local adjacency relationship stability index can reflect the sensitivity of character adjacency topology to mapping disturbance, and can find hidden problems that a single character can be recognized but the adjacency relationship is abnormal; and the finally output visualized marked result can help auditors quickly locate areas that need to be reviewed in key positions, reduce the workload of manual review, and reduce the auditing risks caused by field correlation errors.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, specifically to an intelligent auditing system for financial reimbursement documents based on visual character recognition. Background Technology

[0002] Auditing financial reimbursement documents is a routine part of the financial processes of enterprises and institutions. With the popularization of mobile office, automatically extracting and verifying information by taking pictures of documents with mobile terminals and combining them with visual character recognition technology has become a common method for reimbursement auditing. Existing visual processing workflows typically include several core steps such as image correction, lighting preprocessing, character recognition, and field extraction, which can complete the automatic information entry and preliminary verification of most flat and well-photographed documents.

[0003] To address common issues in mobile photography such as perspective tilt, paper curvature, and non-uniform lighting, existing technologies often employ global perspective transformation, dense grid curvature removal, histogram equalization, or illumination filtering for preprocessing. The processing results are typically evaluated based on overall character recognition rate, page axis alignment, and overall image clarity. These technologies have already been applied in most reimbursement systems.

[0004] In practical applications, even after the above preprocessing, document images may still contain small, localized mapping errors. These errors are minor and do not affect the recognition result of a single character, but they can alter the left-right, top-bottom, or row-level adjacency relationships of adjacent characters, causing characters to be associated with incorrect field regions. Existing evaluation metrics primarily focus on the overall page appearance and single-character recognition results, making it difficult to detect these localized adjacency anomalies. This can easily lead to field extraction errors and affect the accuracy of audit results. Summary of the Invention

[0005] This application provides an intelligent auditing system for financial reimbursement documents based on visual character recognition, which solves the technical problems mentioned in the background.

[0006] This application provides an intelligent auditing system for financial reimbursement documents based on visual character recognition, including: The page normalization module is used to acquire the original document image, and after document boundary segmentation, projection normalization, resampling and grayscale conversion, generate a page grayscale image and a document region mask. The illumination decoupling module is used to calculate the average gray level of the region based on the document region mask and generate an illumination-normalized image using a normalized Gaussian illumination kernel. The dense correction module is used to extract the text line skeleton from the illumination normalized image, form the text tangential angle field, text normal vector and basic correction mapping, and obtain the basic correction image, basic character box, character center, character height, text line set and text curvature through resampling, character detection and text line clustering. The symmetric perturbation module is used to generate a positive perturbation image and a negative perturbation image based on the basic correction mapping, median character height, text normal vector, and text curvature. It matches the basic character box in the two perturbation images and extracts the positive character center, negative character center, positive character height, and negative character height. The topology calculation module is used to establish a fixed character neighborhood based on the character center, calculate the forward adjacency distance, reverse adjacency distance, forward adjacency angle, and reverse adjacency angle, and generate topology coefficients. The anomaly localization module is used to perform character recognition on the basic corrected image and two perturbation images. Based on the topological coefficients, the relative order of characters, and the condition of identical content, it determines the topological anomaly character boxes and forms anomaly regions, anomaly region masks, and anomaly adjacent edge sets. The annotation module is used to reverse map abnormal regions, abnormal region masks, and abnormal adjacent edge sets to the original document image, and overlay region outlines, character lines, character content, and region indexes to generate an audit annotation image.

[0007] The beneficial effects of this application are as follows: It can effectively reduce the global perspective effect caused by shooting angle tilt and distance differences, unify document images under different shooting conditions to a standard coordinate system, and reduce the scene adaptation cost of subsequent processing. The lighting preprocessing stage can maintain the stability of image gray dimensions, suppress the interference of shadows and brightness gradients on character detection and geometric correction, and reduce processing errors introduced by lighting factors. The dense geometric correction stage can adapt to local bending and creases on the paper, and the output curvature parameters can provide a physical deformation reference for subsequent processing. The constructed local adjacency stability index can reflect the sensitivity of character adjacency topology to mapping perturbations, and can discover hidden problems such as single characters being identifiable but adjacency relationships being abnormal. The final output of the visual annotation results can help auditors quickly locate areas that need to be reviewed, reduce the workload of manual review, and reduce the audit risk caused by field association errors. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0009] Figure 1 A schematic diagram of the calculation process of the intelligent auditing system for financial reimbursement documents based on visual character recognition provided in this application embodiment. Detailed Implementation

[0010] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0011] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0012] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0013] like Figure 1 As shown, the intelligent auditing system for financial reimbursement documents based on visual character recognition includes: The page normalization module is used to acquire the original document image, and after document boundary segmentation, projection normalization, resampling and grayscale conversion, generate a page grayscale image and a document region mask. The illumination decoupling module is used to calculate the average gray level of the region based on the document region mask and generate an illumination-normalized image using a normalized Gaussian illumination kernel. The dense correction module is used to extract the text line skeleton from the illumination normalized image, form the text tangential angle field, text normal vector and basic correction mapping, and obtain the basic correction image, basic character box, character center, character height, text line set and text curvature through resampling, character detection and text line clustering. The symmetric perturbation module is used to generate a positive perturbation image and a negative perturbation image based on the basic correction mapping, median character height, text normal vector, and text curvature. It matches the basic character box in the two perturbation images and extracts the positive character center, negative character center, positive character height, and negative character height. The topology calculation module is used to establish a fixed character neighborhood based on the character center, calculate the forward adjacency distance, reverse adjacency distance, forward adjacency angle, and reverse adjacency angle, and generate topology coefficients. The anomaly localization module is used to perform character recognition on the basic corrected image and two perturbation images. Based on the topological coefficients, the relative order of characters, and the condition of identical content, it determines the topological anomaly character boxes and forms anomaly regions, anomaly region masks, and anomaly adjacent edge sets. The annotation module is used to reverse map abnormal regions, abnormal region masks, and abnormal adjacent edge sets to the original document image, and overlay region outlines, character lines, character content, and region indexes to generate an audit annotation image.

[0014] In one alternative implementation, the page normalization module includes the following steps: Step S101: Use a mobile terminal to collect color financial reimbursement documents to form original document images; Original document images refer to three-channel color images obtained by taking pictures of financial reimbursement documents from a mobile terminal equipped with a high-definition camera under natural light or indoor lighting conditions, either facing forward or at an angle, with an image resolution of no less than 1920 x 1080 pixels. Step S102: Perform document boundary segmentation on the original document image, generate a binary document region mask and extract 4 main points. The document region mask is used to limit the calculation range of the average gray level of the region. Document region masking refers to the process of segmenting the foreground and background of the original document image using a semantic segmentation network, marking pixels belonging to the document surface as 1 and background pixels as 0 to obtain a binary image. The main point refers to the four corner points with larger curvature on the outer contour of the document area mask by polygon approximation, and sorted in the order of top left, top right, bottom right and bottom left to obtain the four vertices. Step S103: Establish a projection normalization transformation based on the main character points and the direction of the rectangular correction canvas, so that the four main character points correspond to the four corner points of the rectangular correction canvas respectively. Projection normalization transformation refers to the coordinate transformation relationship obtained by solving the homography matrix using four main points as source points and four vertices of a preset rectangular canvas as target points. Step S104: Perform bilinear resampling and grayscale conversion according to the projection normalization transformation to obtain the page grayscale image.

[0015] A page grayscale image refers to a single-channel image obtained by mapping the original color image onto a rectangular canvas through projection normalization transformation, converting the mapped three-channel pixel values ​​into single-channel grayscale values ​​using a weighted average method, and then filling in blank pixels using bilinear interpolation.

[0016] Furthermore, a lightweight semantic segmentation network is first used to classify the original document images pixel by pixel. During network training, images of expense reports with different lighting, angles, and degrees of wrinkles are used as the training set, and the paper areas of the documents are labeled. After training, inputting the original document image outputs binary segmentation results. Morphological closing operations are then performed on the segmentation results to eliminate small holes, and the outer contour is extracted. The Douglas-Puk algorithm is used to approximate the contour using polygons. When the number of vertices in the approximate polygon is 4, it is directly used as the protagonist point. When the number of vertices is greater than 4, the rotation angle of each vertex is calculated, and four vertices with rotation angles close to 90 degrees are selected and sorted clockwise to ensure that the upper left corner corresponds to the origin of the canvas and the upper right corner corresponds to the upper right corner of the canvas, thus establishing a stable projection transformation relationship. When the expense report document has slight corner occlusion, the segmented contour may have local depressions. Through polygon approximation and corner selection, four protagonist points can be stably extracted, reducing the possibility of misidentifying occluded edge inflection points as document corner points.

[0017] The aspect ratio of the rectangular correction canvas is set to 1.414:1, based on the A5 paper size. The short side of the canvas has 1024 pixels, and the long side has 1448 pixels. The resolution is set to 300 DPI to ensure that the height of the resampled characters is no less than 16 pixels, meeting the resolution requirements for subsequent character detection. When the original input document image resolution is 4032 x 3024 pixels, the output grayscale image size after projection normalization is fixed at 1448 x 1024 pixels. Input images with different shooting distances and resolutions are ultimately mapped onto a canvas of the same size, ensuring parameter universality in subsequent processing steps.

[0018] Understandably, by segmenting the document boundary to obtain the document area, and then extracting four key points to establish a projection normalization transformation, document images from different shooting angles and distances are uniformly mapped to a fixed-size rectangular canvas coordinate system. This reduces global perspective differences caused by tilt, rotation, and scaling during shooting, ensuring that subsequent character detection, geometric correction, and topology calculations are performed in a unified coordinate system. This reduces the propagation of global perspective errors to subsequent local topology stability calculations, providing a consistent coordinate benchmark for local topology anomaly detection. The above embodiment can mitigate the global perspective impact caused by differences in shooting angle, tilt, and distance, uniformly converting expense reimbursement document images acquired under different shooting conditions into standard-sized, standard-oriented grayscale images. This reduces the need to adjust subsequent processing parameters for different shooting scenarios, lowers the computational complexity of subsequent steps such as illumination decoupling, dense correction, and topology calculations, and reduces detection errors caused by scene differences, thereby improving the overall scene adaptability of the system.

[0019] In one alternative implementation, the illumination decoupling module includes the following steps: Step S201: Accumulate the pixel grayscale of the page grayscale image within the page area defined by the document area mask and count the number of pixels, and calculate the average grayscale of the area. The formula for calculating the average gray level of a region is as follows: ; in, Indicates the average gray level of the region. Represents the pixel coordinates of the grayscale image on the page. Indicates a document region mask. This indicates the pixel coordinates of the grayscale image on the page. The grayscale value at that location; The average gray level of a region refers to the gray level value obtained by calculating the arithmetic mean of the gray values ​​of all page grayscale image pixels within the document region mask. Step S202: Set the normalized Gaussian illumination kernel and Gaussian kernel scale, perform convolution on the page grayscale image to obtain the slowly varying illumination component, the sum of the kernel weights of the normalized Gaussian illumination kernel is 1, and the Gaussian kernel scale is a positive value; The normalized Gaussian illumination kernel is a convolution kernel obtained by discrete sampling and normalization of a two-dimensional Gaussian function. The sum of all weights in the kernel is 1, and it is used for low-pass filtering of grayscale images. Gaussian kernel scale refers to the standard deviation of a two-dimensional Gaussian function. The preferred value is 15 to 35 pixels. When setting it, refer to the character size and line spacing on the document to ensure that high-frequency information of character strokes can be smoothed and only the gradually changing lighting distribution on the paper is retained. The slowly varying illumination component refers to the low-frequency component image that reflects the slow changes in the overall brightness of the image, obtained by performing a two-dimensional convolution operation between a normalized Gaussian illumination kernel and the page grayscale image. Step S203: Calculate the ratio using the product of the region's average gray level and the page's gray level as the numerator and the sum of the gradually varying illumination component and the gray level stability term as the denominator. The grayscale stability term refers to a grayscale constant with a value of 1, used to avoid numerical errors where the denominator is zero during division operations; Step S204: Form an illumination-normalized image according to the ratio of each pixel; The formula for calculating the illumination-normalized image is as follows: ; in, This represents the image with normalized illumination at pixel coordinates. grayscale value at that location Indicates the average gray level of the region. This indicates the pixel coordinates of the grayscale image on the page. grayscale value at that location The Gaussian kernel size is... Normalized Gaussian illumination kernel, Represents the Gaussian kernel scale. Represents pixel coordinates The gradually changing lighting component at the location, This represents the grayscale stability term.

[0020] An illumination-normalized image is a brightness-corrected image obtained by dividing the page's grayscale image pixel by the sum of the slowly varying illumination component and the grayscale stability term, and then multiplying by the area's average grayscale, based on the area's average grayscale.

[0021] Furthermore, the value of the Gaussian kernel scale is directly related to the size of the characters and the line spacing in the document image. When setting it, it is necessary to ensure that the effective range of the kernel is greater than the size of a single character and the line spacing of the text, so that the convolution operation can smooth out the high-frequency details of character strokes and text lines, and only retain the large-area gradual brightness changes caused by uneven lighting shadows. When processing conventionally printed VAT invoices or reimbursement forms, the character height is usually 20 to 30 pixels and the line spacing is 8 to 12 pixels. In this case, the Gaussian kernel scale can be set to 25 pixels. When processing thermal paper receipts with smaller font sizes, the character height is usually 12 to 18 pixels and the line spacing is 4 to 6 pixels. In this case, the Gaussian kernel scale can be set to 15 pixels to reduce the problem of increased computation caused by an excessively large kernel scale or the problem of an excessively small kernel scale failing to completely smooth the character information. When the Gaussian kernel scale is set to 5 pixels, the convolution result retains the edge information of the character strokes, causing the illumination component to contain character details. After normalization, the character edges may exhibit over-enhanced artifacts. When the scale is set to 50 pixels, the convolution result has difficulty accurately tracking the boundaries of local shadows, which may lead to insufficient correction of shadow areas.

[0022] The grayscale stability term is preferably set to 1 to avoid division by zero errors when the gradually varying illumination component is 0. Using the region average grayscale as the normalization benchmark can reduce the interference of gradually varying illumination on character detection and text line skeleton extraction.

[0023] Understandably, using the regional average grayscale as a normalization benchmark and correcting for uneven illumination through division operations can reduce character contrast changes or local overexposure problems caused by traditional histogram equalization or gamma correction methods. This ensures that the corrected image maintains stable grayscale characteristics, making it suitable for subsequent character detection and geometric correction steps. The above embodiment can mitigate the impact of uneven brightness caused by local shadows and different light source angles in document images, suppress the interference of shadow boundaries and brightness gradients on character edge detection and text line skeleton extraction, reduce the introduction of additional image artifacts, and ensure stable input image quality for subsequent geometric correction and character detection steps.

[0024] In one alternative implementation, the dense correction module includes the following steps: Step S301: Detect document boundaries and text line skeletons in the illumination normalized image, and generate text tangential angle field and text normal vector perpendicular to the text tangential direction based on the text line skeleton. The text line skeleton refers to the single-pixel width of the text line centerline extracted after refining the connected components of each text line in a normalized illumination image by detecting text line regions; The text tangential angle field refers to the angle between the skeleton tangent and the horizontal direction calculated point by point along the text line skeleton, and then the angle is spread to the entire text line area to obtain the text direction angle corresponding to each pixel position. The text normal vector is a unit direction vector obtained by rotating the text tangent angle by 90 degrees, pointing in the normal direction of the text line; Step S302: Set control points in the rectangular correction canvas, generate a basic correction map based on the control point displacement and interpolation weights, and the sum of the interpolation weights of the same pixel is 1. The formula for calculating the baseline orthodontic mapping is as follows: ; in, Represents the coordinates of the rectangular correction canvas. The corresponding basic correction mapping, Represents the coordinates of the rectangular correction canvas. Indicates the number of control points. Indicates the first The coordinates of each control point on the rectangular correction canvas interpolation weights, Indicates the first Two-dimensional displacement of each control point; The interpolation weights satisfy the following formula: ; in, Indicates the number of control points. Indicates the first The coordinates of each control point on the rectangular correction canvas interpolation weights, Represents the coordinates of the rectangular correction canvas; Control points are discrete coordinate points arranged on a rectangular correction canvas according to a uniform grid. The spacing between control points in the horizontal and vertical directions is preferably set to 32 pixels.

[0025] Control point displacement refers to the horizontal and vertical offset of each control point obtained by optimizing the objective functions of text line horizontality and character spacing uniformity. Interpolation weights refer to the weights of the four control points surrounding each canvas pixel calculated using the bilinear interpolation method. The sum of the weights of the four control points at the same pixel position is 1. The basic correction mapping refers to the mapping relationship from the corrected canvas coordinates to the illumination-normalized image coordinates obtained by adding the initial coordinates of each canvas pixel to the offset of the displacement of the surrounding control points after interpolation weighting. Step S303: Perform bilinear resampling on the illumination normalized image according to the basic correction mapping to generate the basic correction image, detect the basic character boxes and perform text line clustering to obtain the character center, character height and text line set; The baseline correction image refers to the geometrically corrected image obtained by bilinear resampling of the illumination normalized image based on the baseline correction mapping, and mapping the pixel values ​​in the illumination normalized image to the corresponding positions on the correction canvas. The basic character frame refers to the axis-aligned bounding rectangle of each individual character detected on the basic corrected image using a pre-trained character detection model, including the coordinates of the top-left corner and the width and height dimensions of the rectangle; The character center refers to the coordinates of the center point obtained by adding half the width to the horizontal coordinate of the top left corner of the basic character box and adding half the height to the vertical coordinate of the top left corner; Character height refers to the vertical height dimension of the basic character box; The text line set refers to the set of characters in each line obtained by clustering all basic character boxes according to the vertical axis, grouping character boxes whose vertical axis difference is less than one-third of the height of the median character into the same line, and then sorting them according to the horizontal axis. Step S304: Generate the median character height based on the median of all character heights; The formula for calculating the height of the median character is as follows: ; in, Indicates the height of the median character. Indicates the total number of basic character boxes. Indicates the first The character height of each basic character box; The median character height refers to the value located in the middle of the sorted result after all character heights are sorted from smallest to largest; Step S305: Convert the text tangential angle field and text normal vector to a rectangular correction canvas, and generate the text curvature based on the height of the median character and the rate of change of the text tangential angle field along the arc length of the text line; The formula for calculating text curvature is as follows: ; in, Represents the coordinates of the rectangular correction canvas. Text curvature at that location, Indicates the height of the median character. Represents the coordinates of the rectangular correction canvas. The text tangent angle at that location, Represents the arc length coordinates of the text line. This represents the rate of change of the text tangent angle along the arc length of the text line.

[0026] The arc length coordinate of a text line refers to the curve length value corresponding to each skeleton point, obtained by accumulating the distance between adjacent pixels along the skeleton pixel by pixel, starting from the left end of the text line skeleton. Text curvature is the product of the rate of change of the text tangent angle along the arc length of the text line and the height of the median character.

[0027] Furthermore, control points are arranged on a rectangular correction canvas using a uniform grid, with the same spacing in both the horizontal and vertical directions. The crease width of a typical expense report is usually between 20 and 50 pixels. For a 1448 x 1024 pixel correction canvas, 46 control points are arranged horizontally and 33 vertically with a 32-pixel spacing, totaling 1518 control points. This density ensures that the small-scale deformation caused by creases and local warping is properly fitted without generating excessive computational load. When setting up the control points, a ring of control points is first placed around the edge of the canvas, with the displacement of these edge control points fixed at 0 to ensure that the corrected image edge remains rectangular. The inner control points are arranged using a uniform grid, and their displacement is obtained through optimization. When there is a horizontal crease in the middle of the document, the text lines in the crease area are locally curved. The control points with a spacing of 32 pixels can be set with independent displacement parameters on both sides of the crease to fit the local deformation at the crease. If the control point spacing is set to 128 pixels, the deformation at the crease may be smoothed by the control point interpolation, making it difficult to obtain an accurate correction result.

[0028] First, connected component analysis is used to extract all text connected components from the illumination-normalized image. Text connected components in the same row are merged into a text line region. A morphological thinning algorithm is then used to iteratively erode each text line region until the region width becomes a single pixel, resulting in a single-pixel text line skeleton. Next, for each point on the skeleton, a straight line is fitted to the five skeleton points before and after that point. The angle between the fitted line and the horizontal direction is the text tangent angle for that point. Then, a distance transformation method is used to spread the tangent angles from the skeleton to the entire text line region. The tangent angle of each non-skeleton pixel is taken as the tangent angle of the nearest skeleton point, resulting in the text tangent angle field for the entire image. The text normal vector is directly obtained by rotating the tangent angle by 90 degrees, ensuring that the normal vector is perpendicular to the text direction.

[0029] Character detection employs a pre-trained single-character detection model. During training, images of characters from expense reports with different fonts, sizes, and print qualities are used as the training set. The bounding rectangle of each character is labeled. After training, inputting the base corrected image outputs the bounding rectangle positions and confidence scores for all single characters. Detection results with confidence scores below 0.5 are filtered out to obtain the basic character boxes. For text line clustering, the center y-coordinates of all character boxes are first calculated. Mean-shift clustering is used to cluster the y-coordinates, with a clustering bandwidth set to one-third of the median character height. Characters within the same cluster are grouped into the same text line. Then, characters within the same line are sorted by their center x-coordinates from smallest to largest, resulting in an ordered set of text lines. When a line of characters has a local vertical offset, as long as the offset is less than one-third of the median character height, it will still be clustered into the same line, reducing the chance of misclassifying characters in curved text lines as belonging to different lines.

[0030] Understandably, the tangential angle and normal vector are calculated point-by-point from the extracted text line skeleton. The rate of change of the tangential angle is calculated along the arc length of the text line, and then multiplied by the median character height to obtain the text curvature. This curvature is a dimensionless value independent of image resolution and can reflect the degree of local bending of the paper surface. The larger the curvature, the more obvious the creases or local warping of the paper surface. In subsequent perturbation processing, larger perturbation displacements are applied to these locations, and smaller perturbation displacements are applied to flat areas, so that the perturbation amplitude matches the actual physical deformation degree, reducing the topology detection deviation caused by excessively large or small perturbations. The above embodiment can realize dense geometric correction of document images, correcting the curved, creased, and locally warped paper surface to a flat state, and outputting a normalized curvature value that matches the local paper surface bending degree. This provides an amplitude benchmark that conforms to the actual deformation law for subsequent symmetrical perturbations, ensuring the physical rationality of perturbation processing and improving the accuracy of topology detection.

[0031] In one alternative implementation, the symmetric perturbation module includes the following steps: Step S401: Set the perturbation ratio, which satisfies the following formula: ; in, Indicates the perturbation ratio; The perturbation ratio refers to a dimensionless coefficient used to control the magnitude of the perturbation displacement, and its preferred value range is 0.1 to 0.3; Step S402: Along the two opposite directions of the text normal vector, the product of the median character height, text curvature, and perturbation ratio is superimposed onto the basic correction map to generate the forward perturbation map and the reverse perturbation map; the calculation formulas for the forward perturbation map and the reverse perturbation map are as follows: ; in, Represents the coordinates of the rectangular correction canvas. The corresponding positive perturbation mapping, Represents the coordinates of the rectangular correction canvas. The corresponding inverse perturbation mapping, Represents the coordinates of the rectangular correction canvas. Indicates the baseline correction mapping. Indicates the perturbation ratio. Indicates the height of the median character. Represents the coordinates of the rectangular correction canvas. Text curvature at that location, Represents the coordinates of the rectangular correction canvas. The text normal vector at that location; Forward perturbation mapping refers to the coordinate mapping obtained by superimposing perturbation displacements along the positive direction of the text normal vector on each pixel position based on the basic correction mapping. Inverse perturbation mapping refers to the coordinate mapping obtained by superimposing perturbation displacements along the opposite direction of the text normal vector at each pixel position on the basis of the basic correction mapping. Step S403: Perform bilinear resampling on the illumination normalized image according to the forward perturbation mapping and the reverse perturbation mapping respectively to generate the forward perturbation image and the reverse perturbation image; The formulas for calculating the forward perturbation image and the reverse perturbation image are as follows: ; in, This represents the coordinates of the forward perturbation image on the rectangular correction canvas. grayscale value at that location Represents the coordinates of the inverse perturbation image on the rectangular correction canvas. grayscale value at that location Represents a normalized illumination image. This represents a positive perturbation mapping. Indicates the reverse perturbation mapping; A forward perturbation image refers to a perturbation-corrected image obtained by bilinear resampling an illumination-normalized image based on a forward perturbation mapping. The inverse perturbation image refers to the perturbation-corrected image obtained by bilinear resampling of the illumination-normalized image based on the inverse perturbation mapping; Step S404: Using the same character detector as the basic corrected image in the forward perturbation image and the reverse perturbation image, obtain the forward character box, the reverse character box, the forward character center, the reverse character center, the forward character height, and the reverse character height; A forward character bounding box refers to the bounding rectangle of a single character detected on a forward perturbation image using the same character detection model as the basic character bounding box detection. The reverse character box refers to the bounding rectangle of a single character detected on the reverse perturbation image using the same character detection model as the basic character box detection. The forward character center refers to the coordinates of the center point of the forward character box; the reverse character center refers to the coordinates of the center point of the reverse character box. Positive character height refers to the vertical height of the positive character box; negative character height refers to the vertical height of the negative character box. Step S405: Determine the character matching result based on the maximum positive overlap between the base character box and the forward and reverse character boxes, and mark characters without positive overlap as unmatched characters.

[0032] An unmatched character refers to a base character in the forward or reverse perturbation image that does not have a corresponding character frame with an intersection-union ratio greater than 0.

[0033] Furthermore, during character matching, for each basic character box, its intersection-union ratio (IUR) with both the forward and reverse character boxes is calculated. The character box with the highest IUR is selected as the corresponding matching result. A successful match is considered achieved when the highest IUR is greater than 0, and a non-matched character is marked when the highest IUR is equal to 0. If the IUR of a basic character box with two forward character boxes is 0.6 and 0.2 respectively, the forward character box with an IUR of 0.6 is selected as the matching result. If the IUR of a basic character box with all forward character boxes is 0, the character is marked as a non-matched forward character.

[0034] The perturbation ratio needs to be chosen to balance the sensitivity and stability of topology detection. If the value is too small, the perturbation displacement is insufficient, and the changes in the adjacency relationships of unstable regions are not obvious, making it difficult to effectively detect topology anomalies. If the value is too large, the perturbation displacement exceeds the normal character spacing, which may cause changes in the character adjacency relationships of stable regions, resulting in false detections. The perturbation ratio is preferably set to 0.2, and the specific perturbation displacement is determined by the perturbation ratio, the median character height, and the text curvature at the corresponding position.

[0035] Understandably, by applying positive and negative perturbation displacements along the text normal, centered on the base correction mapping, the magnitude of the perturbation displacement is proportional to the local text curvature. The larger the curvature of the crease region, the greater the perturbation displacement, and the smaller the curvature of the flat region, the smaller the perturbation displacement. The generated positive and negative perturbation images exhibit only a slight normal shift, and the overall correction effect is essentially consistent with the base correction image. No additional flat reference image is needed for comparison, thus constructing a self-consistent control image pair, providing a benchmark for subsequent topological stability detection. The above embodiment can generate two sets of control correction images with only a slight normal shift. The overall correction quality of both sets of images is similar to the base correction image, and the character content can be correctly recognized. No standard flat sample is needed as a reference, providing a self-consistent control data source for subsequent topological stability determination, reducing the system's dependence on reference images.

[0036] In one alternative implementation, the topology calculation module includes the following steps: Step S501: For each character center, select the number of neighboring characters with the nearest Euclidean distance from the remaining character centers to establish a fixed character neighborhood; The number of neighboring characters satisfies the following formula: ; in, Indicates the number of neighboring characters. Represents a set of integers; The number of neighboring characters refers to the number of nearest neighbor characters included in the neighborhood calculation for each character. It is preferably set to 4, which can cover the directly adjacent characters above, below, left, and right of the character and reflect the local adjacency relationship. Fixed character neighborhood refers to calculating the Euclidean distance of each basic character to all other basic characters and selecting a neighborhood set consisting of a specified number of characters that are closest to it. Step S502: Generate the forward adjacency distance based on the Euclidean distance between the forward character centers, the average forward character height, and the stable character height term; generate the reverse adjacency distance based on the Euclidean distance between the reverse character centers, the average reverse character height, and the stable character height term. The formulas for calculating forward adjacency distance and reverse adjacency distance are as follows: ; in, Indicates the first The first character and the fixed character in the neighborhood of the first character Positive adjacency distance between characters Indicates the first The first character and the fixed character in the neighborhood of the first character The reverse adjacency distance between characters Indicates the first A positive character center, Represents the number of fixed characters in the neighborhood. A positive character center, Indicates the first A reverse character center, Represents the number of fixed characters in the neighborhood. A reverse character center, Indicates the first One positive character height, Represents the number of fixed characters in the neighborhood. One positive character height, Indicates the first One reverse character height, Represents the number of fixed characters in the neighborhood. One reverse character height, Indicates the character height stability item; The forward adjacency distance refers to the dimensionless distance obtained by dividing the Euclidean distance between the centers of a pair of characters in a forward perturbation image by the sum of the average height of the two characters and the character height stability term. The reverse adjacency distance refers to the dimensionless distance obtained by dividing the Euclidean distance between the centers of a pair of characters in the reverse perturbation image by the sum of the average height of the two characters and the character height stability term. The character height stability term is a pixel-level constant with a value of 1, used to reduce division-by-zero errors when the character height is 0; Step S503: Generate forward adjacent angles based on the difference between the horizontal and vertical coordinates of the forward character centers, and generate reverse adjacent angles based on the difference between the horizontal and vertical coordinates of the reverse character centers; The formulas for calculating the forward adjacency angle and the reverse adjacency angle are as follows: ; in, Indicates the first The first character and the fixed character in the neighborhood of the first character Positive adjacency angles between characters Indicates the first The first character and the fixed character in the neighborhood of the first character The reverse adjacent angle between characters, Indicates the first The horizontal coordinate of the center of each positive character. Represents the number of fixed characters in the neighborhood. The horizontal coordinate of the center of each positive character. Indicates the first The horizontal coordinate of the center of each reverse character Represents the number of fixed characters in the neighborhood. The horizontal coordinate of the center of each reverse character Indicates the first The y-coordinate of the center of each positive character. Represents the number of fixed characters in the neighborhood. The y-coordinate of the center of each positive character. Indicates the first The y-coordinate of the center of each reverse character Represents the number of fixed characters in the neighborhood. The y-coordinate of the center of each reverse character; The forward adjacency angle refers to the angle between the line connecting the centers of neighboring characters and the horizontal direction in a forward perturbation image; the reverse adjacency angle refers to the angle between the line connecting the centers of neighboring characters and the horizontal direction in a reverse perturbation image. Step S504: The absolute value of the difference between the forward adjacency distance and the reverse adjacency distance is used as the distance term, and the product of the result obtained by subtracting the cosine value of the difference between the forward adjacency angle and the reverse adjacency angle from 1 and the adjacency angle weight is used as the direction term. The sum of the distance term and the direction term is averaged in the neighborhood to obtain the topology change. The formula for calculating topological changes is as follows: ; in, Indicates the first Topological changes per character Indicates the first A fixed character neighborhood of a character. Indicates the first The fixed character neighborhood cardinality of a character. Indicates the first The first character and the fixed character in the neighborhood of the first character Positive adjacency distance between characters Indicates the first The first character and the fixed character in the neighborhood of the first character The reverse adjacency distance between characters This represents the adjacency angle weight used for the adjacency angle variation term. Indicates the first The first character and the fixed character in the neighborhood of the first character Positive adjacency angles between characters Indicates the first The first character and the fixed character in the neighborhood of the first character The reverse adjacent angle between characters; The adjacency angle weight is a dimensionless coefficient used to balance the contributions of distance and direction changes. It is preferably set to 1 to balance the contributions of distance and direction changes in topology calculation. Topological change refers to the average value obtained by summing the absolute value of the distance change and the direction change value of all adjacent character pairs in the neighborhood of a fixed character, which reflects the degree of fluctuation of the adjacency relationship under perturbation. Step S505: Perform an exponential operation on the negative of the topological change to generate topological coefficients, and set the topological coefficients of unmatched characters to 0; The formula for calculating the topology coefficient is as follows: ; in, Indicates the first Topological coefficients of 1 character, Indicates the first The topological changes per character; The topological coefficients of the matched characters satisfy the following formula: ; in, Indicates the first The topological coefficient of each matched character.

[0037] The topology coefficient is a value obtained by taking the negative exponent of the topology change, and its value ranges from 0 to 1. The topology coefficient of unmatched characters is directly set to 0.

[0038] Furthermore, the value of the number of neighboring characters needs to ensure that it covers the directly adjacent characters around a character, while not including irrelevant characters that are too far away. The text on a typical expense report is horizontal, and each character has four directly adjacent characters: two characters in the same row to the left and right, and two characters in the adjacent rows above and below. Therefore, the preferred number of neighboring characters is 4, which can cover the local adjacency range of a character and reflect changes in local adjacency relationships. When processing expense reports with larger font sizes and larger line spacing, the number of neighboring characters can be set to 3, only including the left and right adjacent characters and the nearest characters in the adjacent rows above and below. When processing thermal paper receipts with smaller font sizes and denser layout, the number of neighboring characters can be set to 6, including more surrounding characters to improve statistical stability. When the number of neighboring characters is set to 2, it can only cover the two nearest neighboring characters, making it difficult to detect changes in the adjacency relationships of characters in the adjacent rows above and below, and may miss cross-line topological anomalies. When the number of neighboring characters is set to 10, it will include irrelevant characters that are far away. Changes in the adjacency relationships of these characters have a weaker indication effect on local topological anomalies and may reduce the sensitivity of the topology coefficient.

[0039] The adjacency angle weight is used to adjust the proportion of changes in adjacency distance and adjacency direction in the total topology changes. A larger weight value indicates a greater impact of directional changes on the topology coefficients, while a smaller weight value indicates a greater impact of distance changes. Since local topology anomalies can manifest as changes in both adjacency distance and adjacency direction, and both are equally indicative of topology anomalies, the adjacency angle weight is preferably set to 1, ensuring that the numerical ranges of the distance and direction change terms are roughly the same, resulting in comparable contributions. When the adjacency angle weight is set to 0, only distance changes are considered, making it difficult to detect topology anomalies where character pairs are rotated but the distance remains essentially unchanged. When the adjacency angle weight is set to 5, the contribution of directional changes is far greater than that of distance changes, and even small angular noise can cause a significant drop in the topology coefficients, increasing the possibility of false detections.

[0040] Understandably, by calculating the normalized distance and adjacency orientation angle of adjacent character pairs in the forward and reverse perturbation images respectively, and comparing the differences in distance and angle between the two perturbation images, the topological change is obtained through fusion. This is then transformed by a negative exponential transformation to obtain the topological coefficient. The closer the topological coefficient is to 1, the more stable the local adjacency relationship; the closer it is to 0, the more sensitive the local adjacency relationship is to perturbations. This index can quantitatively reflect the stability of character adjacency topology and can identify hidden anomalies where a single character is identifiable but the adjacency relationship has drifted. The above embodiment can generate a quantitative local character topological stability index that reflects the stability of local adjacency relationships without relying on character recognition results. It can identify hidden layout distortions where the content of a single character is identifiable but the adjacent topology has been misaligned, thus overcoming the deficiency of traditional character recognition rate indicators in detecting local topological anomalies.

[0041] In one optional implementation, the anomaly localization module includes the following steps: Step S601: Using the same character recognizer in the base corrected image, the forward perturbation image, and the reverse perturbation image, generate the base character content, forward character content, and reverse character content corresponding to the base character frame, the forward character frame, and the reverse character frame, respectively. Basic character content refers to the text content corresponding to each basic character frame identified on the basic corrected image using an optical character recognition model. Positive character content refers to the text content corresponding to each matching character identified on a positive perturbation image using the same optical character recognition model; Reverse character content refers to the text content corresponding to each matching character identified on the reverse perturbation image using the same optical character recognition model. Step S602: When the topology coefficient is less than the topology coefficient threshold, the basic character box is recorded as a topology abnormal character box. The topology coefficients and the topology coefficient threshold satisfy the following formula: ; in, Indicates the first Topological coefficients of 1 character, This represents the topology coefficient threshold used for topology coefficient determination. The topology coefficient threshold is a critical value used to determine whether a character topology is abnormal. It is preferably set to 0.6 to balance the false negative rate and false positive rate of anomaly detection. Topologically unstable character frames refer to basic character frames that have been determined to have topological instability issues. Step S603: When the character pair in the fixed character neighborhood undergoes a horizontal or vertical relative order flip between the forward character center and the reverse character center, the base character frame of the character pair is recorded as the topologically abnormal character frame. The horizontal relative order reversal satisfies the following formula: ; in, Indicates the first The horizontal coordinate of the center of each positive character. Represents the number of fixed characters in the neighborhood. The horizontal coordinate of the center of each positive character. Indicates the first The horizontal coordinate of the center of each reverse character Represents the number of fixed characters in the neighborhood. The horizontal coordinate of the center of each reversed character; The vertical relative order flipping satisfies the following formula: ; in, Indicates the first The y-coordinate of the center of each positive character. Represents the number of fixed characters in the neighborhood. The y-coordinate of the center of each positive character. Indicates the first The y-coordinate of the center of each reverse character Represents the number of fixed characters in the neighborhood. The y-coordinate of the center of each reverse character; Horizontal relative order flipping refers to the phenomenon that for a pair of adjacent characters, character A is to the left of character B in the forward perturbation image, and character A is to the right of character B in the reverse perturbation image; Vertical relative order flipping refers to the phenomenon that for a pair of adjacent characters, character A is above character B in the forward perturbation image, and character A is below character B in the reverse perturbation image; Step S604: If the content is the same, the basic character content, the forward character content, and the reverse character content are the same. Only when the content is the same, the topological abnormal character box is retained. The topological abnormal character boxes that overlap or have adjacent boundaries are merged according to the text line set to form non-overlapping abnormal areas. The following formula applies when the content is identical: ; in, Indicates the first Basic character content, Indicates the first A positive character content, Indicates the first One reversed character content; An abnormal region refers to a connected rectangular region obtained by merging adjacent or overlapping topological abnormal character boxes, and different abnormal regions do not overlap with each other. Step S605: Generate an abnormal region mask based on the abnormal region, save the character pairs that have undergone relative order flipping, and form an abnormal adjacent edge set, wherein the pixel value covered by the abnormal region is 1, and the value of the remaining pixels is 0.

[0042] Anomaly region masking refers to creating a single-channel image with the same size as the base corrected image, setting the pixel values ​​in the abnormal regions to 1, and setting the pixel values ​​in the remaining regions to 0 to obtain a binary image. An abnormal adjacency set is a set of all character pairs that have undergone horizontal or vertical relative order flipping.

[0043] Furthermore, the topology coefficient threshold was optimized on a labeled test set containing images of expense reports with varying degrees of wrinkling and lighting conditions. Character regions exhibiting topological anomalies were manually marked, and false positive and false negative rate curves were plotted under different thresholds. The optimal threshold was selected based on the smallest weighted sum of the false positive and false negative rates. In typical scenarios, a topology coefficient threshold of 0.6 is preferred. When the threshold is set to 0.8, many normal characters have topology coefficients below 0.8, potentially leading to an increased false positive rate. When the threshold is set to 0.3, only characters with severe topological anomalies are detected, potentially leading to an increased false negative rate and making it difficult to meet audit requirements.

[0044] When merging abnormal regions, the topological abnormal character boxes are grouped according to the text line set, and topological abnormal character boxes that overlap or have common edges or vertices are merged within the same text line to form non-overlapping abnormal regions.

[0045] Understandably, a dual-judgment rule is used to mark abnormal characters. When a character's topological coefficient is below a threshold, it is marked as abnormal. When adjacent character pairs undergo horizontal or vertical relative order flipping, both characters are simultaneously marked as abnormal. Additionally, a three-image character content consistency check is added. Only when the corresponding character's recognition content in the base image, the forward perturbation image, and the reverse perturbation image is completely consistent is the topological anomaly judgment result retained. This reduces topological misjudgments caused by character recognition errors or missing characters, improving the accuracy of anomaly detection. The above embodiment can reduce misjudgments caused by non-topological factors such as pure character recognition errors, character occlusion, and character missing characters. Under the premise that the character content can be normally recognized, it can locate character regions with adjacent topological instability risks, reducing the false detection rate of anomaly detection and improving the accuracy of audit marking.

[0046] In one alternative implementation, the annotation module includes the following steps: Step S701: The coordinates of the rectangular correction canvas of the abnormal region, the abnormal region mask, and the abnormal adjacent edge set are sequentially converted into the coordinates of the original document image using the inverse transformations of the basic correction mapping and the projection normalization transformation; the coordinate transformation satisfies the following formula: ; in, Represents the coordinates of the original document image. Represents the coordinates of the rectangular correction canvas. Represents the coordinates of the rectangular correction canvas. The corresponding basic correction mapping, This represents the inverse transformation of the projection normalization transformation; The coordinates of the original document image refer to the coordinate positions obtained after mapping the coordinates of the abnormal areas and the center coordinates of the characters on the correction canvas back to the original document image through basic correction mapping and projection normalization inverse transformation. Step S702: Perform connected boundary tracing, contour vectorization, and closed contour rasterization on the abnormal region to generate the region contour; Region contour refers to the closed polygon contour lines extracted by tracing the connected domain boundaries of anomaly region masks, and then vectorizing them to obtain a smooth contour path; Step S703: For each character pair in the abnormal adjacent edge set, draw a solid line connecting the centers of the two characters to form a character connection line; Character connection lines refer to solid yellow lines with a width of 2 pixels that connect the centers of each pair of characters in the abnormal adjacent edge set on the original document image; Step S704: Write the basic character content and region index into the overlay layer, with each abnormal region corresponding to a unique region index; A region index refers to a consecutive Arabic numeral identifier starting from 1 assigned to each abnormal region, arranged from top left to bottom right. Step S705: The overlay layer consisting of the region outline, character lines, basic character content and region index is combined with the original document image pixel by pixel to output the audit annotation image.

[0047] Audit annotation image refers to the image obtained by overlaying region outline characters and region index annotations on the original document color image, which is the final output result of the system.

[0048] Furthermore, the coordinate transformation consists of two steps: First, according to the basic correction mapping, the coordinates of the rectangular correction canvas are converted to the illumination-normalized image coordinates; then, according to the inverse transformation of the projection normalization transformation, the illumination-normalized image coordinates are converted to the original document image coordinates. During the mapping, all vertices on the abnormal area contour and the endpoints of the character connections are sequentially transformed to the original document image coordinates twice, and then the contour and connections are drawn in the original document image coordinates. If the vertex coordinates of a certain abnormal area contour on the correction canvas are 100, 200, the corresponding coordinates on the illumination-normalized image are first obtained through the basic correction mapping, and then the corresponding coordinates on the original document image are obtained through the inverse transformation of the projection normalization transformation. After all vertices are transformed, the corresponding abnormal area contour can be drawn on the original image.

[0049] The overlay layer is created as a four-channel transparent image with the same size as the original document image. The region outline is drawn with a 3-pixel wide solid red line, and the interior of the outline is filled with 30% red transparency. Character connections are drawn with a 2-pixel wide solid yellow line. Region indexes are in black 16-point sans-serif font and are placed in the upper right corner of the corresponding abnormal region outline. A 1-pixel wide white stroke is added to the annotation text to ensure clear visibility under different backgrounds. During overlay layer composition, the transparency channels of the four-channel overlay layer are used as weights and blended pixel-by-pixel with the original color image to obtain the final audit annotation image. When the abnormal region is located at the crease in the middle of the document, the red semi-transparent fill indicates the extent of the abnormal region, the yellow lines indicate adjacent misaligned character pairs, and the numeric index in the upper right corner helps auditors identify the abnormal record.

[0050] Understandably, the abnormal regions and adjacent edges detected on the correction canvas are transformed back to the coordinate space of the original captured image through two coordinate transformations. Semi-transparent filled outlines, abnormal character connections, and numerical index annotations are then overlaid on the original image. The annotation results are directly displayed on the original captured document image, consistent with the document image actually seen by the auditor, reducing the need for auditors to perform positional transformations by comparing the corrected image. This embodiment converts abstract topological stability values ​​into intuitive visual annotations, directly locating abnormal positions on the original document image. Auditors can quickly locate areas requiring focused attention and examine the character adjacency relationships of abnormal positions without coordinate transformations, improving the efficiency of manual review.

[0051] This system can be deployed on enterprise and institution financial audit terminals or mobile reimbursement applications. During deployment, all neural network models undergo offline training and parameter initialization, and the entire process can be completed without relying on a network connection. During model training, the training samples for the document segmentation model, single-character detection model, and optical character recognition model cover common reimbursement documents such as VAT invoices, general expense reimbursement forms, and thermal paper receipts with varying degrees of wrinkles, lighting conditions, and shooting angles. Document areas, single-character bounding rectangles, and corresponding character content are manually labeled. The training process uses a stochastic gradient descent optimizer with an initial learning rate of 0.001 and 100 training epochs. After training, the model weights are fixed to ensure detection stability in different scenarios.

[0052] In actual use, after finance personnel upload images of expense reimbursement documents taken with their mobile devices, the system automatically performs a full-process processing including page normalization, illumination correction, density geometry correction, symmetry perturbation comparison, topological stability calculation, and anomaly area annotation, ultimately outputting an audit-annotated image. For example, on a VAT invoice with a horizontal crease in the middle, individual characters in the amount field near the crease can be correctly identified, but there is a local error in the correction mapping of the crease position, causing a slight drift in the left and right adjacent positions of two adjacent numbers in the amount string. The system will generate a semi-transparent red-filled outline of the anomaly area at the corresponding position, connect the adjacent misaligned character pairs with a yellow solid line, and annotate the upper right corner of the area with a numerical index. Auditors can directly locate the suspicious points based on the annotation and focus on verifying the character content and field attribution of the area. The annotated image output by the system can be directly saved into the audit file, and the location and index information of the anomaly area can be synchronously generated into a list of suspicious points to support subsequent audit review work.

[0053] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A smart auditing system for financial reimbursement documents based on visual character recognition, characterized in that: include: The page normalization module is used to acquire the original document image, and after document boundary segmentation, projection normalization, resampling and grayscale conversion, generate a page grayscale image and a document region mask. The illumination decoupling module is used to calculate the average gray level of the region based on the document region mask and generate an illumination-normalized image using a normalized Gaussian illumination kernel. The dense correction module is used to extract the text line skeleton from the illumination normalized image, form the text tangential angle field, text normal vector and basic correction mapping, and obtain the basic correction image, basic character box, character center, character height, text line set and text curvature through resampling, character detection and text line clustering. The symmetric perturbation module is used to generate a positive perturbation image and a negative perturbation image based on the basic correction mapping, median character height, text normal vector, and text curvature. It matches the basic character box in the two perturbation images and extracts the positive character center, negative character center, positive character height, and negative character height. The topology calculation module is used to establish a fixed character neighborhood based on the character center, calculate the forward adjacency distance, reverse adjacency distance, forward adjacency angle, and reverse adjacency angle, and generate topology coefficients. The anomaly localization module is used to perform character recognition on the basic corrected image and two perturbation images. Based on the topological coefficients, the relative order of characters, and the condition of identical content, it determines the topological anomaly character boxes and forms anomaly regions, anomaly region masks, and anomaly adjacent edge sets. The annotation module is used to reverse map abnormal regions, abnormal region masks, and abnormal adjacent edge sets to the original document image, and overlay region outlines, character lines, character content, and region indexes to generate an audit annotation image.

2. The intelligent auditing system for financial reimbursement documents based on visual character recognition according to claim 1, characterized in that, The page unification module includes: Collect color financial reimbursement documents to form original document images; perform document boundary segmentation on the original document images to generate document region masks for calculating the average grayscale of the regions and extract the main character points; A projection normalization transformation is established based on the main character point and the rectangular correction canvas direction; resampling and grayscale conversion are performed according to the projection normalization transformation to obtain a page grayscale image for lighting decoupling.

3. The intelligent auditing system for financial reimbursement documents based on visual character recognition according to claim 1, characterized in that, The illumination decoupling module includes: Accumulate pixel grayscale values ​​and count the number of pixels within the page defined by the document region mask, and calculate the average grayscale value of the region; set the normalized Gaussian illumination kernel and Gaussian kernel scale, and perform convolution on the page grayscale image to obtain the slowly varying illumination components; The ratio is calculated by multiplying the average gray level of the region by the gray level of the page, and by using the sum of the gradually varying illumination component and the gray level stability term as the denominator; a normalized illumination image is then formed based on the ratio of each pixel.

4. The intelligent auditing system for financial reimbursement documents based on visual character recognition according to claim 1, characterized in that, The dense correction module includes: Detect document boundaries and text line skeletons in a normalized illumination image, and generate text tangential angle field and text normal vector perpendicular to the text tangential direction based on the text line skeleton; Set control points in the rectangular correction canvas, and generate a basic correction map based on the control point displacement and interpolation weights. The sum of the interpolation weights for the same pixel is 1. Bilinear resampling is performed on the illumination-normalized image according to the baseline correction map to generate the baseline correction image. Base character boxes are detected and text line clustering is performed to obtain character centers, character heights and text line sets.

5. The intelligent auditing system for financial reimbursement documents based on visual character recognition according to claim 1, characterized in that, The dense correction module also includes: The median character height is generated based on the median of all character heights; Transform the text tangential angle field and text normal vector to a rectangular correction canvas, and generate text curvature based on the median character height and the rate of change of the text tangential angle field along the arc length of the text line.

6. The intelligent auditing system for financial reimbursement documents based on visual character recognition according to claim 1, characterized in that, The symmetric perturbation module includes: Set the perturbation ratio to be greater than 0 and less than 1; Along the two opposite directions of the text normal vector, the product of the median character height, text curvature, and perturbation ratio is superimposed onto the basic correction map to generate the forward perturbation map and the reverse perturbation map. Bilinear resampling is performed on the illumination-normalized image based on the forward perturbation mapping and the reverse perturbation mapping, respectively, to generate the forward perturbation image and the reverse perturbation image; The same character detector as the base corrected image is used in the forward perturbation image and the reverse perturbation image to obtain the forward character box, the reverse character box, the forward character center, the reverse character center, the forward character height, and the reverse character height; The character matching result is determined based on the maximum positive overlap between the base character box and the forward and reverse character boxes, and characters without positive overlap are marked as unmatched characters.

7. The intelligent auditing system for financial reimbursement documents based on visual character recognition according to claim 1, characterized in that, The topology calculation module includes: For each character center, select the number of neighboring characters with the nearest Euclidean distance from the remaining character centers to establish a fixed character neighborhood. The number of neighboring characters is an integer not less than 2. The forward adjacency distance is generated based on the Euclidean distance between the forward character centers, the average forward character height, and the stable character height term. The reverse adjacency distance is generated based on the Euclidean distance between the reverse character centers, the average reverse character height, and the stable character height term. Generate forward adjacent angles based on the difference in the x and y coordinates of the forward character centers, and generate reverse adjacent angles based on the difference in the x and y coordinates of the reverse character centers; The distance term is the absolute value of the difference between the forward and reverse adjacency distances, and the direction term is the product of the result of subtracting the cosine of the difference between the forward and reverse adjacency angles from 1 and the adjacency angle weight. The sum of the distance and direction terms is averaged over the neighborhood to obtain the topological change. Perform an exponential operation on the negative of the topological change to generate topological coefficients, and set the topological coefficient of unmatched characters to 0.

8. The intelligent auditing system for financial reimbursement documents based on visual character recognition according to claim 1, characterized in that, The anomaly localization module determines the topology anomaly character box, including: The same character recognizer is used in the base corrected image, the forward perturbation image and the reverse perturbation image to generate base character content, forward character content and reverse character content corresponding to the base character box, the forward character box and the reverse character box respectively, for the condition of content being the same; When the topology coefficient is less than the topology coefficient threshold, the basic character box is recorded as a topology abnormal character box. The topology coefficient threshold is greater than 0 and less than 1. When a character pair within a fixed character neighborhood undergoes a horizontal or vertical relative order flip between the forward and reverse character centers, the base character frame of the character pair is denoted as a topologically abnormal character frame.

9. The intelligent auditing system for financial reimbursement documents based on visual character recognition according to claim 1, characterized in that, The anomaly localization module forms an anomaly region, an anomaly region mask, and an anomaly adjacent edge set, including: Topological anomaly character boxes are retained only when the basic character content, forward character content, and reverse character content are the same. Overlapping or boundary-connected topological anomaly character boxes are merged according to the text line set to form non-overlapping anomaly regions. An abnormal region mask is generated based on the abnormal region, and the character pairs whose relative order is reversed are saved to form an abnormal adjacent edge set.

10. The intelligent auditing system for financial reimbursement documents based on visual character recognition according to claim 1, characterized in that, The annotation module includes: The coordinates of the rectangular correction canvas of the abnormal region, the abnormal region mask, and the abnormal adjacent edge set are converted into the coordinates of the original document image by the inverse transformation of the basic correction mapping and the projection normalization transformation. Perform connected boundary tracing, contour vectorization, and closed contour rasterization on abnormal regions to generate region contours; For each character pair in the abnormal adjacent edge set, draw a solid line connecting the centers of the two characters to form a character connection; write the basic character content and region index into the overlay layer; The overlay, consisting of the region outline, character lines, basic character content, and region index, is synthesized pixel by pixel with the original document image to output the audit annotation image.

Images