A system for automated processing of financial information using image recognition techniques

Abstract

The application belongs to the field of financial data processing, and discloses a system for automatically processing financial information by using image recognition technology, which comprises a greyscale unit, a noise reduction unit and a binarization unit; the greyscale unit is used for performing greyscale processing on the image of a financial bill to obtain a first image; the noise reduction unit is used for performing noise reduction on the first image to obtain a second image; and the binarization unit is used for performing binarization processing on the second image to obtain a binarized image, including: calculating the binarization threshold value of each pixel point in the second image respectively; and determining the grey value of the pixel point in the second image in the binarized image based on the binarization threshold value. The binarization threshold value calculated by the application can be applied to financial bills of different paper colors, and can reduce the probability of the occurrence of the situation that the characters in the binarized image are incomplete due to uneven illumination, so that more accurate recognition of the image of the financial bill is realized, and the accuracy of the recognized financial information is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of financial data processing, and in particular to a system for automatically processing financial information using image recognition technology. BACKGROUND

[0002] It is a common financial informatization technology to obtain the recorded financial information of paper financial bills such as invoices through image recognition technology. After obtaining the image of the financial bill, it is usually necessary to perform binaryzation processing on the image to obtain a binaryzation image, and then perform OCR recognition on the binaryzation image to obtain the text information in the image. In the prior art, when performing binaryzation processing on the image of the financial bill, it is usually processed based on a preset threshold value or based on a global threshold value, but the preset threshold value cannot be applied to financial bills with different paper colors, and the global threshold value has a high probability of causing some pixel points belonging to text to be incorrectly divided into pixel points belonging to the background when the light is uneven, which results in incomplete text in the binaryzation image, affecting the accuracy of the final recognized financial information. SUMMARY

[0003] The purpose of the present application is to disclose a system for automatically processing financial information using image recognition technology, which solves the technical problems raised in the background art.

[0004] In order to achieve the above purpose, the present application provides the following technical scheme:

[0005] The present application provides a system for automatically processing financial information using image recognition technology, comprising an image preprocessing module, the image preprocessing module comprising a grayscale unit, a noise reduction unit and a binaryzation unit;

[0006] The grayscale unit is used to perform grayscale processing on the image of the financial bill to obtain a first image;

[0007] The noise reduction unit is used to perform noise reduction on the first image to obtain a second image;

[0008] The binaryzation unit is used to perform binaryzation processing on the second image to obtain a binaryzation image, comprising:

[0009] The binaryzation threshold value of each pixel point in the second image is calculated respectively;

[0010] The gray value of the pixel point in the second image in the binaryzation image is determined based on the binaryzation threshold value;

[0011] The binaryzation threshold value of each pixel point in the second image is calculated respectively, comprising:

[0012] First, the i-th pixel point p in the second image is obtained i, i ∈ [1, N], N is the total number of pixel points in the second image;

[0013] Secondly, calculating p i The corresponding binary threshold thre i If the gray value of p i is greater than thre i , p i is saved to the set F.

[0014] Thirdly, judging whether i is less than N, if yes, the value of i is added by 1, and the first step is returned, if no, the calculation is stopped.

[0015] The calculation formula of thre i is as follows:

[0016] thre i = k × To + (1-k) × Tp

[0017] K represents an adaptive weight, To represents a global threshold corresponding to the second image, and Tp represents an adjusted threshold calculated based on p i .

[0018] Preferably, the image preprocessing module further comprises a correction unit.

[0019] The correction unit is configured to perform distortion correction on the binary image to obtain a corrected image.

[0020] Preferably, the image preprocessing module further comprises a text region detection module.

[0021] The text region detection module is configured to identify the corrected image to obtain a region containing text in the corrected image.

[0022] Preferably, the image preprocessing module further comprises a character recognition module.

[0023] The character recognition module is configured to identify the region containing text to obtain text in the region containing text.

[0024] Preferably, the image preprocessing module further comprises a content filling module.

[0025] The content filling module is configured to fill the obtained text into a preset data template.

[0026] Preferably, the image preprocessing module further comprises an image acquisition module.

[0027] The image acquisition module is configured to capture the financial bill to obtain an image of the financial bill.

[0028] Preferably, the first image is denoised to obtain the second image, comprising:

[0029] The first image is denoised using a median filtering algorithm or a bilateral filtering algorithm to obtain a second image.

[0030] Preferably, determining the gray value of the pixel point cp

[0031] The size of the binary image is the same as that of the second image, and the gray value of the pixel point cp j,m , j ∈ [1, J], m ∈ [1, M], J and M are the number of rows and columns of the pixel points in the binary image, respectively, and cp j,m The determination process of the gray value is as follows:

[0032] Obtaining the pixel point bp j,m in the jth row and the mth column in the second image;

[0033] If the gray value of bp j,m is greater than the corresponding binary threshold value of bp j,m , the gray value of cp j,m is 255, otherwise the gray value of cp j,m is 0.

[0034] Preferably, the obtaining process of Tp includes:

[0035] In the second image, the pixel points with p i as the center and the distance between p i and p u is less than the adaptive distance S are stored in the set NB.

[0036] Tp is calculated using the following formula:

[0037]

[0038] d u is the distance between the pixel point u and p i , FL u represents the gradient value of the pixel point u, FL i represents the gradient value of p i , FLm represents the maximum value of the gradient values of all pixel points in NB, G u represents the gray value of the pixel point u, st represents the standardization processing of the variables in the parentheses, and λ is the threshold calculation weight.

[0039] Preferably, the calculation formula of the adaptive weight is:

[0040]

[0041] SU represents the set of pixel points in the second image, NumSU represents the number of pixel points in the second image, G vGray value of a pixel point v in the SU, aveGSU represents the average of the gray values of the pixel points in the SU, aveGu represents the average of the gray values of the pixel points in the NB, and NumNB represents the number of the pixel points in the NB.

[0042] Advantages:

[0043] In the process of identifying the image of the financial bill, the system not only performs binarization processing based on a global threshold value, but also considers an adjusted threshold value calculated based on pixel points in the binarization process. Thus, the binarization threshold value calculated by the system can be applicable to financial bills of different paper colors, and the adjusted threshold value calculated based on the gray distribution of the position of the pixel points can reduce the probability of incomplete characters in the binarized image caused by uneven illumination, so that the image of the financial bill can be more accurately identified, and the accuracy of the identified financial information is improved. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0045] Figure 1 FIG. 1 is a schematic diagram of a system for automatically processing financial information by using image recognition technology according to the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0047] The present application provides a system for automatically processing financial information by using image recognition technology, which comprises an image preprocessing module, the image preprocessing module comprising a gray scale unit, a noise reduction unit and a binarization unit.

[0048] The gray scale unit is used for performing gray scale processing on the image of the financial bill to obtain a first image.

[0049] The noise reduction unit is used for performing noise reduction on the first image to obtain a second image.

[0050] The binarization unit is used to perform binarization processing on the second image to obtain a binary image, including:

[0051] Calculate the binarization threshold for each pixel in the second image;

[0052] The grayscale value of the pixel in the second image is determined based on the binarization threshold.

[0053] Calculate the binarization threshold for each pixel in the second image, including:

[0054] The first step is to obtain the i-th pixel p in the second image. i , i∈[1,N], where N is the total number of pixels in the second image;

[0055] The second step is to calculate p. i The corresponding binarization threshold thre i If p i The grayscale value is greater than the threshold. i Then p i Save to collection F;

[0056] The third step is to determine whether i is less than N. If yes, increment the value of i by 1 and go back to the first step. If no, stop the calculation.

[0057] thre i The calculation formula is:

[0058] thre i = k×To+(1-k)×Tp

[0059] k represents the adaptive weight, To represents the global threshold corresponding to the second image, and Tp represents the threshold based on p. i The calculated adjustment threshold.

[0060] In the aforementioned binarization process, an adjustment threshold calculated based on pixels is also considered. Thus, the binarization threshold calculated by this invention is not only applicable to financial documents of different paper colors, but also, because the adjustment threshold is calculated based on the grayscale distribution of the pixel's location, it can reduce the probability of incomplete text in the binarized image when the lighting is uneven, thereby enabling more accurate recognition of financial document images.

[0061] The financial documents of this invention include invoices, receipts, checks, delivery notes, etc.

[0062] In this invention, the global threshold can be a threshold calculated using the maximum inter-class variance method on the second image.

[0063] If the lighting conditions of the image are not uniform, a global threshold may not be able to adapt to the brightness differences in different regions. For example, one part of the image can be brighter, while another part is darker. Using a single threshold can cause the bright areas or dark areas to be incorrectly binarized.

[0064] For example: a picture of black text on a white background, the brightness of the background will affect the selection of the threshold. When using a global threshold, some dark background parts may be mistakenly considered as foreground, or some bright foreground areas may be mistakenly considered as background.

[0065] Therefore, the present application improves the existing threshold calculation method, and adds an adjustment threshold, which is calculated based on the gray value situation around the pixel point. Therefore, even if the brightness of the image as a whole is not uniform, the accuracy of the calculated threshold can be improved because the gray value situation around the pixel point is added in the threshold calculation process.

[0066] Preferably, the image preprocessing module further comprises a correction unit;

[0067] The correction unit is used for distortion correction of the binarized image to obtain a corrected image.

[0068] In the process of invoice image recognition to extract invoice information, distortion correction is a very important step, especially in the image acquisition process, image distortion may occur due to shooting angle, camera lens distortion or image deformation.

[0069] The necessity of distortion correction includes:

[0070] Ensure accuracy: The information in the invoice image usually needs to extract specific text, barcodes or other identifiers. If there is distortion in the image, these information may be distorted, affecting the accuracy of the recognition system.

[0071] Alignment problem: When the shooting angle of the invoice image is not correct or the document is not completely parallel to the camera, it will cause perspective distortion between parts of the image. Distortion correction can help restore the correct proportion of the image, so that the text and graphics in the image can be correctly aligned, which is convenient for subsequent optical character recognition (OCR) processing.

[0072] Improve OCR recognition rate: OCR (Optical Character Recognition) technology works best when dealing with clear, distortion-free text. Distortion can cause difficulties in character recognition, so by correcting the image to make it more "flat", the recognition accuracy of the OCR system can be significantly improved.

[0073] Common distortion correction methods:

[0074] Perspective Transformation: If the image is taken at an angle, perspective transformation can map the four corners of the image to an orthographic view, thus "flattening" the image.

[0075] Lens Distortion Correction: Distortion caused by the lens (e.g. barrel distortion or pincushion distortion) can be corrected by specific algorithms, common methods include distortion correction based on camera calibration.

[0076] Image Preprocessing: In some cases, preprocessing of the image (e.g. contrast enhancement, edge detection, etc.) can also help alleviate the problems caused by distortion.

[0077] Preferably, it further comprises a text region detection module;

[0078] The text region detection module is used to identify the corrected image and obtain the region containing text in the corrected image.

[0079] Image segmentation techniques (e.g. connected component analysis) can be used to extract regions that may contain text, and deep learning methods (e.g. convolutional neural networks, CNN) can also be used for region detection to automatically identify text regions on the bill.

[0080] Preferably, it further comprises a text recognition module;

[0081] The text recognition module is used to identify the region containing text and obtain the text in the region containing text.

[0082] Specifically, OCR technology (e.g. Tesseract, Google Vision API, Microsoft OCR) can be used to recognize text in the image. OCR technology can recognize characters in the text region and convert them into machine-readable text.

[0083] Preferably, it further comprises a content filling module;

[0084] The content filling module is used to fill the obtained text into the preset data template.

[0085] Specifically, different bills generally have fixed formats, so corresponding data templates can be set for different types of bills in advance. After obtaining the region containing text, the position of the region can be used to determine the data item corresponding to the text in the region, so that the text in the region can be filled into the data template. There are multiple data items in the data template, for example, for an invoice data template, there are invoice number, invoice date, amount, etc. For the invoice date, it is generally located in the upper right corner of the invoice. Of course, during image recognition, coordinates can be used to more accurately locate the region where the text of the invoice date is located.

[0086] Preferably, the image acquisition module is further included.

[0087] The image acquisition module is configured to capture the financial bill to obtain an image of the financial bill.

[0088] The financial bill can be captured by a scanner or a camera to obtain the corresponding image.

[0089] Preferably, the first image is denoised to obtain a second image, comprising:

[0090] The first image is denoised using a median filter algorithm or a bilateral filter algorithm to obtain the second image.

[0091] The median filter replaces each pixel value in the image with the median of its neighborhood pixel values, thereby removing extreme values (i.e., noise points). This method preserves the edge information of the image, which is very helpful for text recognition.

[0092] The bilateral filter smooths the image by considering both the distance of the spatial neighborhood and the similarity of the pixel values. It can remove noise in the image while maintaining the clarity of the edges.

[0093] Preferably, the gray value of the pixel point cp

[0094] The size of the binary image is the same as that of the second image. For the pixel point cp j,m , j ∈ [1, J], m ∈ [1, M], J and M are the number of rows and columns of pixel points in the binary image, respectively, and cp j,m The determination process of the gray value is as follows:

[0095] Obtain the pixel point bp j,m in the jth row and mth column of the second image.

[0096] If the gray value of bp j,m is greater than the corresponding binary threshold value of bp j,m , the gray value of cp j,m is 255, otherwise the gray value of cp j,m is 0.

[0097] By thresholding, the pixel points in the text region and the pixel points in the non-text region in the second image can exhibit significantly different gray values in the binary image, which can improve the speed of subsequent text recognition.

[0098] Preferably, the process of obtaining Tpcomprises:

[0099] In the second image, the pixel point p idistance between p i and p u is less than adaptive distance S, the pixel point is stored in set NB;

[0100] Tp is calculated using the following formula:

[0101]

[0102] d u distance between p i and p u , FL i represents the gradient value of p i , FLm represents the maximum value of the gradient values of all pixel points in NB, G u represents the gray value of pixel point u, st represents the standardization processing of the variables in the parentheses, and λ is a threshold calculation weight.

[0103] Tp of the present application is calculated based on the pixel points whose distance from p i is less than adaptive distance, and the adaptive distance can be adaptively changed with the change of the contrast of the second image, and effective trade-off can be made between the calculation efficiency of Tp and the effectiveness of Tp. If the value of S is too large, too many pixel points will participate in the calculation process of Tp, affecting the calculation efficiency of Tp, and if the value of S is too small, the number of pixel points participating in the calculation process of Tp may be insufficient when the contrast of the second image is low, so that the calculated binary threshold is greatly affected by the global threshold, increasing the probability of incorrectly classifying pixel points belonging to text as pixel points belonging to background.

[0104] The influence of the pixel points in NB of the present application on the result of Tp changes with the change of d u , the absolute value of the difference between FL u and FL i , the smaller d u is, the greater the absolute value of the difference between FL u and FL i , and the greater the influence of pixel point u on the result of Tp, so that Tp can effectively represent the gray distribution around p i from two angles, so that the calculated binary threshold can accurately take into account the gray distribution around p i , improving the adaptability of the calculated binary threshold.

[0105] The standardization processing of the present application can be based on the method of minimum-maximum standardization.

[0106] The value range of the threshold calculation weight of the application can be [0.3, 0.7], and further, the threshold calculation weight of the application can be 0.5.

[0107] Preferably, the calculation formula of the adaptive weight is:

[0108]

[0109] SU represents a set of pixel points in the second image, NumSU represents the number of pixel points in the second image, G v represents the gray value of the pixel point v in SU, aveGSU represents the mean value of the gray values of the pixel points in SU, aveGu represents the mean value of the gray values of the pixel points in NB, and NumNB represents the number of pixel points in NB.

[0110] The adaptive weight can change with the change of the contrast of the pixel points in SU, the greater the contrast of the pixel points in SU, that is, the greater the molecular part of the application, the greater the adaptive weight, and the greater the influence of To on the calculation result in the process of calculating the binary threshold, so that the influence of Tp on the result of the binary threshold is greater when the contrast of the pixel points in SU is smaller and the binary accuracy of the global threshold is lower. Therefore, the adaptive weight can effectively adaptively control the calculation result of the binary threshold based on the contrast of the second image, so that the calculated binary threshold is adapted to the actual contrast condition, and a more accurate binary result is obtained.

[0111] Preferably, the calculation formula of the adaptive distance S is:

[0112]

[0113] NF represents the number of pixel points belonging to F contained in a square region with a side length of 5 centered on p i , mr is a preset distance, and η is a distance determination weight.

[0114] The adaptive distance of the application is weighted calculated based on the number of pixel points belonging to F in the region with a specified size and the contrast of the pixel points in SU, the greater the contrast of the pixel points in SU, the greater the value of NF, and the smaller the value of S, so that the accuracy of the binary result obtained based on the global threshold is higher, the more pixel points belonging to the text are contained in the positive direction region, the smaller the value of S is, and the smaller the pixel points participating in the calculation process of Tp is. Not only can the efficiency of obtaining the binary threshold be improved, but also the text edge can be further strengthened; the smaller the contrast of the pixel points in SU, the smaller the value of NF, and the more pixel points are used to calculate Tp to obtain a more accurate result.

[0115] The preset distance of the present application can be 10. The distance determining weight can be 0.6.

[0116] The preferred embodiments of the application disclosed above are only used to illustrate the present application. The preferred embodiments do not describe all of the details and limit the present application to the specific embodiments described. Obviously, many modifications and variations can be made in light of the contents of the specification. The embodiments are chosen and described in order to best explain the principles of the application and its practical application, so that others skilled in the art can better understand and utilize the application. The application is only limited by the claims and their full scope and equivalents.

Claims

1. A system for automated processing of financial information using image recognition technology, characterized by, The image preprocessing module comprises a greying unit, a noise reduction unit and a binarization unit. The greying unit is configured to perform greying processing on the image of the financial bill to obtain a first image. The noise reduction unit is configured to perform noise reduction on the first image to obtain a second image. The binarization unit is configured to perform binarization processing on the second image to obtain a binarization image, comprising: calculating a binarization threshold of each pixel point in the second image respectively; determining a gray value of the pixel point in the binarization image based on the binarization threshold; calculating a binarization threshold of each pixel point in the second image respectively, comprising: In a first step, a first image is obtained i , i∈[1,N], N is the total number of pixels in the second image; Second step, calculate p i Corresponding binary threshold thre i , if the gray value of p i is greater than thre i , save p i to the set F; In the third step, it is judged whether i is less than N, if yes, the value of i is added by 1, and the first step is returned, if not, the calculation is stopped. thre i The calculation formula is: thre i = k x To + (1 - k) x Tp k represents an adaptive weight, To represents a global threshold value corresponding to the second image, Tp represents a local threshold value based on p i the computed adjusted threshold value; The obtaining process of Tp comprises: In the second image, the pixel points centered at p i with the distance between p i smaller than the adaptive distance S are stored in the set NB; Tp is calculated by using the following formula: d u For the distance between the pixel points u and p, FL i u Indicates the gradient value of the pixel point u, FL i Indicates the gradient value of p i , FLm indicates the maximum value of the gradient values of all pixel points in the NB, G u Indicates the gray value of the pixel point u, st indicates the standardization processing of the variables in the parentheses, and λ is the threshold calculation weight;​ The calculation formula of the adaptive weight is: SU denotes a set of pixel points in the second image, NumSU denotes a number of pixel points in the second image, G v denotes a gray value of a pixel point v in the SU, aveGSU denotes a mean value of the gray values of the pixel points in the SU, aveGu denotes a mean value of the gray values of the pixel points in the NB, and NumNB denotes a number of pixel points in the NB; The calculation formula of the adaptive distance S is: NF represents the number of pixel points belonging to F contained in a square region with a side length of 5 centered on p i mr is a preset distance, and η is a distance determination weight.

2. The system for automatic processing of financial information using image recognition technology according to claim 1, wherein, The image preprocessing module further comprises a correction unit. The correction unit is configured to perform distortion correction on the binarization image to obtain a corrected image.

3. The system for automatic processing of financial information using image recognition technology according to claim 2, wherein, Further comprising a text region detection module; The text region detection module is configured to recognize the corrected image to obtain a region containing text in the corrected image.

4. The system for automatic processing of financial information using image recognition technology according to claim 3, wherein, Further comprising a character recognition module; The character recognition module is configured to recognize the region containing text to obtain the text in the region containing text.

5. The system for automated processing of financial information using image recognition technology as claimed in claim 4, wherein, Further comprising a content filling module; The content filling module is configured to fill the obtained text into a preset data template.

6. The system for automatic processing of financial information using image recognition technology according to claim 1, wherein, Further comprising an image acquisition module; The image acquisition module is configured to capture the financial bill to obtain the image of the financial bill.

7. The system for automated processing of financial information using image recognition technology as claimed in claim 1 wherein, The noise reduction on the first image to obtain the second image comprises: The noise reduction on the first image to obtain the second image comprises:

8. The system for automatic processing of financial information using image recognition technology according to claim 1, wherein, The determination of the gray value of the pixel point in the binarization image based on the binarization threshold comprises: The size of the binary image is the same as the size of the second image, and for a pixel point cp j,m , j ∈ [1, J], m ∈ [1, M], J and M are respectively the number of rows and columns of pixel points in the binary image, cp j,m The determination process of the gray value is as follows: acquiring a pixel point bp of a jth row and an mth column in the second image j,m ; If bp j,m The grayscale value is greater than bp j,m The corresponding binarization threshold, then cp j,m The grayscale value is 255, otherwise cp j,m The grayscale value is 0.

Images