High-precision analysis system and method of invoice amount based on OCR

Abstract

The present invention discloses a high-precision analysis system for invoice amounts based on OCR, including a system terminal and a server; the system terminal provides users with an image scanner for reading the invoices required for analysis; the server stores a pre-trained network model and a complete set of system source codes; the network model includes image classification, OCR text recognition CRNN unit and image feature matching unit; the image classification adopts the MobileNet V3 classification network; the image feature matching unit adopts the SIFT algorithm to perform feature matching, positioning and extraction; the OCR text recognition CRNN unit identifies the required information; the complete set of system source codes realizes the integration of the network model and the analysis visualization interface. The image processing module of the present invention realizes the automatic classification of multiple bills by combining the use of the MobileNet V3 classification network and the SIFT algorithm for feature matching, positioning and extraction, and performs targeted positioning of the modules that need to be identified, thereby reducing unnecessary processes, saving time, and improving efficiency and recognition accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of data collection and processing, and in particular to a high-precision analysis system and method for invoice amounts based on OCR. Background Art

[0002] The adaptive learning-driven approach of deep learning can deal with these problems well, because the parameter preprocessing process is basically much less, and it basically achieves end-to-end processing. The OCR method based on deep learning divides some complicated processes into two main steps, one is text detection (mainly used to locate the position of the text), and the other is text recognition (mainly used to identify the specific content of the text). Although the OCR method based on deep learning has become a very important research direction in the field of computer vision, and the application fields involved are also diverse, the problem of scene text recognition based on Chinese still needs to be continuously improved.

[0003] Traditional OCR processing methods use a lot of image processing knowledge: first, data preprocessing, then data feature extraction, feature dimension reduction, etc., and then send it to some typical classifiers (such as SVM, Softmax) for classification, and finally post-processing for data structuring. Although this is basically in line with human visual processing logic, the entire processing flow has too many steps and is serial, resulting in errors being continuously transmitted and amplified. Secondly, the entire process involves too much manual design, and manual design may not necessarily grasp the essence of the problem. And despite exhausting various methods, there are still many problems that cannot be handled at all. How to improve the accuracy and precision of recognition has become a problem that must be solved. Summary of the invention

[0004] The purpose of the present invention is to provide an OCR-based invoice amount high-precision analysis system and method with simple algorithm, high accuracy, fast use speed and time saving in view of the above-mentioned shortcomings.

[0005] The present invention is implemented as follows: a high-precision analysis system for invoice amounts based on OCR, characterized in that the system includes a system terminal and a server; the system terminal provides users with an image scanner that automatically reads the invoices required for analysis; the server stores a pre-trained network model and a complete set of system source code; the system terminal and the server can be connected for communication via a public communication network;

[0006] The network model includes image classification, OCR text recognition CRNN unit and image feature matching unit; the image classification adopts MobileNet V3 classification network to classify the images of different invoices into corresponding templates;

[0007] The image feature matching unit adopts SIFT algorithm to perform feature matching, positioning and extraction;

[0008] The OCR character recognition CRNN unit recognizes the required information;

[0009] The whole system source code realizes the integration of image classification, image feature matching unit, OCR text recognition CRNN unit and generation of Excel tables and analysis visualization interface, making it a coherent whole.

[0010] The method for invoice analysis based on the OCR-based invoice amount high-precision analysis system is characterized by:

[0011] (1) Scan the invoice with a scanner, and upload the scanned invoice to the server as an invoice image;

[0012] (2) The server starts working and classifies the images using the MobileNet V3 classification network;

[0013] (3) According to the different invoice templates formed after classification, find the corresponding invoice template and use SIFT algorithm to locate the features;

[0014] (4) After successful positioning, intercept the valid invoice information, including: amount and related verification information;

[0015] (5) Use the text recognition CRNN unit to identify valid information;

[0016] (6) Import the identified data into a table; perform operations such as duplicate checking and statistics based on the table information;

[0017] (7) Generate a data analysis webpage and a downloadable Excel spreadsheet to which users can jump;

[0018] (8) When positioning in step (3) fails, try other classifications for positioning;

[0019] (9) After successfully trying other categories to locate, repeat steps (4), (5), (6), and (7);

[0020] (10) If other classifications are attempted and positioning fails, the image information is returned to the user.

[0021] Preferably, the OCR text recognition CRNN unit includes a data set generation module, an invoice recognition module and a data analysis and visualization module.

[0022] Preferably, the data set generation module adopts the text recognition image generation technology TRDG to generate more simulation data under the condition of limited data set, generate corresponding and different training sets for information of different parts, and perform image processing on the generated pictures.

[0023] Preferably, the invoice recognition module includes a deep learning character training framework, uses the data set obtained by the data set generation module to train the framework, constructs a CRNN model, and trains the network to converge for character recognition.

[0024] Preferably, the data visualization module uses the pyecharts tool to display the read invoice information in a visual manner. The pyecharts API interface is used to implement a visual analysis page for valuable invoice information.

[0025] Preferably, the image classification is achieved by training a specific network using a GPU.

[0026] Preferably, the SIFT algorithm is used to capture information required by the user as key points of a local image.

[0027] Beneficial effects of the present invention:

[0028] 1. By using the MobileNet V3 classification network and the SIFT algorithm for feature matching, positioning and extraction, the automatic classification of various bills can be achieved, and the modules that need to be identified can be targeted, reducing unnecessary processes, saving time, improving efficiency and improving recognition accuracy.

[0029] 2. Through text recognition image generation technology and the construction of CRNN model, it is possible to generate more simulation data with a limited data set, and combine with CRNN model to further improve the accuracy of character recognition.

[0030] 3. Use the big law to binarize all pictures, eliminate the interference of invoice pictures, and light-colored high-speed rail ticket patterns, so as to make the training more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flow chart of the present invention.

[0032] Figure 2 This is the architecture diagram of the biochemical module of the dataset.

[0033] Figure 3 This is the CRNN network architecture diagram.

[0034] Figure 4 A data visualization analysis chart. DETAILED DESCRIPTION

[0035] The present invention is further summarized below with reference to the accompanying drawings.

[0036] like Figure 1 As shown, the OCR-based high-precision invoice amount analysis system is characterized in that the system includes a system terminal and a server; the system terminal provides users with an image scanner that automatically reads the invoices required for analysis; the server stores a pre-trained network model and a complete set of system source code; the system terminal and the server can be communicatively connected via a public communication network.

[0037] Furthermore, the network model includes image classification, OCR text recognition CRNN unit and image feature matching unit; the image classification uses MobileNet V3 classification network to classify the images of different invoices into corresponding templates; it is realized by training the specific network through GPU; the image feature matching unit uses SIFT algorithm to perform feature matching positioning extraction; the information required by the user is intercepted as the key point of the local image. The OCR text recognition CRNN unit recognizes the required information; the whole system source code realizes the integration of image classification, image feature matching unit, OCR text recognition CRNN unit and the generation of excel tables and analysis visualization interface, making it a coherent whole.

[0038] Furthermore, the OCR text recognition CRNN unit includes a data set generation module, an invoice recognition module and a data analysis visualization module; the data set generation module adopts the text recognition image generation technology TRDG to generate more simulation data under a limited data set, generate corresponding and different training sets for information intercepted from different parts, and perform image processing on the generated pictures; the invoice recognition module includes a deep learning character training framework, uses the data set obtained by the data set generation module to train the framework, constructs a CRNN model, and trains the network to converge for character recognition; the data analysis visualization module adopts the pyecharts tool to display the read invoice information in a visual manner.

[0039] The method for invoice analysis based on the OCR-based invoice amount high-precision analysis system is characterized by:

[0040] (1) Scan the invoice with a scanner, and upload the scanned invoice to the server as an invoice image;

[0041] (2) The server starts working and classifies the images using the MobileNet V3 classification network;

[0042] (3) According to the different invoice templates formed after classification, find the corresponding invoice template and use SIFT algorithm to locate the features;

[0043] (4) After successful positioning, intercept the valid invoice information, including: amount and related verification information;

[0044] (5) Use the text recognition CRNN unit to identify valid information;

[0045] (6) Import the identified data into a table; perform operations such as duplicate checking and statistics based on the table information;

[0046] (7) Generate a data analysis webpage and a downloadable Excel spreadsheet to which users can jump;

[0047] (8) When positioning in step (3) fails, try other classifications for positioning;

[0048] (9) After successfully trying other categories to locate, repeat steps (4), (5), (6), and (7);

[0049] (10) If other classifications are unsuccessful, the image information is returned to the user.

[0050] Furthermore, after adopting the text recognition image generation technology TRDG, the dataset generation module constructs the training set method:

[0051] (1) Generate data with different inclinations and color depths using a data set generation algorithm;

[0052] (2) Randomly add top, bottom, left, and right blanks to the data image;

[0053] (3) A portion of the data is processed by template (accounting for about 1 / 4);

[0054] (4) Binarize all images using the big law;

[0055] (5) Obtain the training set used.

[0056] Working principle: Figure 1 As shown, the whole system is divided into a system terminal and a server. The system terminal provides users with an image scanner that automatically reads the required invoices for analysis; the server stores the pre-trained network model and the entire system source code; the system terminal and the server are connected through a public communication network.

[0057] The user uses the system terminal, such as the image scanner that automatically reads the invoices that need to be reimbursed for the user's elevator control, compresses the obtained batch of invoice photos, and finally selects the compressed file of the invoice image in the system submission interface and clicks upload.

[0058] The process of users obtaining invoice information uses the image classification, OCR text recognition CRNN unit and image feature matching unit in the network model; the image classification uses the MobileNet V3 classification network for classification, and it is packaged in zip format and uploaded to the server. The user interface will see a waiting page, and the system will first use the lightweight image classification network MobileNet V3 to classify the transmitted invoice pictures. MobileNet has the advantages of faster speed and smaller model; the basic unit of MobileNet is depthwise separable convolution. For depthwise separable convolution, we use depthwise convolution to convolve different input channels separately, and then use pointwise convolution to combine the above outputs. The overall effect is similar to a standard convolution, but it will greatly reduce the amount of calculation and model parameters. Assume that the input feature map size is Df×Df×M, and the output feature map size is Df×Df×N, where Df is the width and height of the feature map, which is assumed to be the same, and refers to the number of channels (channels or depth). It is also assumed here that the input and output feature map sizes (width and height) are the same. Then, for the standard convolution kernel Dk×Dk, the computational cost will be Dk×Dk×M×N×Df×Df, while the total computational cost of depthwiseseparable convolution is: Dk×Dk×M×Df×Df+M×N×Df×Df, which is compared as follows: Generally, N is relatively large. If a 3x3 convolution kernel is used, the depthwise separable convolution can reduce the amount of computation by about 9 times compared to the standard convolution.

[0059] Furthermore, the SIFT algorithm (Scale Invariant Feature Transform Matching Algorithm) is used to extract effective information from the corresponding template specified position of the classified image; compared with the text detection method, the network model such as CTPN, TextBoxes, etc. has better effect; the SIFT algorithm (Scale Invariant Feature Transform Matching Algorithm) is used, which has the following advantages: 1. It has a higher accuracy rate than text detection algorithms such as CTPN. 2. Compared with the text detection algorithm, SIFT does not need to filter the redundant information detected. 3. There is no need to manually or use other algorithms to correct the direction of the image.

[0060] Furthermore, by using the text recognition image generation technology TRDG (Text Recognition Data Generator) in the data set generation module, it is possible to generate more simulation data with a limited data set. At the same time, a targeted amount and number training set is generated (the generated image uses methods such as randomly adding blanks, Gaussian blur, blurring edges, and motion blurring to simulate images in real scenes), and cross-mixed training with the manually intercepted real training set; the image is binarized using the Otsu method and then sent to the CRNN network model for training, and the binarization eliminates many factors that interfere with the invoice image, and adjusts the network structure of CRNN to make it more suitable for number and amount recognition; the number of network layers of CRNN is reduced, and the Dropout layer is added to make it more suitable for number and amount recognition and increase the generalization ability of the network; thereby further improving the authenticity of the data. Our approach has the following advantages:

[0061] 1. Training different CRNN models according to different image conditions (such as character set differences, font differences) greatly improves the accuracy of character recognition.

[0062] 2. Optimized the pattern problem in the background of high-speed rail tickets.

[0063] 3. Real invoice images are difficult to collect, and even the data on the Internet are mostly blurry, and key information is sometimes blocked by mosaics. More training sets are generated, and the tedious manual labeling process is omitted.

[0064] Furthermore, the deep learning character training framework in the invoice recognition module trains the data set obtained by the data set generation module, builds a CRNN model, and uses GPU training to quickly converge the network for character recognition. This model has the following features: there is no need to explicitly add the text cutting link, but instead converts text recognition into a sequence learning problem. Although the input image scales are different and the text lengths are different, after passing through CNN and RNN, after a certain translation in the output stage (the CTC algorithm is used to convert the sequence feature information learned by the network into the final recognition result, which is more suitable for Chinese text recognition than the attention mechanism, and can make the model more robust), the entire text image can be recognized. In other words, text cutting is integrated into deep learning. Therefore, end-to-end text recognition is achieved. Figure 3As shown in the figure, the CRNN network architecture consists of three parts, including convolutional layer, recurrent layer and transcription layer, from bottom to top. The convolutional layer uses CNN to automatically extract feature sequences from each input image; the recurrent layer uses RNN to predict the label (true value) distribution of the feature sequence obtained from the convolutional layer; the transcription layer uses CTC to convert the label distribution obtained from the recurrent layer into the final recognition result through operations such as de-reintegration; although CRNN consists of different types of network architectures (such as CNN and RNN), it can be jointly trained through a loss function.

[0065] Further, if Figure 4 As shown in the figure, the data visualization module writes the recognized information into an Excel table in sequence, adjusts and adds decimal points at the corresponding positions according to the fixed positions of the decimal points in the invoices of different templates, filters, integrates and analyzes the invoice information, and finally uses the pyecharts API interface of the pyecharts tool to display the read invoice information in the form of a visual analysis page.

[0066] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. High-precision invoice amount analysis system based on OCR, characterized by: The system includes a system terminal and a server; the system terminal provides users with an image scanner that automatically reads the invoices required for analysis; the server stores a pre-trained network model and a complete set of system source code; the system terminal and the server can be connected for communication via a public communication network; The network model includes image classification, OCR text recognition CRNN unit and image feature matching unit; the image classification adopts Classification network, classifies the images of different invoices into corresponding templates; The image feature matching unit adopts SIFT algorithm to perform feature matching, positioning and extraction; The OCR character recognition CRNN unit recognizes the required information; The whole system source code realizes the integration of image classification, image feature matching unit, OCR text recognition CRNN unit and generation of Excel tables and analysis visualization interface, making it a coherent whole.

2. The OCR-based invoice amount high-precision analysis system according to claim 1 is characterized in that: The OCR text recognition CRNN unit includes a data set generation module, an invoice recognition module and a data analysis and visualization module.

3. The OCR-based invoice amount high-precision analysis system according to claim 2 is characterized in that: The data set generation module adopts the text recognition image generation technology TRDG to generate more simulation data under the condition of limited data set, generate corresponding and different training sets for information of different parts, and perform image processing on the generated pictures.

4. The OCR-based invoice amount high-precision analysis system according to claim 2 is characterized in that: The invoice recognition module includes a deep learning character training framework, uses the data set obtained by the data set generation module to train the framework, builds a CRNN model, and trains the network to converge for character recognition.

5. The OCR-based invoice amount high-precision analysis system according to claim 2 is characterized in that: The data analysis visualization module uses the pyecharts tool to display the read invoice information in a visual manner.

6. The OCR-based invoice amount high-precision analysis system according to claim 1 is characterized in that: The image classification is achieved by training a specific network through a GPU.

7. The OCR-based invoice amount high-precision analysis system according to claim 1 is characterized in that: The SIFT algorithm is used to capture the information required by the user as the key points of the local image.

8. A method for invoice analysis using the OCR-based invoice amount high-precision analysis system according to any one of claims 1 to 7, characterized in that: (1) Scan the invoice with a scanner, and upload the scanned invoice to the server as an invoice image; (2) The server starts working, through The classification network classifies the images; (3) According to the different invoice templates formed after classification, find the corresponding invoice template and use the SIFT algorithm to locate the features; (4) After successful positioning, intercept the valid invoice information, including: amount and related verification information; (5) Use the text recognition CRNN unit to identify valid information; (6) Import the identified data into a table; perform operations such as duplicate checking and statistics based on the table information; (7) Generate a data analysis webpage and a downloadable Excel spreadsheet for users to jump to; (8) When positioning in step (3) fails, try other classifications for positioning; (9) After successfully trying other categories to locate the target, repeat steps (4), (5), (6), and (7); (10) If other classifications are attempted and positioning fails, the image information is returned to the user.

9. The method according to claim 8, characterized in that The text recognition CRNN unit recognizes valid information, including a data set generation module to construct a training set method: (1) Generate data with different inclinations and color depths using a data set generation algorithm; (2) Randomly add top, bottom, left, and right blanks to the data image; (3) Process part of the data into templates; (4) Binarize all images using the big law; (5) Obtain the training set used.

Images