Receipt information comparison and verification system based on OCR technology

Abstract

The invention discloses a receipt information comparison and verification system based on the OCR technology, and relates to the technical field of OCR, and the system specifically comprises a receipt information identification module, a receipt information comparison module and a receipt information verification module, the receipt information identification module is used for collecting the image of a receipt in real time, and the receipt information verification module is used for verifying the image of the receipt; the receipt information detection module is used for determining a text content detection area and an image content detection area of a receipt, identifying receipt information of the text content detection area by using a CTPN text detection algorithm through an OCR technology to obtain an identification result of the receipt information, and the receipt information comparison module is used for formatting the identification result and sending the formatted identification result to the server. And the check module is used for checking the identified and formatted information with the information in the bar code, and the receipt information check module is used for checking the receipt information of which the identification result is inconsistent with the bar code information, correcting and segmenting the image through binarization processing, inter-class variance calculation and a Hough transformation method, and checking the identification result by using a semantic analysis model.

Description

Technical Field

[0001] The present invention belongs to the technical field of OCR, and particularly relates to a receipt information comparison and verification system based on OCR technology. Background Art

[0002] With the rapid development of technology, OCR (Optical Character Recognition) technology has been widely used in various fields due to its efficient and accurate automatic recognition ability. Especially in business management that requires optical scanning and image processing, OCR technology has brought revolutionary changes to asset management and transaction information verification. With the increasing deepening of enterprise informatization construction, the scale of information systems continues to expand, and the accuracy and integrity of information in the commercial transaction process become increasingly crucial. Receipts, as important vouchers for commercial transactions, the accuracy of their information plays an irreplaceable role in protecting consumer rights and interests, maintaining merchant reputation, and promoting the smooth progress of commercial activities. However, most traditional receipt verification methods rely on manual operations, which are not only time-consuming and laborious with low efficiency, but also prone to errors due to human negligence or fatigue, and it is difficult to meet the urgent needs of modern commerce for high-efficiency and high-precision inspection.

[0003] In view of this, it is particularly important to design a receipt information comparison and verification system based on OCR (Optical Character Recognition) technology. OCR technology can automatically identify and extract text information in images and convert it into text data that can be processed by a computer, thereby realizing fast and accurate comparison and verification of receipt information. The introduction of this technology will greatly improve the automation level of receipt verification, reduce manual intervention, and reduce the error rate, providing an efficient and reliable solution path for the processing of commercial transaction information. Summary of the Invention

[0004] The purpose of the present invention is to provide a receipt information comparison and verification system based on OCR technology to solve the technical problems in the prior art that receipt verification relies on manual operations, is time-consuming and laborious, and is prone to errors.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions:

[0006] A receipt information comparison and verification system based on OCR technology, including: a receipt information recognition module, a receipt information comparison module, and a receipt information verification module;

[0007] The receipt information recognition module is used to collect the image of the receipt in real time, determine the text content detection area and the image content detection area of the receipt, and obtain the recognition result of the receipt information through OCR technology;

[0008] The receipt information comparison module is used to format the recognition result of the receipt information, and compare the recognized and formatted information with the information in the barcode; detect whether the content and format of the receipt recognition result are consistent with the information in the barcode. If they are consistent, the receipt information does not need to be verified. If they are inconsistent, the receipt information needs to be verified;

[0009] The receipt information verification module is used to verify the receipt information with inconsistent recognition results and barcode information. It segments the image through binarization processing, corrects the image recognition angle through the Hough transform method, and verifies the recognition result through a semantic analysis model to identify misspelled words or text content with chaotic structural order in the recognized text, thereby correcting the content of the receipt recognition result.

[0010] Furthermore, the method for the receipt information recognition module to collect the image of the receipt in real time and determine the text content detection area and the image content detection area of the receipt includes:

[0011] Collect the image of the receipt in real time through a high-definition camera device, obtain a receipt image that meets the actual requirements by adjusting the clarity and resolution of the image captured by the high-definition camera device, use a filtering algorithm to remove the noise in the filtered image set, and determine the content detection area of the receipt according to the format content of the receipt to be recognized. The receipt information in the receipt content detection area includes the receipt transaction content, the sales order number, and the barcode. Divide the receipt transaction content and the sales order number into the text content detection area, and divide the barcode into the image content detection area.

[0012] Furthermore, the method for the receipt information recognition module to obtain the recognition result of the receipt information through OCR technology is as follows:

[0013] Use the CTPN text detection algorithm in OCR technology to recognize the receipt information in the text content detection area, adopt the VGG16 network to extract the text features of the receipt, use the region detection algorithm to identify and extract the key information of the receipt, segment the detected text area into single characters based on the pre-determined character spacing, connectivity, and format features of the receipt text, use the trained character classification model to recognize the segmented characters, map them into machine-understandable text information, and transform to obtain the recognition result of the receipt information.

[0014] Furthermore, the method for the receipt information comparison module to compare the recognized and formatted information with the information in the barcode is as follows:

[0015] According to the layout and format of the receipt, format the recognition result. Detect the receipt barcode in the content detection area of the image through a barcode reader, extract the key information in the barcode, and check the key information in the formatted recognition result with the key information in the barcode to detect whether the content and format of the receipt recognition result are consistent with the information in the barcode. If they are consistent, the receipt information does not need to be verified; if they are inconsistent, the receipt information needs to be verified.

[0016] Furthermore, the method for the receipt information verification module to segment the image through binarization processing is as follows:

[0017] Perform binarization processing on the image for which the receipt information needs to be verified. Set the threshold M. According to M, divide the receipt image into two groups, calculate the gray value of each group of receipt images respectively, obtain the between-class variance of the gray values of the two groups of receipt images, adjust the size of the threshold M to change the size of the between-class variance, and thus determine the true value of the threshold M when the between-class variance is the largest. The specific steps include: determining the number of pixel points in the image, set as N, clarifying the range of gray values in the image, setting that there are m gray values, then the value range of the threshold M is from 0 to m - 1. Divide the gray image into two groups according to the M value, denoted as A0 and A1 respectively, where A0 contains pixel points with gray values in the range from 0 to M, and A1 contains pixel points in the range from M + 1 to m - 1. Using The calculation formula representing the between-class variance, h represents the between-class variance, i represents the gray value of i, R(i) represents the probability that the pixel point with gray value i appears, that is, the ratio relationship between the number of pixel points with gray value i and the total number of pixel points in the image, v1 represents the average gray value of the A1 group of gray images, v0 represents the average gray value of the A0 group of gray images, use the calculation formula of the between-class variance to determine the threshold M when the between-class variance is the largest, and segment the image through binarization processing according to the size of this threshold.

[0018] Furthermore, the method for the receipt information verification module to correct the image recognition angle through the Hough transform method is as follows:

[0019] A rectangular coordinate system is established with the center point of the image as the origin. The edge information in the image is highlighted through an edge detection algorithm, and the line segment between any row of text and the character edge points in the image is determined, denoted as the edge segment. This segment is represented by the formula y = kx + a, where b1 ≤ x ≤ b2. Here, y and x represent the variables of the edge line in the rectangular coordinate system, and k and a represent the parameters of the edge line. Among them, k is the slope, reflecting the inclination degree of the edge line; a is the intercept, determining the position of the line on the y-axis. The edge detection algorithm may detect multiple edge segments, and each segment has its own slope k and intercept a. The endpoint coordinates of the edge segment are determined by finding the true values of b1 and b2. Through the Hough transform, the positions of the edge line parameters and variables are swapped to obtain the parameter expression form a = -kx + y in the parameter space, where b1 ≤ x ≤ b2. According to the point-line duality, the intersection point (k0, b0) of two lines in the parameter space is mapped to a line in the variable space. The line in the original variable space is detected by calculating the intersection point in the parameter space. Here, k0 in the parameter space represents the slope of the line in the variable space. The inclination angle of this line can be obtained according to the slope and expressed as d = xcosθ + ysinθ, where b1 ≤ x ≤ b2. Here, d represents the distance from the line to the origin, and θ represents the inclination angle of the line. The image can be corrected for inclination by rotating this inclination angle in the reverse direction.

[0020] In summary, due to the adoption of the above technical solutions, the beneficial effects of the present invention are as follows:

[0021] The present invention corrects and segments the image through methods such as binarization processing, between-class variance calculation, and Hough transform to improve the format accuracy of the receipt recognition result. A semantic analysis model is used to verify the recognition result and correct the text content with misspelled words or chaotic structure order to ensure the accuracy of the receipt information after verification. The present invention improves the processing efficiency of receipt information and helps to enhance the accuracy and reliability of receipt recognition information. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following-described drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 It is a structural diagram of a receipt information comparison and verification system module based on OCR technology. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0025] As Figure 1 shown, a receipt information comparison and verification system based on OCR technology specifically includes a receipt information recognition module, a receipt information comparison module, and a receipt information verification module;

[0026] The receipt information recognition module: Real-time collects the image of the receipt through a high-definition camera device, obtains a receipt image that meets the actual requirements by adjusting the image clarity and resolution of the high-definition camera device, uses a filtering algorithm to remove the noise in the filtered image set, determines the content detection area of the receipt according to the format content of the receipt to be recognized. The receipt information in the receipt content detection area includes receipt transaction content, sales order number, and barcode. Among them, the receipt transaction content and the sales order number are divided into text content detection areas, and the barcode is divided into an image content detection area. The receipt information in the text content detection area is recognized by using the CTPN text detection algorithm through OCR technology, the text features of the receipt transaction content and the sales order number are extracted by using the VGG16 network, and the key information of the receipt transaction content and the sales order number, such as product name, price, quantity, date, is identified and extracted by using the region detection algorithm. Based on the pre-determined receipt text character spacing, connectivity, and format features, the detected text area is segmented into individual characters, and the segmented characters are recognized by using a trained character classification model and mapped into machine-understandable text information to obtain the recognition result of the receipt information.

[0027] The receipt information comparison module: formats the recognition result according to the layout and format of the receipt, performs grayscale value statistics on the image content detection area containing the barcode, obtains a threshold, and then performs binarization processing to convert the image into black and white. Then, the receipt barcode in the image content detection area is recognized by a barcode reader, and the key information in the barcode, including: product name, price, quantity, date, is extracted. The key information in the formatted recognition result is compared with the key information in the barcode to detect whether the content and format of the receipt recognition result are consistent with the information in the barcode. If they are consistent, the receipt information does not need to be verified; if they are inconsistent, the receipt information needs to be verified.

[0028] The receipt information verification module performs binarization on the image of the receipt information to be verified, sets a threshold M, divides the receipt image into two groups according to M, calculates the grayscale values of each group of receipt images respectively, obtains the between-class variance of the grayscale values of the two groups of receipt images, and adjusts the size of the threshold M to change the size of the between-class variance, so as to determine the true value of the threshold M when the between-class variance is the largest. The specific method is as follows:

[0029] Determine the number of pixel points in the image, assumed to be N, clarify the range of grayscale values in the image. Assume there are m grayscale values. The value range of the threshold M is from 0 to m - 1. Secondly, divide the grayscale image into two groups according to the M value, denoted as A0 and A1 respectively. Among them, A0 contains the pixel points with grayscale values in the range from 0 to M, and A1 contains the pixel points in the range from M + 1 to m - 1. The calculation formula of the between-class variance is specifically as follows:

[0030]

[0031] Among them, h represents the between-class variance, i represents the grayscale value i, R(i) represents the probability that the pixel points with grayscale value i appear, that is, the ratio relationship between the number of pixel points with grayscale value i and the total number of pixel points in the image, v1 represents the average grayscale value of the grayscale image of group A1, and v0 represents the average grayscale value of the grayscale image of group A0.

[0032] By detecting the character features and text features in the image, determine the area of each line of text and character content, and correct the tilt of the image caused by taking pictures or scanning through the Hough transform. The specific method is as follows:

[0033] A rectangular coordinate system is established with the center point of the image as the origin. The edge information in the image is highlighted through an edge detection algorithm. The line segment between any row of text and the edge points of characters in the image is determined and denoted as the edge segment. This segment is represented by the formula y = kx + a, where b1 ≤ x ≤ b2. Here, y and x represent the variables of the edge line in the rectangular coordinate system, and k and a represent the parameters of the edge line. k is the slope, reflecting the inclination degree of the edge line; a is the intercept, determining the position of the line on the y-axis. The edge detection algorithm may detect multiple edge segments, and each segment has its own slope k and intercept a. The endpoint coordinates of the edge segment are determined by finding the true values of b1 and b2. Through the Hough transform, the positions of the parameters and variables of the edge line are swapped to obtain the parameter expression form a = -kx + y, where b1 ≤ x ≤ b2 in the parameter space. Suppose there are two points A(x0, y0) and B(x1, y1) in the variable space. After performing the Hough transform on points A and B, they intersect at point (k0, b0) in the parameter space. According to the point-line duality, the intersection point (k0, b0) of two lines in the parameter space is mapped to a line in the variable space, and this line is the line formed by points A and B. The line in the original variable space is detected by calculating the intersection point in the parameter space. Here, k0 represents the slope of the line formed by points A and B. The inclination angle of this line can be obtained based on the slope and is expressed as d = xcosθ + ysinθ, where b1 ≤ x ≤ b2. Here, d represents the distance from the line to the origin, and θ represents the inclination angle of the line. The image can be corrected for inclination by rotating this inclination angle in the reverse direction, thereby improving the format accuracy of the receipt recognition result. The recognition result is verified through a semantic analysis model to identify misspelled words or text with chaotic structural order in the recognized text, thus correcting the content of the receipt recognition result.

[0034] As described above, the above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent substitutions or changes, should be covered by the protection scope of the present invention.

[0035] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not elaborate on all details and do not limit the present invention to only the specific embodiments. Obviously, many modifications and changes can be made according to the content of this specification. These embodiments are selected and specifically described in this specification to better explain the principle and practical application of the present invention, so that those skilled in the art in the relevant technical field can well understand and utilize the present invention. The present invention is only limited by the claims and their full scope and equivalents.

Claims

1. A receipt information comparison and verification system based on OCR technology, characterized in that: include: Receipt information recognition module, receipt information comparison module and receipt information verification module; The receipt information recognition module is used to collect the image of the receipt in real time, determine the text content detection area and image content detection area of ​​the receipt, and obtain the recognition result of the receipt information through OCR technology; The receipt information comparison module is used to format the recognition result of the receipt information, and check the recognized and formatted information with the information in the barcode; check whether the content and format of the receipt recognition result are consistent with the information in the barcode. If they are consistent, the receipt information does not need to be verified, and if they are inconsistent, the receipt information needs to be verified; The receipt information verification module is used to verify the receipt information whose recognition result is inconsistent with the barcode information. The image is segmented through binarization processing, the image recognition angle is corrected through the Hough transform method, and the recognition result is verified through the semantic analysis model. The text contained in the recognized text contains typos or text content with disordered structure and order, thereby correcting the content of the receipt recognition result.

2. According to claim 1, a receipt information comparison and verification system based on OCR technology is characterized in that: The receipt information recognition module collects the image of the receipt in real time, and the method for determining the text content detection area and the image content detection area of ​​the receipt includes: The image of the receipt is collected in real time by a high-definition camera. The clarity and resolution of the image taken by the high-definition camera are adjusted to obtain a receipt image that meets the actual requirements. A filtering algorithm is used to remove noise in the filtered image set. According to the format and content of the receipt to be identified, the content detection area of ​​the receipt is determined. The receipt information in the receipt content detection area includes the transaction content of the receipt, the sales order number and the barcode. The transaction content of the receipt and the sales order number are divided into the text content detection area, and the barcode is divided into the image content detection area.

3. According to the OCR technology-based receipt information comparison and verification system of claim 1, it is characterized in that: The method by which the receipt information recognition module obtains the receipt information recognition result through OCR technology is as follows: The CTPN text detection algorithm is used through OCR technology to identify the receipt information in the text content detection area, the VGG16 network is used to extract the text features of the receipt, and the region detection algorithm is used to identify and extract the key information of the receipt. Based on the predetermined receipt text character spacing, connectivity and format features, the detected text area is segmented into single characters, and the segmented characters are recognized using the trained character classification model, mapped into machine-understandable text information, and converted to obtain the recognition result of the receipt information.

4. According to claim 1, a receipt information comparison and verification system based on OCR technology is characterized in that: The method by which the receipt information comparison module compares the identified and formatted information with the information in the barcode is as follows: According to the layout and format of the receipt, the recognition result is formatted, and the receipt barcode in the image content detection area is recognized by a barcode reader to extract the key information in the barcode. The key information of the formatted recognition result is compared with the key information in the barcode to check whether the content and format of the receipt recognition result are consistent with the information in the barcode. If they are consistent, the receipt information does not need to be verified. If not, the receipt information needs to be verified.

5. According to claim 1, a receipt information comparison and verification system based on OCR technology is characterized in that: The receipt information verification module uses binarization processing to segment the image as follows: Binarize the image that needs to be verified for receipt information, set a threshold M, divide the receipt image into two groups according to M, calculate the grayscale value of each group of receipt images respectively, obtain the inter-class variance of the grayscale values ​​of the two groups of receipt images, adjust the size of the threshold M to change the size of the inter-class variance, and thus determine the true value of the threshold M when the inter-class variance is the largest. The specific steps include: determine the number of pixels in the image, set it to N, clarify the range of grayscale values ​​in the image, set there are m grayscale values, then the value range of the threshold M is 0 to m-1, divide the grayscale image into two groups according to the M value, and record them as A0 and A1 respectively, where A0 contains pixels with grayscale values ​​in the range of 0 to M, and A1 contains pixels in the range of M+1 to m-1, and use represents the calculation formula of inter-class variance, h represents the inter-class variance, i represents the grayscale value i, R(i) represents the probability of occurrence of a pixel with grayscale value i, that is, the ratio of the number of pixels with grayscale value i to the total number of pixels in the image, v1 represents the average grayscale value of the grayscale images of group A1, v0 represents the average grayscale value of the grayscale images of group A0, and the calculation formula of inter-class variance is used to determine the threshold M when the inter-class variance is the largest. The image is segmented by binarization according to the size of the threshold.

6. According to claim 1, a receipt information comparison and verification system based on OCR technology is characterized in that: The receipt information verification module uses the Hough transform method to correct the image recognition angle. The specific method is as follows: A rectangular coordinate system is established with the center point of the image as the origin. The edge information in the image is highlighted through the edge detection algorithm. The line segment between any line of text in the image and the edge point of the character is determined, recorded as an edge line segment, and the line segment is represented by the formula y=kx+a, b1≤x≤b2, where y and x represent the variables of the edge line in the rectangular coordinate system, k and a represent the parameters of the edge line, where k is the slope, reflecting the degree of inclination of the edge line; a is the intercept, which determines the position of the line on the y-axis. The edge detection algorithm may detect multiple edge line segments, each with its own slope k and intercept a. The endpoint coordinates of the edge line segment are determined by determining the true values ​​of b1 and b2, and the Hou The gh transformation swaps the positions of the edge line parameters and variables to obtain the parameter expression in the parameter space in the form of a=-kx+y, b1≤x≤b2. According to the point-line duality, the intersection of two straight lines in the parameter space (k0, b0) is mapped to the variable space as a straight line. The straight line in the original variable space is detected by calculating the intersection in the parameter space, where k0 in the parameter space represents the slope of the straight line in the variable space. The inclination angle of the straight line can be calculated based on the slope and is expressed as d=xcosθ+ysinθ, b1≤x≤b2, where d represents the distance from the straight line to the origin, and θ represents the inclination angle of the straight line. The tilt correction of the image can be achieved by rotating the inclination angle in the opposite direction.

Images