Receipt data identification processing method and device, equipment and storage medium

Abstract

The invention relates to a document data recognition processing method and device, equipment and a storage medium, and the method comprises the steps: obtaining a template picture corresponding to a target document picture, and determining the preset features of a template element in the template picture; based on the preset features of the template elements, extracting the template elements in the target document picture, and obtaining the actual features of the template elements of the target document picture; obtaining a correction parameter of the target document picture according to a preset feature of a template element in the template picture and an actual feature of the template element in the target document picture; and correcting the pixel points in the target document picture according to the correction parameters, and performing image recognition on the corrected target document picture to obtain form data. According to the method, the receipts in different formats are identified based on the template elements, the applicability and the flexibility are improved, the pixel points are optimized through the correction parameters, the influence of different local contrast ratios caused by printing quality differences is eliminated, and the accuracy and the reliability of form data extraction are improved.

Description

Technical Field

[0001] The present invention relates to the field of document identification technology, and in particular to a document data identification and processing method, device, equipment and storage medium. Background Art

[0002] Automated identification and data entry of document data can quickly process a large number of manually filled or printed paper documents, complete the conversion of paper documents to electronic forms, and provide assistance for corporate decision-making through multi-dimensional analysis of the identified electronic form data. Document data recognition technology can realize the automated processing and extraction of document information, thereby saving the labor and time costs required for form entry, and plays an important role in actual production and life.

[0003] However, due to differences in printing quality and different handwriting strengths, current documents may have different local contrasts (for example, some places are printed too dark, and some places are printed too light), which brings troubles to subsequent image recognition.

[0004] Therefore, people need a document recognition processing method that can cope with the situation of different local contrasts. Summary of the invention

[0005] Therefore, the present invention provides a document data recognition and processing method, device, equipment and storage medium to solve the problem in the prior art that documents have different local contrasts, resulting in low recognition accuracy.

[0006] The present invention provides a method for identifying and processing document data, comprising:

[0007] Obtain a template image corresponding to the target document image, and determine preset features of template elements in the template image;

[0008] Based on the preset features of the template elements, the template elements in the target document image are extracted, and the actual features of the template elements in the target document image are obtained;

[0009] According to the preset features of the template elements in the template image and the actual features of the template elements in the target document image, correction parameters of the target document image are obtained;

[0010] The pixels in the target document image are corrected according to the correction parameters, and image recognition is performed on the corrected target document image to obtain the form data.

[0011] Furthermore, the determining of the preset features of the template elements in the template image includes:

[0012] Determine a plurality of preset key points corresponding to each template element in the template image, and determine a first relative position coordinate and a first pixel value of each preset key point;

[0013] The pixel cumulative values ​​in multiple directions within the preset neighborhood of each preset key point are calculated to obtain the first gradient feature vector of each preset key point.

[0014] Furthermore, the step of extracting the template elements in the target document image based on the preset features of the template elements and obtaining the actual features of the template elements in the target document image includes:

[0015] Extracting text information from the target document image, and determining a plurality of detection key points corresponding to the template elements in the target document image according to the relative position of the text information in the target document image and the first relative position coordinates of the preset key points in the template image;

[0016] The second relative position coordinates and the second pixel value of each detected key point are obtained, and the second gradient feature vector of each detected key point is calculated.

[0017] Furthermore, the correction parameter includes a pixel offset value, a pixel brightness difference value and a gradient feature difference value, and the correction parameter re is expressed by the formula:

[0018]

[0019] Among them, k1, k1, k3 are weight coefficients, f1[(x1, y1), (x2, y2)] is the pixel offset value, which is obtained according to the first relative position coordinate (x1, y1) of the preset key point and the second relative coordinate (x2, y2) of the corresponding detection key point; f2(θ1, θ2) is the pixel brightness difference, which is obtained according to the first brightness value θ1 of the preset key point and the second brightness value θ2 of the corresponding detection key point; is the gradient feature difference, according to the first gradient feature vector of the preset key point and the second gradient feature vector corresponding to the detected key point get.

[0020] Furthermore, the step of correcting the pixels in the target document image according to the correction parameters includes:

[0021] Calibrate the positions of the pixels in the target image based on the average pixel offset of the detected key points, so that the positions of the template elements in the target image and the template image match;

[0022] According to the brightness difference between the key points detected after position matching and the preset key points, the brightness of the target image pixels is corrected;

[0023] A second gradient feature vector is obtained according to the pixel accumulation value of the key point detected in multiple directions within a preset range, and pixel feature correction is performed on the pixel values ​​in each direction according to the second gradient feature vector and the first gradient feature vector corresponding to the preset key point.

[0024] Further, pixel feature correction is performed on pixel values ​​in each direction according to the second gradient feature vector and the first gradient feature vector corresponding to the preset key point, including:

[0025] Calculate the difference between the pixel accumulation values ​​of each detected key point and the corresponding preset key point within a preset range in each gradient direction;

[0026] Based on the difference in pixel cumulative values ​​in each gradient direction and the pixel coordinates of the preset key points, pixel values ​​of the pixels in the preset data area in the gradient direction in the target document image are corrected.

[0027] Furthermore, the correction parameters also include color differences of pixels;

[0028] The color difference is the cosine similarity between the detection key point and the preset key point on multiple color channel vectors.

[0029] The present invention also provides a document data recognition and processing system, comprising:

[0030] A data acquisition unit, used to acquire a template image corresponding to a target document image, and determine preset features of template elements in the template image;

[0031] A feature extraction unit, used to extract the template elements in the target document image based on the preset features of the template elements, and obtain the actual features of the template elements in the target document image;

[0032] A parameter calculation unit, used to obtain correction parameters of the target document image according to preset features of the template elements in the template image and actual features of the template elements in the target document image;

[0033] The recognition unit is used to correct the pixel points in the target document image according to the correction parameters, perform image recognition on the corrected target document image, and obtain form data.

[0034] The present invention also provides an electronic device, comprising:

[0035] Memory and processor;

[0036] The memory is used to store programs, and the processor is used to execute the steps in any one of the above-mentioned document data identification and processing methods when executing the program.

[0037] The present invention also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the program or instructions are executed by a processor, the steps in the method for identifying and processing document data in any of the above-mentioned implementation modes can be implemented.

[0038] The beneficial effects of adopting the above embodiment are:

[0039] 1. By obtaining the template image corresponding to the target document image and determining the template element features in the template image, documents of different formats or layouts can be quickly identified based on the template elements, thereby improving the applicability and flexibility of processing.

[0040] 2. According to the preset features of the template elements and the actual features of the template elements in the target document image, the correction parameters of the target document image are obtained, and the pixels in the target document image are corrected based on the correction parameters. This can ensure that the corrected image is more consistent with the template image, eliminate the impact of different local contrasts caused by differences in printing quality, make the eliminated data clearer, and improve the accuracy and reliability of form data extraction.

[0041] 3. The entire process of this method can be automated by writing scripts or programs, which improves processing efficiency and consistency and reduces the need for human intervention. It is particularly effective when processing a large number of document images and has strong practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A method flow chart of an embodiment of a method for identifying and processing document data provided by the present invention;

[0043] Figure 2 A schematic diagram of the layout of an embodiment of a form template image;

[0044] Figure 3 A schematic diagram of the structure of an embodiment of a document data recognition and processing system provided by the present invention;

[0045] Figure 4 A schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] A specific embodiment of the present invention discloses a method for identifying and processing document data. Figure 1 is a schematic flow chart of the method, comprising:

[0048] Step S101: obtaining a template image corresponding to a target document image, and determining preset features of template elements in the template image;

[0049] Step S102: extracting the template elements in the target document image based on the preset features of the template elements, and obtaining the actual features of the template elements in the target document image;

[0050] Step S103: obtaining correction parameters of the target document image according to preset features of the template elements in the template image and actual features of the template elements in the target document image;

[0051] Step S104: Correct the pixels in the target document image according to the correction parameters, perform image recognition on the corrected target document image, and obtain form data.

[0052] Combine the following Figure 2 The above technical solution is explained. If the target document image is a printed paper sales list image, then you first need to obtain the template image corresponding to the target document image, such as Figure 2 As shown, Figure 2 This is the sales list template image. The dotted box in the image is the template element (such as table headers, table lines and other elements). In order to locate the template elements in the target document and correct the printing quality, it is necessary to compare the preset features of the template elements in the template image with the actual features of the template elements in the target document image to obtain the correction parameters of the target document, and then correct the pixels in the target document image according to the correction parameters, and perform image recognition on the corrected target document image to obtain the form data.

[0053] Compared with the prior art, the document data recognition and processing method provided in this embodiment, firstly, by obtaining the template image corresponding to the target document image and determining the template element features in the template image, can quickly identify documents of different formats or layouts based on the template elements, thereby improving the applicability and flexibility of the processing. Secondly, according to the preset features of the template elements and the actual features of the template elements in the target document image, the correction parameters of the target document image are obtained, and the pixels in the target document image are corrected based on the correction parameters, which can ensure that the corrected image is more consistent with the template image, eliminate the influence of different local contrasts caused by differences in printing quality, make the eliminated data clearer, and improve the accuracy and reliability of form data extraction. Finally, the entire process can be automated by writing scripts or programs, which improves processing efficiency and consistency, reduces the need for manual intervention, and is particularly effective when processing a large number of document images, and has strong practicality.

[0054] In order to locate the template element from the target document more quickly and quantitatively analyze the correction value between the template element and the actual form, it is necessary to first characterize the template element. In some embodiments, determining the preset features of the template element in the template image includes:

[0055] Determine a plurality of preset key points corresponding to each template element in the template image, and determine a first relative position coordinate and a first pixel value of each preset key point;

[0056] The pixel cumulative values ​​in multiple directions within the preset neighborhood of each preset key point are calculated to obtain the first gradient feature vector of each preset key point.

[0057] By describing the preset features of the template elements by using preset key points, the matching effect between the template image and the target image can be improved, thereby more accurately identifying the template elements in the target image. It should be noted that each preset key point is not only set at the edge area of ​​the template element, but also preset key points within the contour range of the template element. Furthermore, the preset neighborhood range can be a rectangular area of ​​N×N pixels. By calculating the cumulative value of pixels in multiple directions within the preset neighborhood range, the adaptability of the template elements to changes in different directions can be enhanced, and the robustness of recognition can be improved. The printing quality can be quickly identified when there are missing lines or unclear printing in a certain direction.

[0058] In some embodiments, the extracting of the template elements in the target document image based on the preset features of the template elements and obtaining the actual features of the template elements in the target document image includes:

[0059] Extracting text information from the target document image, and determining a plurality of detection key points corresponding to the template elements in the target document image according to the relative position of the text information in the target document image and the first relative position coordinates of the preset key points in the template image;

[0060] The second relative position coordinates and the second pixel value of each detected key point are obtained, and the second gradient feature vector of each detected key point is calculated.

[0061] Here, the text information in the target document image is extracted (the specific content of the text does not need to be recognized here), for example Figure 2 For the sales form in the example, you can use the text information on the edge of the form to locate it, that is, the "sales list" at the top and the line numbers "1, 2, 3..." on the left side of the form. By comparing the position information of the text information, you can quickly match the position of the template elements.

[0062] Furthermore, in addition to determining all template elements in the target form by matching the positions of text information, the borders or cell lines of the table may also be matched by extracting the outline of the target form.

[0063] By obtaining the actual features of each detected key point, the recognition accuracy of template elements can be improved, thereby improving the overall recognition accuracy.

[0064] In some embodiments, the correction parameter includes a pixel offset value, a pixel brightness difference value, and a gradient feature difference value. The correction parameter re is expressed by the formula:

[0065]

[0066] Among them, k1, k1, k3 are weight coefficients, f1[(x1, y1), (x2, y2)] is the pixel offset value, which is obtained according to the first relative position coordinate (x1, y1) of the preset key point and the second relative coordinate (x2, y2) of the corresponding detection key point; f2(θ1, θ2) is the pixel brightness difference, which is obtained according to the first brightness value θ1 of the preset key point and the second brightness value θ2 of the corresponding detection key point; is the gradient feature difference, according to the first gradient feature vector of the preset key point and the second gradient feature vector corresponding to the detected key point get.

[0067] In the above embodiment, in order to improve the recognition efficiency of the document, the target document image and the template image placed in the same direction are obtained under the same shooting angle and camera focal length, without considering image scaling and rotation. However, due to printing quality problems, the form may be tilted as a whole. At this time, the target document needs to be corrected by the pixel offset value, so as to achieve the alignment of the template elements and lay the foundation for the subsequent form data recognition.

[0068] For machine-printed paper forms, there may be some regular missing pixels and other problems, such as missing columns, that is, the brightness of pixels after fixed intervals of columns becomes low. At this time, it is necessary to correct the brightness of the pixels of the target forms corresponding to these forms to improve the clarity of the form data. At this time, the average value of the pixels on both sides of the missing column can be calculated, and the average value can be applied to the pixels of the missing column to smoothly correct the brightness change, making the overall form visual effect more coherent, or adding a suitable interpolation algorithm (such as bilinear interpolation, cubic spline interpolation, etc.) to fill the missing pixels, and interpolate through the information of the surrounding pixels to restore the brightness of the pixels in the missing column.

[0069] In actual operation, the brightness of pixels within a certain range around the template element can be adjusted based on the ratio of the brightness of the preset key points and the detected key points in the form area according to the template element, thereby improving the brightness and contrast of the pixels.

[0070] In some embodiments, the step of correcting the pixels in the target document image according to the correction parameters includes:

[0071] Calibrate the positions of the pixels in the target image based on the average pixel offset of the detected key points, so that the positions of the template elements in the target image and the template image match;

[0072] According to the brightness difference between the key points detected after position matching and the preset key points, the brightness of the target image pixels is corrected;

[0073] A second gradient feature vector is obtained according to the pixel accumulation value of the key point detected in multiple directions within a preset range, and pixel feature correction is performed on the pixel values ​​in each direction according to the second gradient feature vector and the first gradient feature vector corresponding to the preset key point.

[0074] In some embodiments, pixel feature correction is performed on pixel values ​​in each direction according to the second gradient feature vector and the first gradient feature vector corresponding to the preset key point, including:

[0075] Calculate the difference between the pixel accumulation values ​​of each detected key point and the corresponding preset key point within a preset range in each gradient direction;

[0076] Based on the difference in pixel cumulative values ​​in each gradient direction and the pixel coordinates of the preset key points, pixel values ​​of the pixels in the preset data area in the gradient direction in the target document image are corrected.

[0077] Specifically, in order to correct the data in the form data filling area, it is necessary to determine the location based on the pixel coordinates of the preset key points. For example, taking the preset key points on the cell outline as the object, if the operator in the writing area fills in the first line of data with normal strokes, but the second line of data changes due to changes in writing force, resulting in unclear strokes, then there will be differences in the pixel cumulative values ​​in different gradient directions. According to the size and directionality of the difference, by adjusting the values ​​of the relevant pixels, or interpolating the pixel values ​​of the unclear stroke area, the missing parts are filled in to make the strokes clear again. Here, the repair of the missing parts can be trained using a convolutional neural network, and feature learning is performed on the continuity of writing, thereby improving the accuracy of repairing the missing data.

[0078] In some embodiments, the correction parameters further include color differences of pixels;

[0079] The color difference is calculated based on the cosine similarity between the detected key point and the preset key point on multiple color channel vectors.

[0080] By introducing the color difference of pixels, the situation of filling out forms with colored pens (blue or red) can be solved. In the RGB color model, each pixel is composed of the values ​​of three channels: red (R), green (G), and blue (B). Words written by pens of different colors will have different numerical distributions on these three channels, resulting in different color performances. The degree of difference in color space is determined based on the cosine similarity values ​​on different color channel vectors, and the colored fonts are converted to black. The contrast of black text on a white background can facilitate subsequent data extraction.

[0081] In some embodiments, image recognition is performed on the corrected target document image to obtain form data, and the extracted form data can be recognized and entered using OCR technology or a deep learning network.

[0082] Combination Figure 3 As shown, the present invention also provides a document data recognition and processing system 300, comprising:

[0083] The data acquisition unit 301 is used to acquire a template image corresponding to a target document image and determine preset features of template elements in the template image;

[0084] A feature extraction unit 302 is used to extract the template elements in the target document image based on the preset features of the template elements, and obtain the actual features of the template elements in the target document image;

[0085] A parameter calculation unit 303, used to obtain correction parameters of the target document image according to preset features of the template elements in the template image and actual features of the template elements in the target document image;

[0086] The recognition unit 304 is used to correct the pixel points in the target document image according to the correction parameters, and perform image recognition on the corrected target document image to obtain form data.

[0087] It should be noted here that the corresponding document data identification and processing system 300 provided in the above embodiment can implement the technical solutions described in the above method embodiments. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above method embodiments, which will not be repeated here.

[0088] refer to Figure 4 , Figure 4 The structure diagram of an electronic device provided by an embodiment of the present invention is shown. In this embodiment, the electronic device includes:

[0089] Memory 410 and processor 420;

[0090] The memory 410 is used to store programs, and the processor 420 is used to execute the document data identification and processing method in the above embodiment when executing the program.

[0091] This embodiment further provides a computer-readable storage medium on which a document data recognition processing program is stored. When the document data recognition processing program is executed by a processor, the steps in the above embodiment can be implemented.

[0092] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0093] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying and processing document data, characterized in that: include: Obtain a template image corresponding to the target document image, and determine preset features of template elements in the template image; Based on the preset features of the template elements, the template elements in the target document image are extracted, and the actual features of the template elements in the target document image are obtained; According to the preset features of the template elements in the template image and the actual features of the template elements in the target document image, correction parameters of the target document image are obtained; The pixels in the target document image are corrected according to the correction parameters, and image recognition is performed on the corrected target document image to obtain the form data.

2. The method for identifying and processing document data according to claim 1, characterized in that: The step of determining the preset features of the template elements in the template image includes: Determine a plurality of preset key points corresponding to each template element in the template image, and determine a first relative position coordinate and a first pixel value of each preset key point; The pixel cumulative values ​​in multiple directions within the preset neighborhood of each preset key point are calculated to obtain the first gradient feature vector of each preset key point.

3. The method for identifying and processing document data according to claim 2, characterized in that: The step of extracting the template elements in the target document image based on the preset features of the template elements and obtaining the actual features of the template elements in the target document image includes: Extracting text information from the target document image, and determining a plurality of detection key points corresponding to the template elements in the target document image according to the relative position of the text information in the target document image and the first relative position coordinates of the preset key points in the template image; The second relative position coordinates and the second pixel value of each detected key point are obtained, and the second gradient feature vector of each detected key point is calculated.

4. The method for identifying and processing document data according to claim 3, characterized in that: The correction parameters include pixel offset value, pixel brightness difference value and gradient feature difference value. The correction parameter re is expressed by the formula: Among them, k1, k1, k3 are weight coefficients, f1[(x1, y1), (x2, y2)] is the pixel offset value, which is obtained according to the first relative position coordinate (x1, y1) of the preset key point and the second relative coordinate (x2, y2) of the corresponding detection key point; f2(θ1, θ2) is the pixel brightness difference, which is obtained according to the first brightness value θ1 of the preset key point and the second brightness value θ2 of the corresponding detection key point; is the gradient feature difference, according to the first gradient feature vector of the preset key point and the second gradient feature vector corresponding to the detected key point get.

5. The method for identifying and processing document data according to claim 4, characterized in that: The step of correcting the pixel points in the target document image according to the correction parameters includes: Calibrate the positions of the pixels in the target image based on the average pixel offset of the detected key points, so that the positions of the template elements in the target image and the template image match; According to the brightness difference between the key points detected after position matching and the preset key points, the brightness of the target image pixels is corrected; A second gradient feature vector is obtained according to the pixel accumulation value of the key point detected in multiple directions within a preset range, and pixel feature correction is performed on the pixel values ​​in each direction according to the second gradient feature vector and the first gradient feature vector corresponding to the preset key point.

6. The method for identifying and processing document data according to claim 5, characterized in that: Performing pixel feature correction on pixel values ​​in each direction according to the second gradient feature vector and the first gradient feature vector corresponding to the preset key point includes: Calculate the difference between the pixel accumulation values ​​of each detected key point and the corresponding preset key point within a preset range in each gradient direction; Based on the difference in pixel cumulative values ​​in each gradient direction and the pixel coordinates of the preset key points, pixel values ​​of the pixels in the preset data area in the gradient direction in the target document image are corrected.

7. The method for identifying and processing document data according to claim 5, characterized in that: The correction parameters also include color differences of pixels; The color difference is calculated based on the cosine similarity between the detected key point and the preset key point on multiple color channel vectors.

8. A document data recognition and processing system, characterized in that: include: A data acquisition unit, used to acquire a template image corresponding to a target document image, and determine preset features of template elements in the template image; A feature extraction unit, used to extract the template elements in the target document image based on the preset features of the template elements, and obtain the actual features of the template elements in the target document image; A parameter calculation unit, used to obtain correction parameters of the target document image according to preset features of the template elements in the template image and actual features of the template elements in the target document image; The recognition unit is used to correct the pixel points in the target document image according to the correction parameters, perform image recognition on the corrected target document image, and obtain form data.

9. An electronic device, characterized in that: include: Memory and processor; The memory is used to store programs, and the processor is used to execute the steps in the document data identification and processing method according to any one of claims 1 to 7 when executing the program.

10. A computer-readable storage medium, characterized in that: Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the document data identification and processing method described in any one of claims 1 to 7.

Images