Method, storage medium and processing equipment for identifying handwritten tampering of orders

Abstract

The present invention discloses a method, storage medium and processing equipment for identifying handwritten tampering of an order, and relates to the technical field of text image recognition. The present invention comprises the following steps: S11, extracting structured information of an order; S12, calculating the amount of each commodity item divided by the price value according to the position and text content of the identified text box, and identifying the commodity item whose quantity is inconsistent with the calculated value as quantity tampering; S13, calculating the standard LAB value of the text content corresponding to the specification name of the commodity item for all commodity items; S14, for commodity items identified as non-quantity tampering, comparing the LAB value of the text content corresponding to the specification name of the commodity item with the standard LAB value, and identifying the commodity item greater than the threshold as specification name tampering; S15, executing tampering recognition of the next order. The method of the present invention can effectively identify incorrect tampering of an order, with high accuracy and fast recognition speed.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent order recognition, and in particular to a method, storage medium and processing device for recognizing handwritten tampering of an order. Background Art

[0002] In the tobacco retail industry, manufacturers need to conduct intelligent identification and compliance review of retailer orders to provide a basis for identity verification, event reward distribution, etc. However, as long as it is profitable, retailers will cheat and tamper with orders to meet the requirements of manufacturers in order to obtain more rewards.

[0003] There are two common types of order tampering: quantity tampering: modifying the quantity of an existing product item, while keeping the unit price and amount unchanged; specification tampering: forging a new product item in the blank space of the form, including the complete specification name, quantity, unit price, and amount.

[0004] Existing order recognition technology can only extract structured order information, including header information (date, document number, customer name, etc.) and product item information (specification name, quantity, unit price, amount), and cannot determine whether the order has been tampered with. Summary of the invention

[0005] The purpose of the present invention is to provide a method, storage medium and processing device for identifying handwritten tampering of an order, so as to solve the above technical problems existing in the prior art. The preferred technical solution among the many technical solutions provided by the present invention can produce many technical effects as described below.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] The present invention provides a method for identifying handwritten tampering of an order, comprising the following steps:

[0008] S11, extracting structured information of the order, identifying the positions of the text boxes corresponding to the specification names, quantities, unit prices and amounts of all commodity items and the text contents recorded in the text boxes;

[0009] S12, calculating the amount of each of the commodity items divided by the price according to the identified position of the text box and the text content, and identifying the commodity items whose quantity is inconsistent with the calculated value as quantity tampering;

[0010] S13, for all the commodity items, calculating the standard LAB value of the text content corresponding to the specification name of the commodity item;

[0011] S14, for the commodity items identified as not tampered with in quantity, comparing the LAB value of the text content corresponding to the specification name of the commodity item with the standard LAB value, and identifying the commodity items with a value greater than a threshold as tampered with in specification name;

[0012] S15. Return to step S11 and perform tampering identification for the next order.

[0013] Further, step S13 includes the following steps:

[0014] S131, extracting image blocks of the text boxes corresponding to the specification names of all identified commodity items;

[0015] S132, performing grayscale and binarization operations on each extracted image block to obtain a text mask of the text content of the image block;

[0016] S133, calculating the average LAB value of all pixels in the text mask for each image block, and using the average LAB value as the LAB value of the text box corresponding to the image block;

[0017] S134. In all the product entries, the text boxes corresponding to the specification names are sorted in the order of multiple columns from left to right and from top to bottom within the columns, and using a preset sorting logic;

[0018] S135 , taking an average of the LAB values ​​of a certain proportion of the text boxes in the front, and using the average as the standard LAB value of the specification name of the product item.

[0019] Furthermore, the preset sorting logic in step S134 includes the following steps:

[0020] S1341, calculating the average width avg_width of the text boxes corresponding to all the specification names;

[0021] S1342, selecting the vertex coordinates (x1, y1) and (x2, y2) of the upper left corners of the two text boxes T1 and T2 to be compared from the text boxes corresponding to all the specification names respectively;

[0022] S1343: Whether the first condition is met; if so, execute step S1344; otherwise, execute step S1345;

[0023] S1344, arrange the text box corresponding to the smaller value of x1 and x2 in the front, update the text box and its coordinates in the back to T1 and (x1, y1) respectively, and select one of the remaining unselected text boxes as T2, obtain the vertex coordinates of its upper left corner (x2, y2), and return to step S1343;

[0024] S1345: Whether the second condition is met; if so, execute step S1346; otherwise, execute step S1347;

[0025] S1346, arrange the text box corresponding to the smaller value of y1 and y2 in the front, update the text box and its coordinates in the back to T1 and (x1, y1) respectively, and select one of the remaining unselected text boxes as T2, obtain the vertex coordinates of its upper left corner, (x2, y2), and return to step S1343;

[0026] S1347. The order of the text boxes is T1, T2, and return to step S1342.

[0027] Furthermore, the first condition in step S1343 is: abs(x1-x2)>1.5*avg_width; the second condition in step S1345 is: abs(y1-y2)>1.5*avg_width; wherein abs is an absolute value function.

[0028] Further, step S14 includes the following steps:

[0029] S141, comparing the calculated LAB value of each of the text boxes with the standard LAB value one by one;

[0030] S142: Whether the difference comparison result is greater than the threshold; if so, execute step S143; otherwise, execute step S144;

[0031] S143, identifying the commodity entry whose number is greater than the threshold as a tampered specification name, and executing step S15;

[0032] S144. Execute step S15.

[0033] Furthermore, in step S142, the Euclidean distance D in the LAB color space is used as the difference comparison result; the calculation formula of the Euclidean distance D is:

[0034]

[0035] Among them, L1, A1, and B1 are the values ​​corresponding to the three color components of the text box LAB value, and L0, A0, and B0 are the values ​​corresponding to the three color components of the standard LAB value.

[0036] Furthermore, in step S11, an OCR recognition method is used to identify the positions of text boxes corresponding to the specification names, quantities, unit prices and amounts of all commodity items and the text contents recorded in the text boxes.

[0037] According to another aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed, the method for identifying handwritten tampering of an order as described above is implemented.

[0038] According to another aspect of the present invention, a universal text and image duplicate checking and processing device is provided, comprising a memory, an output module and one or more processors; the processor is connected to the memory and the output module; the memory is used to store one or more computer programs, and one or more processors are used to execute one or more computer programs stored in the memory, so that one or more processors execute the method for identifying handwritten tampering of an order as described above; the output module is used to output the results of the processor executing the method for identifying handwritten tampering of an order as described above.

[0039] Implementing one of the above technical solutions of the present invention has the following advantages or beneficial effects:

[0040] The present invention combines OCR recognition technology with LAB color recognition to identify the tampering of the quantity and specification name of the commodity order, so as to facilitate the standardized management of the retailer by the manufacturer. The method can effectively identify the tampering of the order, has high accuracy and fast recognition speed, and can be widely used in the management of retailers. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. It is obvious that the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0042] Figure 1 This is a flow chart of a method for identifying handwritten tampering of an order according to an embodiment of the present invention;

[0043] Figure 2 is a flowchart of step S13 in a method for identifying handwritten tampering of an order according to an embodiment of the present invention;

[0044] Figure 3 It is a flowchart of step S14 in a method for identifying handwritten tampering of an order according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present invention clearer, the various exemplary embodiments to be described below will refer to the corresponding drawings, which constitute a part of the exemplary embodiments, wherein various exemplary embodiments that may be used to implement the present invention are described. Unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation methods described in the following exemplary embodiments do not represent all implementation methods consistent with the present disclosure. It should be understood that they are only examples of processes, methods, devices, etc. that are consistent with some aspects of the present disclosure as detailed in the attached claims, and other embodiments may also be used, or the embodiments listed herein may be modified in structure and function without departing from the scope and essence of the present invention.

[0046] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", etc. indicate the orientation or positional relationship based on the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the elements referred to must have a specific orientation, be constructed and operated in a specific orientation. The terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. The term "multiple" means two or more. The terms "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, an integral connection, a mechanical connection, an electrical connection, a communication connection, a direct connection, an indirect connection through an intermediate medium, and can be the internal connection of two elements or the interaction relationship between two elements. The term "and / or" includes any and all combinations of one or more related listed items. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0047] In order to illustrate the technical solution of the present invention, a specific embodiment is used below for description, and only the parts related to the embodiment of the present invention are shown.

[0048] Embodiment 1:

[0049] like Figure 1 As shown, the present invention provides a method for identifying handwritten tampering of an order, comprising the following steps:

[0050] S11, extract the structured information of the order, identify the position of the text box corresponding to the specification name, quantity, unit price and amount of all commodity items and the text content recorded in the text box. In this step, the structured information is extracted according to the general steps of order recognition (OCR + post-processing), and the text box position and text content of the specification name, quantity, unit price and amount of all commodity items can be obtained. OCR recognizes some scattered text boxes and text contents, and post-processing is to analyze the relationship between these text boxes based on the prior knowledge of the document structure and extract structured information from them. OCR (Optical Character Recognition) refers to the process in which an electronic device (such as a scanner or a digital camera) examines the characters printed on paper, determines their shape by detecting dark and light patterns, and then translates the shape into computer text using a character recognition method; that is, for printed characters, the text in the paper document is converted into a black and white dot matrix image file by optical means, and the text in the image is converted into a text format by recognition software for further editing and processing by word processing software. It should be noted that because OCR has requirements for text clarity and post-processing has requirements for the composition structure of product items, this step can filter out some tampering types such as price tampering, amount tampering, and incomplete information of forged product items.

[0051] S12. Calculate the amount of each commodity item divided by the price value based on the position and text content of the identified text box, and identify the commodity items whose quantity is inconsistent with the calculated value as quantity tampering. This step uses the OCR recognition results to search for the numerical content recorded in the text boxes corresponding to the quantity, unit price, and amount for each commodity item in the recognition results, and compare the value obtained by dividing the queried amount by the unit price with the quantity. If the quantity has been tampered with, the calculated value must be inconsistent with the identified quantity, and the quantity tampering can be effectively identified. Next, the tampering of the commodity specifications can be further judged, that is, the tampering of the specification name in the identified commodity item can be identified;

[0052] S13. For all product items, calculate the standard LAB value of the text content corresponding to the specification name of the product item. The specific steps are as follows:

[0053] S131, extracting image blocks of text boxes corresponding to the specification names of all recognized commodity items;

[0054] S132, performing grayscale and binarization operations on each extracted image block to obtain a text mask of the text content of the image block;

[0055] S133, calculating the average LAB value of all pixels in the text mask for each image block, and using the average LAB value as the LAB value of the text box corresponding to the image block;

[0056] S134. In all product entries, the text boxes corresponding to the specification names are sorted in the order of multiple columns from left to right and from top to bottom within the columns, and using a preset sorting logic. Further, the preset sorting logic includes the following steps:

[0057] S1341. Calculate the average width avg_width of the text boxes corresponding to all specification names;

[0058] S1342, from the text boxes corresponding to all specification names, respectively select the vertex coordinates (x1, y1) and (x2, y2) of the upper left corners of two text boxes T1 and T2 to be compared;

[0059] S1343: Whether the first condition is met; if so, execute step S1344; otherwise, execute step S1345. Specifically, the first condition is: abs(x1-x2)>1.5*avg_width;

[0060] S1344, put the text box corresponding to the smaller value of x1 and x2 in the front, update the text box and its coordinates in the back to T1 and (x1, y1) respectively, and select one of the remaining unselected text boxes as T2, obtain the vertex coordinates of its upper left corner (x2, y2), and return to step S1343;

[0061] S1345: Whether the second condition is met; if so, execute step S1346; otherwise, execute step S1347. Specifically, the second condition is: abs(y1-y2)>1.5*avg_width;

[0062] S1346, put the text box corresponding to the smaller value of y1 and y2 in the front, update the text box and its coordinates in the back to T1 and (x1, y1) respectively, and select one of the remaining unselected text boxes as T2, obtain the vertex coordinates of its upper left corner, (x2, y2), and return to step S1343;

[0063] S1347. The order of the text boxes is T1, T2, and return to step S1342.

[0064] S135. Take the average of the LAB values ​​of a certain proportion of the text boxes in the front, and use the average value as the standard LAB value of the specification name of the product item. The L component in the LAB color space is used to represent the brightness of the pixel, and the value range is [0,100], representing from pure black to pure white; A represents the range from red to green, and the value range is [127,-128]; B represents the range from yellow to blue, and the value range is [127,-128]. Taking the average value means taking the average of each component separately to form a standard LAB value. Furthermore, the certain proportion in this embodiment is 70%. Because specification tampering can only appear in the last blank cell in the order table, and the number of specification tampering is generally not too many, it is assumed that the first 70% of the specification names after sorting are not specification tampering, and the text color of these specification names is taken as the text color of the entire order product item;

[0065] S14: For the commodity items identified as non-quantity tampering, the LAB value of the text content corresponding to the specification name of the commodity item is compared with the standard LAB value, and the commodity items with a value greater than the threshold are identified as tampering with the specification name. The detailed steps are as follows:

[0066] S141, comparing the calculated LAB value of each text box with the standard LAB value one by one;

[0067] S142: Whether the difference comparison result is greater than the threshold; if so, execute step S143; otherwise, execute step S144. In this embodiment, the Euclidean distance D of the LAB color space is used as the difference comparison result, and the calculation formula of the Euclidean distance D is:

[0068]

[0069] Among them, L1, A1, B1 are the values ​​corresponding to the three color components of the text box LAB value, and L0, A0, B0 are the values ​​corresponding to the three color components of the standard LAB value. Furthermore, the above threshold is 10;

[0070] S143, identifying the commodity entries with a value greater than the threshold as tampered specification names, and executing step S15;

[0071] S144, execute step S15;

[0072] S15. Return to step S11 and perform tampering identification for the next order.

[0073] In summary, this embodiment combines OCR recognition technology with LAB color recognition to identify the tampering of the quantity and specification name of the commodity order, so as to facilitate the standardized management of the retailer by the manufacturer. This method can effectively identify the tampering of the order, has high accuracy and fast recognition speed, and can be widely used in the management of retailers.

[0074] Embodiment 2:

[0075] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed, the method for identifying handwritten tampering of an order described in Example 1 is implemented.

[0076] Those skilled in the art will appreciate that all or part of the features / steps of the above-mentioned method embodiments may be implemented through methods, data processing systems or computer programs, and these features may be implemented without hardware, entirely through software or through a combination of hardware and software.

[0077] Embodiment three:

[0078] The present invention also provides a correction processing device for the result of order commodity quantity identification, including a memory, an output module and one or more processors, and the processors are connected to the memory and the output module. The memory is used to store one or more computer programs, and the one or more processors are used to execute the one or more computer programs stored in the memory, so that the one or more processors execute the order handwriting tampering identification method as described in Example 1; the output module is used to output the identification result of the processor executing the order handwriting tampering identification method as described in Example 1. Further, the memory includes: a static hard disk, a solid state hard disk, a random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), an optical storage device, a magnetic storage device, a flash memory, a magnetic disk or an optical disk and / or a combination of the above devices, that is, it can be implemented by any type of volatile or non-volatile storage device or a combination thereof. The output module can be a display, a computer terminal and / or a mobile phone terminal.

[0079] The above description is only the preferred embodiment of the present invention. It is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application belong to the protection scope of the present invention.

Claims

1. A method for identifying handwritten tampering of orders, It is characterized in that The steps include: S11, extracting structured information of the order, identifying the positions of text boxes corresponding to the specification names, quantities, unit prices and amounts of all commodity items and the text contents recorded in the text boxes; using an OCR recognition method to identify the positions of text boxes corresponding to the specification names, quantities, unit prices and amounts of all commodity items and the text contents recorded in the text boxes; S12, calculating the amount of each of the commodity items divided by the price according to the identified position of the text box and the text content, and identifying the commodity items whose quantity is inconsistent with the calculated value as quantity tampering; S13, for all the commodity items, calculating the standard LAB value of the text content corresponding to the specification name of the commodity item; S14, for the commodity items identified as not tampered with in quantity, comparing the LAB value of the text content corresponding to the specification name of the commodity item with the standard LAB value, and identifying the commodity items with a value greater than a threshold as tampered with in specification name; S15. Return to step S11 and perform tampering identification for the next order.

2. The method for identifying handwritten tampering of an order according to claim 1, It is characterized in that Step S13 includes the following steps: S131, extracting image blocks of the text boxes corresponding to the specification names of all identified commodity items; S132, performing grayscale and binarization operations on each extracted image block to obtain a text mask of the text content of the image block; S133, calculating the average LAB value of all pixels in the text mask for each image block, and using the average LAB value as the LAB value of the text box corresponding to the image block; S134. In all the product entries, the text boxes corresponding to the specification names are sorted in the order of multiple columns from left to right and from top to bottom within the columns, and using a preset sorting logic; S135 , taking an average of the LAB values ​​of a certain proportion of the text boxes in the front, and using the average as the standard LAB value of the specification name of the product item.

3. The method for identifying handwritten tampering of an order according to claim 2, It is characterized in that The preset sorting logic in step S134 includes the following steps: S1341, calculating the average width avg_width of the text boxes corresponding to all the specification names; S1342, selecting the vertex coordinates (x1, y1) and (x2, y2) of the upper left corners of the two text boxes T1 and T2 to be compared from the text boxes corresponding to all the specification names respectively; S1343: Whether the first condition is met; if so, execute step S1344; otherwise, execute step S1345; S1344, arrange the text box corresponding to the smaller value of x1 and x2 in the front, update the text box and its coordinates in the back to T1 and (x1, y1) respectively, and select one of the remaining unselected text boxes as T2, obtain the vertex coordinates of its upper left corner (x2, y2), and return to step S1343; S1345: Whether the second condition is met; if so, execute step S1346; otherwise, execute step S1347; S1346, arrange the text box corresponding to the smaller value of y1 and y2 in the front, update the text box and its coordinates in the back to T1 and (x1, y1) respectively, and select one of the remaining unselected text boxes as T2, obtain the vertex coordinates of its upper left corner, (x2, y2), and return to step S1343; S1347. The order of the text boxes is T1, T2, and return to step S1342.

4. The method for identifying handwritten tampering of an order according to claim 3, It is characterized in that The first condition in step S1343 is: abs(x1-x2)>1.5*avg_width; The second condition in step S1345 is: abs(y1-y2)>1.5*avg_width; Among them, abs is the absolute value function.

5. The method for identifying handwritten tampering of an order according to claim 4, It is characterized in that Step S14 includes the following steps: S141, comparing the calculated LAB value of each of the text boxes with the standard LAB value one by one; S142: Whether the difference comparison result is greater than the threshold; if so, execute step S143; otherwise, execute step S144; S143, identifying the commodity entry whose number is greater than the threshold as a tampered specification name, and executing step S15; S144. Execute step S15.

6. The method for identifying handwritten tampering of an order according to claim 5, It is characterized in that In step S142, the Euclidean distance D in the LAB color space is used as the difference comparison result; The calculation formula of the Euclidean distance D is: Among them, L1, A1, and B1 are the values ​​corresponding to the three color components of the text box LAB value, and L0, A0, and B0 are the values ​​corresponding to the three color components of the standard LAB value.

7. A computer-readable storage medium, It is characterized in that The storage medium stores a computer program, which, when executed, implements the method for identifying handwritten tampering of an order as described in any one of claims 1-6.

8. A device for correcting and processing order quantity recognition results, It is characterized in that It includes a memory, an output module and one or more processors; the processor is connected to the memory and the output module; The memory is used to store one or more computer programs, and the one or more processors are used to execute the one or more computer programs stored in the memory, so that the one or more processors execute the method for identifying handwritten tampering of an order as described in any one of claims 1 to 6; The output module is used to output the recognition result of the processor executing the order handwriting tampering recognition method as described in any one of claims 1-6.

Images