Form recognition method and device based on computer vision algorithm and related components

Abstract

The application discloses a form recognition method and device based on a computer vision algorithm and related components, and relates to the field of image processing.The method comprises the following steps: performing character recognition on a received target image to obtain all text content and performing screening, and performing merging on the text content meeting a preset merging rule; obtaining position information and recognition direction information of each field name based on received template matching information; matching a preset sorting rule based on the recognition direction information corresponding to the current field name, and sorting all the merged text content to obtain a text content set; obtaining target text content from the text content set based on the position information of the field name and the template matching information; and traversing all the field names to output a target form.The method can automatically match the text content of the uploaded target image, so that the content of the output target form is the same as that in the target image, that is, the input efficiency and accuracy are improved.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a form recognition method, apparatus, and related components based on computer vision algorithms. Background Technology

[0002] With the development of artificial intelligence and 5G technologies, artificial intelligence has made outstanding contributions in many fields and has a huge market in document entry. Long hours of tedious information entry may lead to data entry staff fatigue and errors. At this time, OCR custom form recognition algorithms are needed. They can automatically recognize table information in images and automatically generate the corresponding correct information to fill in the table.

[0003] Current OCR form recognition algorithms on the market cannot accurately fill in the recognized text content into the correct position for tables of variable length or dynamically changing tables, resulting in a large number of incorrectly entered content in the output form. Summary of the Invention

[0004] The purpose of this invention is to provide a form recognition method, device, and related components based on computer vision algorithms, aiming to solve the problem that existing form recognition algorithms have poor applicability to forms of variable length or dynamically changing forms.

[0005] To solve the above-mentioned technical problems, the objective of this invention is achieved through the following technical solution: providing a form recognition method based on computer vision algorithms, comprising:

[0006] Receive the target image and use OCR text recognition technology to perform text recognition on the target image to obtain all text content;

[0007] All the text content is filtered, and the text content that meets the preset merging rules is merged to obtain the merged text content;

[0008] Based on the received template matching information, obtain the position information and recognition direction information of each field name;

[0009] Based on the recognition direction information corresponding to the current field name, match the preset sorting rules;

[0010] Based on the sorting rules, all merged text content is sorted to obtain a text content set;

[0011] Based on the location information of the field name and the template matching information, the target text content is obtained from the text content set;

[0012] Iterate through all field names, retrieve the corresponding target text content, and then output the target form.

[0013] Furthermore, the technical problem to be solved by the present invention is to provide a form recognition device based on a computer vision algorithm, which includes:

[0014] The text recognition unit is used to receive the target image and use OCR text recognition technology to recognize the text in the target image to obtain all text content;

[0015] The merging unit is used to filter all the text content, merge the text content that meets the preset merging rules, and obtain the merged text content.

[0016] The information acquisition unit is used to acquire the location information and recognition direction information of each field name based on the received template matching information;

[0017] The matching unit is used to match a preset sorting rule based on the recognition direction information corresponding to the current field name;

[0018] A sorting unit is used to sort all the merged text content according to the sorting rules to obtain a text content set;

[0019] The target acquisition unit is used to acquire target text content from the text content set based on the location information of the field name and the template matching information;

[0020] The output unit is used to iterate through all field names, retrieve the corresponding target text content, and then output the target form.

[0021] In addition, this embodiment of the invention provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the form recognition method based on computer vision algorithm described in the first aspect above.

[0022] In addition, embodiments of the present invention also provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the form recognition method based on computer vision algorithms described in the first aspect above.

[0023] This invention discloses a form recognition method, apparatus, and related components based on computer vision algorithms. The method includes: receiving a target image and performing text recognition on the target image using OCR text recognition technology to obtain all text content; filtering all the text content and merging text content that meets preset merging rules to obtain merged text content; obtaining the position information and recognition direction information of each field name based on received template matching information; matching a preset sorting rule based on the recognition direction information corresponding to the current field name; sorting all merged text content based on the sorting rule to obtain a text content set; obtaining target text content from the text content set based on the position information of the field names and the template matching information; traversing all field names and obtaining the corresponding target text content, and then outputting the target form. This method can automatically match the text content of the uploaded target image, ensuring that the output target form matches the content in the target image, thus improving input efficiency and accuracy. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating the form recognition method based on computer vision algorithms provided in an embodiment of the present invention.

[0026] Figure 2 This is a schematic diagram of the form structure in the form recognition method based on computer vision algorithms provided in an embodiment of the present invention;

[0027] Figure 3 A schematic diagram of the structure of the target image in the form recognition method based on computer vision algorithm provided in an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram of the structure of the text content identified in the form recognition method based on computer vision algorithms provided in an embodiment of the present invention;

[0029] Figure 5 A schematic block diagram of a form recognition device based on computer vision algorithms provided in an embodiment of the present invention;

[0030] Figure 6 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0033] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0034] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0035] Please see Figure 1 , Figure 1 This is a flowchart illustrating the form recognition method based on computer vision algorithms provided in an embodiment of the present invention.

[0036] like Figure 1 As shown, the method includes steps S101 to S107.

[0037] S101. Receive the target image and use OCR text recognition technology to perform text recognition on the target image to obtain all text content;

[0038] In this embodiment, the target image can be captured by a device with a camera function, such as a mobile phone. After receiving the target image sent by the device, the terminal of this application recognizes the text in the target image using OCR text recognition technology, thereby obtaining all the text content in the target image. It should be noted that the table layout in the target image should be consistent with the table format pre-established on the terminal, that is, the template matching information of the table needs to be pre-entered on the terminal.

[0039] The template matching information includes field names, recognition direction information, recognition quantity information, stop key attributes, category information, and table attribute values. Specifically, the field names are unique within the table, for example... Figure 2 As shown, the field names include the renting unit, repair unit, serial number, equipment name, specifications, and manufacturer. The recognition direction information corresponding to the field name indicates the direction of the target text content relative to that field name, including rightward or downward recognition. For example, recognizing "renting unit" to the right yields "Mining Equipment Leasing Branch (Shuangliu Coal Mine)," while recognizing "serial number" downward yields "1." The recognition quantity information indicates how many results are matched in the corresponding direction based on the field name. For example, "renting unit" matches one result to the right based on "rightward recognition," and "serial number" matches one result downward based on "downward recognition" (automatically determining the quantity of target text content). The stop key attribute indicates when the terminal stops outputting text after recognizing the stop key. For example, "serial number" stops outputting text after recognizing "equipment damage status." Finally, the category information indicates the type of text content corresponding to the field name, such as "serial number." The target field for the field name is an integer type, while the target field for the "Device Name" field is a string type. The table attribute value indicates whether the current field name belongs to a table type and defines the table name. For example, if the serial number, device name, specifications, and manufacturer belong to the same table, it can be defined as table1 for later use.

[0040] It should be noted that the specific content and design specifications of the form can also be different during the setup process; this application's... Figure 2 This is only used as an example, so it will not be elaborated further.

[0041] In one specific embodiment, before step S101, the following steps are included:

[0042] S10. Identify all frame frames in the target image and calculate the perimeter data of each frame frame. Use the frame frame corresponding to the perimeter data that meets the conditions as the target contour.

[0043] S11. Perform perspective transformation on the target contour.

[0044] In this embodiment, since the target image captured by the device contains a table with table borders, these borders need to be made transparent to obtain the following result: Figure 4 The diagram shown illustrates how this method facilitates the recognition of text in a target image using OCR text recognition technology.

[0045] S102. Filter all the text content, merge the text content that meets the preset merging rules, and obtain the merged text content.

[0046] In this embodiment, step S102 includes the following steps:

[0047] S20. After obtaining the center point coordinates of adjacent text content, calculate the text spacing between the adjacent text content.

[0048] It should be added that the target image is composed of multiple pixels, with the position of each pixel serving as its corresponding coordinate. The text content is composed of, for example,... Figure 3 The diagram shown has each text content as an independent diagram. For example, the initially identified text content is as follows: "The bevel gear of the cutter head reducer of the tunneling machine is damaged, the cutter head teeth and seat are severely worn, and the cutter head needs to be replaced"; "The cutter head needs to be replaced; the track plates are severely worn; the tail rollers and scraper sprockets of the intermediate conveyor are severely worn and the drive wheels need to be replaced"; "Some oil pipes and joints in the hydraulic system of the tunneling machine are leaking oil"; "The spline of the rotating disk of one side of the rake claw is severely worn and the rotating disk needs to be replaced". However, in the actual content of the table, these four text contents should belong to the same sentence, but they are separated into different lines. Therefore, this application needs to merge this part of the text content.

[0049] In this embodiment, the coordinates of the middle pixel on the left and the middle pixel on the right of the current frame image are obtained. The values ​​of the horizontal coordinates of the two coordinates are added together and the average is calculated to obtain the coordinates of the center point of the current frame image. After obtaining the coordinates of the center points of two adjacent text contents, the difference between the horizontal coordinates and the difference between the vertical coordinates of the two center points are calculated to obtain the corresponding text spacing.

[0050] S21. After determining that the text spacing is less than the preset text spacing threshold, determine whether the center x-coordinates of the two center point coordinates are equal; if the center x-coordinates of the two center point coordinates are equal, then execute step S22; if the center x-coordinates of the two center point coordinates are not equal, then obtain the coordinates of the left endpoint corresponding to the adjacent text content, and then execute step S23.

[0051] In this embodiment, if the text spacing is less than the preset text spacing threshold, it means that the distance between the two text contents is relatively close. Therefore, it can be determined that the two text contents belong to the same text contents, but are on different lines. Specifically, it is determined whether the horizontal coordinates of the center points of the two adjacent text contents are equal. If they are equal, it can be directly determined that the two adjacent text contents belong to the same text contents. Then, the two box diagrams are merged into one box diagram to represent that they belong to the same text contents.

[0052] S22. Merge the adjacent text content into one text content, and update the coordinates of the center point;

[0053] It should be noted that since two box diagrams are merged into one box diagram, that is, two text contents are merged into one text content, the center point coordinates of the merged text content need to be updated.

[0054] S23. Calculate the alignment difference between the left endpoint x-coordinates of the two left endpoint coordinates, and proceed to step S24;

[0055] In actual operation, in addition to the layout format of center alignment, adjacent text content may also have a layout format of left alignment. This will result in the x-coordinates of the two center points belonging to the same text content being different. Therefore, this application obtains the x-coordinates of the coordinates of the upper left corner endpoints of the two frame diagrams and calculates the alignment difference between the two x-coordinates.

[0056] S24. After determining that the alignment difference is less than the alignment threshold, merge the adjacent text content into one text content and update the center point coordinates.

[0057] In this embodiment, by comparing the alignment difference and the alignment threshold, it can be determined whether two adjacent texts belong to the same text content. If they do, the two frame diagrams are merged into one frame diagram, and the center point coordinates are updated.

[0058] In this embodiment, adjacent text content is judged according to the following formula:

[0059]

[0060] in, This represents the x-coordinate of the top-left endpoint of one of the block diagrams. This represents the x-coordinate of the top-left endpoint of another block diagram. This represents the ordinate of the center point of one of the box diagrams. This represents the ordinate of the center point coordinates of another box diagram. This represents the x-coordinate of the center point of one of the box diagrams. Let represent the x-coordinate of the center point of the other bounding box, w represent the width of the target image, and w represent the difference between the x-coordinates of the top-left and top-right corners of the target image. This indicates the alignment threshold and text spacing threshold.

[0061] In one specific embodiment, after step S102:

[0062] S30. Obtain the center point coordinates of adjacent text content, wherein the center point coordinates include the center x-coordinate and the center y-coordinate;

[0063] S31. Calculate the width difference between the two center point coordinates, and after determining that the width difference is less than the width threshold, unify the center horizontal coordinates of the two center point coordinates.

[0064] S32. Calculate the height difference between the two center point coordinates, and after determining that the height difference is less than the height threshold, unify the center ordinate of the two center point coordinates.

[0065] In this embodiment, due to the tilt of the image during the shooting process, the text content will be displaced. For example, the center coordinates of the field name "Renting Unit" are (50, 50), and the center coordinates of the field name "Repair Unit" are (200, 45). Therefore, the field names "Renting Unit" and "Repair Unit" are not at the desired horizontal position. Thus, the position of all text content needs to be corrected using the following formula:

[0066]

[0067]

[0068] Where h represents the difference between the ordinates of the top-left and bottom-left corners of the target image, as stated in the above formula. This represents the width threshold, as stated in the above formula. This indicates the height threshold.

[0069] Using the above formula, the center point coordinates of the field name "Repair Unit" are updated from (200, 45) to (200, 50). In other words, this embodiment can compare and correct all text content, ensuring that the difference in the x-coordinates of all center point coordinates is less than the width. The x-coordinate of the center point of all text content is made the same. Similarly, the difference between the y-coordinates of all center point coordinates is less than the height. The center point y-coordinate of the text content is made the same to improve the accuracy of subsequent sorting rules.

[0070] S103. Based on the received template matching information, obtain the position information and recognition direction information of each field name, wherein the recognition direction information includes right recognition direction information and downward recognition direction information;

[0071] In this embodiment, the template matching information includes pre-input field names, recognition direction information, recognition quantity information, stop key attributes, category information, and table attribute values. Among them, the field name includes its position information in the form, that is, it carries the center point coordinate information representing its position. In this way, the center point coordinates of the recognized text content can be matched with the center point coordinates in the template matching information, so as to know that the field names in the recognized text content, such as "renting unit", "repair unit", "serial number", etc., can be matched with the field name positions in the target form.

[0072] S104. Based on the recognition direction information corresponding to the current field name, match the preset sorting rules;

[0073] S105. Based on the sorting rules, sort all the merged text content to obtain a text content set;

[0074] In this embodiment, step S105 includes the following steps:

[0075] S40. Obtain the center coordinates of all text content in order from left to right;

[0076] S41. Based on the values ​​of the x-coordinates of the center points in all center point coordinates, arrange the corresponding text content in ascending order to obtain a set of text content to the right. When it is determined that the x-coordinates of the center points are the same, obtain the values ​​of the y-coordinates of the center points in the corresponding center point coordinates and sort the corresponding text content in ascending order.

[0077] In this embodiment, for ease of understanding, the center point coordinates of "Opinion of the Leader in Charge of the Rental Unit" are set to (45, 250), the center point coordinates of "Serial Number" are (48, 80), the center point coordinates of "1" are (48, 100), the center point coordinates of "Renting Unit" are (50, 50), the center point coordinates of "Equipment Name" are (60, 80), the center point coordinates of "Tunneling Machine" are (60, 80), the center point coordinates of "Mining Equipment Leasing Branch (Shuangliu Coal Mine)" are (70, 50), and the center point coordinates of "Specifications and Manufacturer" are (70, 80).

[0078] After locating the current field name "Renting Unit" in the form, based on the template matching information, it is known that the current field name "Renting Unit" is identified from the right. Therefore, the remaining field names are sorted from left to right according to the value of the horizontal coordinate of the center point in the center point coordinate system, resulting in the following order: Opinion of the Leader in Charge of Renting Unit, Serial Number / 1, Renting Unit, Equipment Name / Tunneling Machine, Mining Equipment Leasing Branch (Shuangliu Coal Mine) / Specifications and Manufacturer. It can be seen that since the horizontal coordinates of Serial Number / 1, Equipment Name / Tunneling Machine, and Mining Equipment Leasing Branch (Shuangliu Coal Mine) / Specifications and Manufacturer are equal, sorting by horizontal coordinate is not in any particular order. Therefore, this application re-sorts the above order based on the vertical coordinate, resulting in the following sorting order: Opinion of the Leader in Charge of Renting Unit, Serial Number, 1, Renting Unit, Equipment Name, Tunneling Machine, Mining Equipment Leasing Branch (Shuangliu Coal Mine), Specifications and Manufacturer.

[0079] S42. Obtain the coordinates of the center point of all text content in top-to-bottom order;

[0080] S43. Based on the values ​​of the ordinates of the center points in all center point coordinates, arrange the corresponding text content in ascending order to obtain a set of text content below. When it is determined that the ordinates of the center points are the same, obtain the values ​​of the abscissas of the corresponding center points and sort the corresponding text content in ascending order.

[0081] Similarly, after locating the position of the current field name "Serial Number" in the form, according to the template matching information, the recognition direction information of the current field name "Serial Number" is that the current field name "Renting Unit" is recognized downwards. Therefore, in the order from top to bottom, according to the value of the vertical coordinate of the center point in the center point coordinate, all the remaining field names are sorted (the equipment name above the serial number, the mining equipment rental branch (Shuangliu Coal Mine), the repair unit, and Shanxi Fenxi Huayi Industrial Co., Ltd. have been entered into the form and will be deleted from the text content set), resulting in the following order: 1 / Tunneling machine / EBZ-260H Zhongchuan Heavy Industry / Original Baijia also ground storage, repaired 43 (4) 07 bottom extraction The vertical coordinates of the center point coordinates of the above field name “1 / tunneling machine / EBZ-260H Zhongchuan Heavy Industry / formerly Baijia also stored on the ground, repaired 43 (4) 07 bottom extraction tunnel use / unit / 1 / 748.3 million / 150 million / 8 years” are the same, so it is necessary to judge the horizontal coordinates of the center point coordinates of the above field name again, so as to obtain the latest set of downward text content: 1, tunneling machine, EBZ-260H Zhongchuan Heavy Industry, formerly Baijia also stored on the ground, repaired 43 (4) 07 bottom extraction tunnel use, unit, 1, 748.3 million, 150 million / 8 years, equipment damage situation…; it can be seen that since the vertical coordinates of the center point coordinates of the above field name “1 / tunneling machine / EBZ-260H Zhongchuan Heavy Industry / formerly Baijia also stored on the ground, repaired 43 (4) 07 bottom extraction tunnel use, unit, 1, 748.3 million, 150 million / 8 years, equipment damage situation…” are the same, so it is necessary to judge the horizontal coordinates of the center point coordinates of the above field name again, so as to obtain the latest set of downward text content: 1, tunneling machine, EBZ-260H Zhongchuan Heavy Industry, formerly Baijia also stored on the ground, repaired 43 (4) 07 bottom extraction tunnel use, unit, 1, 748.3 million, 150 million / 8 years, equipment damage situation…

[0082] S106. Based on the position information of the field name and the template matching information, obtain the target text content from the text content set;

[0083] In this embodiment, step S106 includes the following steps:

[0084] S50. Based on the template matching information, obtain the table attribute values ​​and the recognition quantity information carried by the field names;

[0085] It should be noted that, based on the table attribute values, it can be determined whether the current field name belongs to a table. In this application, a table refers to a regular table frame, such as the table frame containing the serial number, equipment name, specification serial number and manufacturer, original and repaired location, unit, quantity, original equipment value, repair price, years of use, and last major overhaul time, as well as the corresponding table frame where the target text content needs to be entered. Other table frames corresponding to field names such as rental unit, repair unit, and equipment damage status do not belong to a table.

[0086] S51. Based on the table attribute value, determine whether the current field name is a table field; if the current field name is not a table field, proceed to step S52; if the current field name belongs to a table, proceed to step S53.

[0087] In this embodiment, since the template matching information carries the table attribute value of the current field name, the table attribute value of the identified field name can be obtained, thereby determining whether the field name belongs to a table, i.e., whether it is a table field. For example, if the table attribute value of the rental unit is 0, then the rental unit does not belong to a table and is not a table field. If the table attribute value of the serial number is 1, then the serial number belongs to a table and is a table field.

[0088] S52. Obtain the recognition quantity information and recognition direction information carried by the current field name, and based on the recognition quantity information and recognition direction information, output the target text content from the corresponding text content set.

[0089] In this embodiment, for example, if the current field name is "renting unit", and the identification quantity information is 1 target text content, and the identification direction information is to the right, then a set of text content to the right is obtained based on the identification direction information. Then, from the set of text content to the right, the first text content that is equal to the horizontal coordinate of the center point of the current field name is selected, which is "mining equipment leasing branch (Shuangliu Coal Mine)". For example, if the current field name is "equipment name", and the identification quantity information is 1 target text content, and the identification direction information is downward, then a set of text content to the downward is obtained based on the identification direction information. Then, from the set of text content to the downward is the first text content that is equal to the vertical coordinate of the center point of the current field name, which is "tunneling machine".

[0090] For example, if the recognition quantity information of the current field name "Serial Number" is 1 and the recognition direction information is downward, then the text content represented by the center ordinate closest to the center ordinate of the previous field name "Serial Number" is obtained from the downward text content set, which is the text content "1". The text content "1" is in the order of the following (because other text content with the same center ordinate as text content "1" is sorted after text content "1" according to the sorting rules, and text content "1" is deleted from the downward text content set and the rightward text content set after the target text content of the current field name "Serial Number" is output).

[0091] Similarly, if the recognition quantity information of the current field name "Internal Repair Budget Price" is 1 and the recognition direction information is to the right, then the text content represented by the center horizontal coordinate that has the same center vertical coordinate as the current field name "Internal Repair Budget Price" and is closest to the center horizontal coordinate of the previous field name "Serial Number" can be obtained from the right text content set, that is, the text content "1.5 million".

[0092] S53. Determine whether the current table field carries a stop key attribute. If the current table field carries a stop key attribute, proceed to step S54; if the current table field does not carry a stop key attribute, proceed to step S56.

[0093] S54. Based on the stop key attribute and the recognition direction information carried by the current table field, output all target text content, and based on the recognition direction information, obtain the coordinates of the center point coordinates of the last output target text content and use them as the boundary threshold. Determine whether the boundary threshold is equal to the boundary threshold of the previous table field. If they are not equal, proceed to step S55.

[0094] S55. Update the boundary thresholds of the current table fields based on the update rules;

[0095] In this embodiment, to improve recognition efficiency, this application uses table fields for the same table, such as the first table, including fields like "Serial Number," "Equipment Name," "Specifications and Manufacturer," etc., and "Last Overhaul Time." The "Serial Number" field is the first field name in this table, and therefore has a stop-key attribute (the stop-key attribute represents the center coordinate point of the equipment damage status). Since the recognition direction of the "Serial Number" field is downward, text content with the same center horizontal coordinate as the "Serial Number" field is obtained from the downward text content set, and the corresponding target text content, i.e., the target text content "1," is output according to the order of its center vertical coordinate. It should be noted that the center vertical coordinate of the target text content is smaller than the position of the stop-key attribute; that is, the center vertical coordinate of the field name "1" should be smaller than the center vertical coordinate of the field name "Equipment Damage Status."

[0096] After outputting the field name "1", the center ordinate of the center point coordinate of the field name "1" is used as the boundary threshold. Then, in the recognition process of the next table field "Equipment Name", the stop key attribute of the table field "Equipment Name" is the field name "Equipment Damage Status", so the corresponding target text content is "Tunneling Machine". The center point ordinate of the target text content "Tunneling Machine" is equal to the center point ordinate of the previous field name "1", so the boundary thresholds of the two field names are the same and there is no need to update the boundary threshold.

[0097] If the table field "Equipment Name" corresponds to two field names (e.g., "First Tunneling Machine" and "Second Tunneling Machine"), the target text content "First Tunneling Machine" and "Second Tunneling Machine" will be output. At this time, the center point coordinates of the target text content "Second Tunneling Machine" are used as the new boundary threshold. The center point coordinates of the previous target text content "1" are discarded (because during the form recognition process, the target text content corresponding to "Serial Number" may be missed, that is, only "1" is recognized and "2" is not recognized). Therefore, in this way, the subsequent field names can be entered quickly.

[0098] In short, when the recognition direction is rightward, it checks whether the center_x (center x-coordinate) of the current table field's center point is greater than the boundary threshold X (center x-coordinate of the previous table field's center point). If it is greater, the output of the target text content is stopped. When the recognition direction is downward, it checks whether the center_y (center y-coordinate) of the current table field's center point is greater than the boundary threshold X (center y-coordinate of the previous table field's center point). If it is greater, the output of the target text content is stopped.

[0099] S56. Obtain the boundary threshold of the previous table field, and output all target text content based on the boundary threshold of the previous table field.

[0100] In this embodiment, if the current table field does not carry a stop key attribute, the boundary threshold of the previous table field is obtained, and all target text content is output based on the boundary threshold of the previous table field.

[0101] S107. After traversing all field names and obtaining the corresponding target text content, output the target form.

[0102] In one specific embodiment, the form recognition method based on computer vision algorithms provided by this invention further includes the following steps:

[0103] S108. Based on the template matching information, obtain the category information of the current field name;

[0104] S109. Determine whether the target text content corresponding to the current field name matches the category information. If it does not match, delete the target text content.

[0105] In this embodiment, after obtaining the target text content corresponding to the current field name, it is determined whether the target text content conforms to the category information carried by the current field name. For example, if the category information is an integer type, but the obtained target text content is a string type, it indicates that the identified target text content is incorrect, so the target text content is deleted when generating the target form.

[0106] This application can automatically match the text content of the uploaded target image, so that the content of the output target form is the same as that in the target image, that is, it improves the input efficiency and reduces the cost of manual input.

[0107] This invention also provides a form recognition device based on a computer vision algorithm, which is used to execute any of the aforementioned embodiments of the form recognition method based on a computer vision algorithm. Specifically, please refer to... Figure 5 , Figure 5 This is a schematic block diagram of a form recognition device based on computer vision algorithms provided in an embodiment of the present invention.

[0108] like Figure 5 As shown, the form recognition device 500 based on computer vision algorithms includes:

[0109] The text recognition unit 501 is used to receive the target image and use OCR text recognition technology to perform text recognition on the target image to obtain all text content;

[0110] The merging unit 502 is used to filter all the text content, merge the text content that meets the preset merging rules, and obtain the merged text content.

[0111] The information acquisition unit 503 is used to acquire the position information and recognition direction information of each field name based on the received template matching information;

[0112] Matching unit 504 is used to match a preset sorting rule based on the recognition direction information corresponding to the current field name;

[0113] The sorting unit 505 is used to sort all the merged text content according to the sorting rules to obtain a text content set.

[0114] The target acquisition unit 506 is used to acquire target text content from the text content set based on the position information of the field name and the template matching information;

[0115] Output unit 507 is used to traverse all field names, obtain the corresponding target text content, and then output the target form.

[0116] This device only requires uploading an image, and it can automatically return a target form with matching information. It is cheaper than manual data entry, significantly reduces the difficulty of use, and has a recognition rate of >90%.

[0117] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0118] The aforementioned form recognition device based on computer vision algorithms can be implemented as a computer program, which can, for example, Figure 6 It runs on the computer device shown.

[0119] Please see Figure 6 , Figure 6 This is a schematic block diagram of a computer device provided in an embodiment of the present invention. The computer device 1100 is a server, which can be a standalone server or a server cluster composed of multiple servers.

[0120] See Figure 6 The computer device 1100 includes a processor 1102, a memory, and a network interface 1105 connected via a system bus 1101. The memory may include a non-volatile storage medium 1103 and internal memory 1104.

[0121] The non-volatile storage medium 1103 may store an operating system 11031 and a computer program 11032. When the computer program 11032 is executed, it causes the processor 1102 to execute a form recognition method based on a computer vision algorithm.

[0122] The processor 1102 provides computing and control capabilities to support the operation of the entire computer device 1100.

[0123] The internal memory 1104 provides an environment for the execution of the computer program 11032 in the non-volatile storage medium 1103. When the computer program 11032 is executed by the processor 1102, the processor 1102 can execute a form recognition method based on computer vision algorithms.

[0124] The network interface 1105 is used for network communication, such as providing data transmission. Those skilled in the art will understand that... Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device 1100 to which the present invention is applied. The specific computer device 1100 may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0125] Those skilled in the art will understand that Figure 6The embodiments of the computer device shown do not constitute a limitation on the specific configuration of the computer device. In other embodiments, the computer device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. For example, in some embodiments, the computer device may include only memory and a processor. In such embodiments, the structure and function of the memory and processor are different from those shown. Figure 6 The embodiments shown are consistent and will not be described again here.

[0126] It should be understood that, in this embodiment of the invention, the processor 1102 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0127] In another embodiment of the invention, a computer-readable storage medium is provided. This computer-readable storage medium may be a non-volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, wherein when executed by a processor, the computer program implements the form recognition method based on a computer vision algorithm according to an embodiment of the invention.

[0128] The storage medium is a physical, non-transient storage medium, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk, or any other physical storage medium capable of storing program code.

[0129] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0130] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A form recognition method based on computer vision algorithms, characterized in that, include: Receive the target image and use OCR text recognition technology to perform text recognition on the target image to obtain all text content; All the text content is filtered, and the text content that meets the preset merging rules is merged to obtain the merged text content; After using OCR text recognition technology to perform text recognition on the target image and obtain all text content, the process includes: obtaining the center point coordinates of adjacent text content, wherein the center point coordinates include a center x-coordinate and a center y-coordinate; calculating the width difference between two center point coordinates, and after determining that the width difference is less than a width threshold, unifying the center x-coordinates of the two center point coordinates; calculating the height difference between two center point coordinates, and after determining that the height difference is less than a height threshold, unifying the center y-coordinates of the two center point coordinates. The step of filtering all the text content and merging the text content that meets the preset merging rules to obtain merged text content includes: after obtaining the center point coordinates of adjacent text content, calculating the text spacing of the adjacent text content; after determining that the text spacing is less than a preset text spacing threshold, determining whether the center x-coordinates of the two center point coordinates are equal; if the center x-coordinates of the two center point coordinates are equal, then the adjacent text content is merged into one text content, and the center point coordinates are updated; if the center x-coordinates of the two center point coordinates are not equal, then the left endpoint coordinates of the adjacent text content are obtained, and the alignment difference of the left endpoint x-coordinates of the two left endpoint coordinates is calculated; after determining that the alignment difference is less than the alignment threshold, the adjacent text content is merged into one text content, and the center point coordinates are updated. Based on the received template matching information, obtain the position information and recognition direction information of each field name; Based on the recognition direction information corresponding to the current field name, match the preset sorting rules; Based on the sorting rules, all merged text content is sorted to obtain a text content set; Based on the location information of the field name and the template matching information, the target text content is obtained from the text content set; Iterate through all field names, retrieve the corresponding target text content, and then output the target form.

2. The form recognition method based on computer vision algorithm according to claim 1, characterized in that, Before using OCR text recognition technology to perform text recognition on the target image to obtain all text content, the process includes: Identify all frame borders in the target image and calculate the perimeter data of each frame border, then use the frame borders corresponding to the perimeter data that meet the conditions as the target contour. Perform perspective transformation on the target contour.

3. The form recognition method based on computer vision algorithm according to claim 1, characterized in that, Based on the sorting rules, all merged text content is sorted to obtain a text content set, including: Obtain the center coordinates of all text content in order from left to right; Based on the values ​​of the x-coordinates of the center points in all center point coordinates, the corresponding text content is arranged in ascending order to obtain the set of text content to the right. When it is determined that the x-coordinates of the center points are the same, the values ​​of the y-coordinates of the corresponding center points are obtained, and the corresponding text content is sorted in ascending order. Obtain the center point coordinates of all text content in top-to-bottom order; Based on the values ​​of the ordinates of all center points, the corresponding text content is arranged in ascending order to obtain a set of text content below. When it is determined that there are identical ordinates of the center points, the values ​​of the x-coordinates of the corresponding center points are obtained, and the corresponding text content is sorted in ascending order.

4. The form recognition method based on computer vision algorithm according to claim 3, characterized in that, The method of obtaining target text content from the text content set based on the location information of the field name and the template matching information includes: Based on the template matching information, obtain the table attribute values ​​and the recognition quantity information carried by the field names; Based on the table attribute values, determine whether the current field name is a table field; If the current field name is a table field, then it continues to determine whether the current table field carries a stop key attribute. If the current table field carries a stop key attribute, then based on the stop key attribute and the recognition direction information carried by the current table field, all target text content is output, and based on the recognition direction information, the coordinates of the center point of the last output target text content are obtained and used as a boundary threshold. It is determined whether the boundary threshold is equal to the boundary threshold of the previous table field. If they are not equal, then the boundary threshold of the current table field is updated based on the update rule. If the current table field does not carry a stop key attribute, then the boundary threshold of the previous table field is obtained, and based on the boundary threshold of the previous table field, all target text content is output. If the current field name is not a table field, the recognition quantity information and recognition direction information carried by the current field name are obtained, and the target text content is output from the corresponding text content set based on the recognition quantity information and recognition direction information.

5. The form recognition method based on computer vision algorithm according to claim 4, characterized in that, Also includes: Based on the template matching information, obtain the category information of the current field name; Determine whether the target text content corresponding to the current field name matches the category information. If it does not match, delete the target text content.

6. A form recognition device based on computer vision algorithms, characterized in that, include: The text recognition unit is used to receive the target image and use OCR text recognition technology to recognize the text in the target image to obtain all text content; The merging unit is used to filter all the text content, merge the text content that meets the preset merging rules, and obtain the merged text content. After using OCR text recognition technology to perform text recognition on the target image and obtain all text content, the process includes: obtaining the center point coordinates of adjacent text content, wherein the center point coordinates include a center x-coordinate and a center y-coordinate; calculating the width difference between two center point coordinates, and after determining that the width difference is less than a width threshold, unifying the center x-coordinates of the two center point coordinates; calculating the height difference between two center point coordinates, and after determining that the height difference is less than a height threshold, unifying the center y-coordinates of the two center point coordinates. The step of filtering all the text content and merging the text content that meets the preset merging rules to obtain merged text content includes: after obtaining the center point coordinates of adjacent text content, calculating the text spacing of the adjacent text content; after determining that the text spacing is less than a preset text spacing threshold, determining whether the center x-coordinates of the two center point coordinates are equal; if the center x-coordinates of the two center point coordinates are equal, then the adjacent text content is merged into one text content, and the center point coordinates are updated; if the center x-coordinates of the two center point coordinates are not equal, then the left endpoint coordinates of the adjacent text content are obtained, and the alignment difference of the left endpoint x-coordinates of the two left endpoint coordinates is calculated; after determining that the alignment difference is less than the alignment threshold, the adjacent text content is merged into one text content, and the center point coordinates are updated. The information acquisition unit is used to acquire the location information and recognition direction information of each field name based on the received template matching information; The matching unit is used to match a preset sorting rule based on the recognition direction information corresponding to the current field name; A sorting unit is used to sort all the merged text content according to the sorting rules to obtain a text content set; The target acquisition unit is used to acquire target text content from the text content set based on the location information of the field name and the template matching information; The output unit is used to iterate through all field names, obtain the corresponding target text content, and then output the target form.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the form recognition method based on computer vision algorithm as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the form recognition method based on a computer vision algorithm as described in any one of claims 1 to 5.

Images