A template-based key-value pair extraction method and system
By calculating fixed and variable anchor points in the image and performing projection transformation correction, the problem of overly strict anchor points matching rules in the prior art is solved, the accuracy and robustness of image text recognition are improved, and a higher Template matching rate.
Patent Information
- Application Number
- CN202111191056.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-13
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-10-13
AI Technical Summary
The existing image text recognition algorithm based on template matching is too strict for anchor points matching rules and cannot tolerate the fault tolerance in text recognition, resulting in low recognition accuracy and matching rates.
By finding the bounding box of text information in the image, fixed anchor points and variable anchor points are calculated, and projection transformation correction is performed. It is suitable for correction and extraction of various text information, and is optimized by regular expressions and custom anchor points function. Anchor coordinates.
It improves the accuracy and robustness of image text recognition, improves the template matching rate, and is suitable for the correction and extraction of various text information.
Smart Images

Figure CN113869320B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a template-based key-value pair extraction method and system. Background Art
[0002] At present, after the text recognition of image data, it is necessary to extract key information from these texts according to business needs. There are many key information of image data that are related to their layout positions, and their relative positions are usually fixed. Therefore, a document template can be customized to specify the bounding box where each key information is located, and by determining the corresponding anchor point position in the actual image, a perspective transformation is performed so that the transformed actual image can fit the template. The existing image text recognition algorithms based on template matching have very strict matching rules for anchor points. The text must be completely fixed and unchanging, and no error tolerance is allowed in the text recognition of the actual image, otherwise the corresponding anchor point cannot be matched. The method of this patent expands the anchor point matching rules and optimizes the image recognition details after matching, which greatly improves the template matching rate and text recognition accuracy of the image text recognition model. Summary of the invention
[0003] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention provides a template-based key-value pair extraction method and system.
[0004] In order to achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0005] According to a first aspect of the present invention, a template-based key-value pair extraction method includes: S10, receiving an image, finding text information in the image using a text detection algorithm DB, and forming a bounding box around the text information; S20, finding keyword information in the image according to a pre-entered template, and calculating a fixed anchor point according to the keyword information and the coordinates of the bounding box; S30, calculating a variable anchor point according to the bounding box coordinates and the fixed anchor point; S40, performing a projection transformation correction on the image according to the fixed anchor point, the variable anchor point and the template; S50, identifying and extracting the text information in the corrected bounding box.
[0006] According to some embodiments of the present invention, step S10 may include step S1, wherein step S1 includes entering or deleting a template, wherein the template includes key layout and keyword information.
[0007] According to some embodiments of the present invention, step S20 further includes calculating the coordinates of the fixed anchor point according to a regular expression.
[0008] According to some embodiments of the present invention, step S20 further includes customizing a function of the fixed anchor point to optimize the coordinates of the fixed anchor point according to the coordinates of the bounding box.
[0009] According to some embodiments of the present invention, step S40 further includes step S41, wherein step S41 includes forming a bounding box around all the bounding boxes, and dividing the bounding box into a plurality of areas from top to bottom according to the template, each area including at least one key-value pair.
[0010] According to a second aspect of the present invention, a template-based key-value pair extraction system includes: a positioning module, which is used to receive an image, find text information in the image using a text detection algorithm DB, and form a bounding box around the text information; a fixed anchor point generation module, which finds keyword information in the image according to a pre-entered template, and calculates a fixed anchor point according to the keyword information and the coordinates of the bounding box; a variable anchor point calculation module, which calculates a variable anchor point according to the bounding box coordinates and the fixed anchor point; a correction module, which performs a projection transformation correction on the image according to the fixed anchor point, the variable anchor point and the template; and an information extraction module, which identifies and extracts the text information in the corrected bounding box.
[0011] According to some embodiments of the present invention, an input module is further included, and the input module includes an input or deletion template, and the template includes key layout and keyword information.
[0012] According to some embodiments of the present invention, the fixed anchor point generation module further includes calculating the coordinates of the fixed anchor point according to a regular expression.
[0013] According to some embodiments of the present invention, the fixed anchor point generation module further includes a function of customizing the fixed anchor point to optimize the coordinates of the fixed anchor point according to the coordinates of the bounding box.
[0014] According to some embodiments of the present invention, a key-value pair partitioning module is further included, which includes forming a bounding box around all the bounding boxes, and dividing the bounding box into a number of areas from top to bottom according to the template, each area including at least one key-value pair.
[0015] A template-based key-value pair extraction method and system according to an embodiment of the present invention have at least the following beneficial effects: projection transformation correction is performed according to anchor information based on keywords or regular expressions or custom anchor functions, which is suitable for correction and extraction of various text information, improves the accuracy of image text recognition, and the actual accuracy and robustness are better than the existing template key-value pair matching methods.
[0016] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0018] Figure 1 It is a flowchart of the present invention;
[0019] Figure 2 is a schematic diagram of forming a bounding box and an anchor point according to the present invention;
[0020] Figure 3 It is a schematic diagram of forming a bounding box according to the present invention. DETAILED DESCRIPTION
[0021] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the specific implementation method of the present invention is further described below in conjunction with the accompanying drawings.
[0022] The technical solutions in the embodiments of the present invention are described in full below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0023] The present invention provides a key-value pair extraction method based on a template, such as Figure 1 The flowchart of the present invention is shown. According to some embodiments, S10, receiving an image, finding text information in the image using a text detection algorithm DB, and forming a bounding box around the text information; S20, finding keyword information in the image according to a pre-entered template, and calculating a fixed anchor point according to the keyword information and the coordinates of the bounding box; S30, calculating a variable anchor point according to the bounding box coordinates and the fixed anchor point; S40, performing a projection transformation correction on the image according to the fixed anchor point, the variable anchor point and the template; S50, identifying and extracting the text information in the corrected bounding box.
[0024] Based on the above embodiment, an image is received, and text information is found in the image through the text detection algorithm DB and a bounding box is formed around the text information, and keyword information is found in the text of the bounding box. The bounding box is a rectangular bounding box, and the coordinates of the four points of the bounding box are extracted, and the coordinate position of the fixed anchor point is calculated around the keyword information according to the coordinates of the bounding box. Figure 2As shown in the figure, the fixed anchor point is a black dot set on the left side of the keyword information and is set at the midpoint of the short side of the bounding box. It is set on the left side and will not be affected by the length of the key information, such as the length of the specific name and the specific place name. The white dot on the right is a variable anchor point, which is calculated and positioned by the coordinates of the fixed anchor point and the coordinates of the bounding box, and is used to increase the number of anchor points and improve the accuracy of the projection transformation. The variable anchor point will change its coordinates according to specific key information, such as the length of the specific name and the specific place name. Through the fixed anchor point and the variable anchor point, the projection transformation matrix is calculated, and the bounding box of the original image is mapped to the template image for correction. After the image is corrected, the text information in the bounding box is identified. After the recognition is completed, if the text information has an incorrect expression, such as missing text, unrecognizable, and text recognition errors, it can be corrected manually and automatically through regular expressions and collected information in this article.
[0025] According to some embodiments, the step S10 further includes a step S1, wherein the step S1 includes entering or deleting a template, wherein the template includes key layout and keyword information.
[0026] Based on the above embodiments, based on the above embodiments, when there is unimportant text information in the image, such as human-written unimportant text information on a photocopy, after the photocopy image is photographed by a high-definition camera, the keyword information can be more accurately located based on the keyword information and key layout, and the key information can be found based on the keyword information, wherein the keyword information is "name", "gender", "birth", "year", "month", "day", etc., and the key information is the specific names and place names before and after the keyword information.
[0027] According to some embodiments, step S20 further includes calculating the coordinates of the fixed anchor point according to a regular expression.
[0028] According to some embodiments, the step S20 further includes customizing a function of the fixed anchor point to optimize the coordinates of the fixed anchor point according to the coordinates of the bounding box.
[0029] Based on the above embodiment, the coordinate calculation method of the fixed anchor point also includes calculation according to a regular expression, which collects the coordinates of multiple located fixed anchor points, and calculates the coordinates of the unlocated fixed anchor points through the regular expression and the coordinates of the collected fixed anchor points. The user can also define a function of the fixed anchor point to calculate the coordinates of the fixed anchor point through the coordinates of the four points of the bounding box. Obtaining the coordinates of the anchor point in multiple ways is more practical.
[0030] According to some embodiments, step S40 further includes step S41, wherein step S41 includes forming a bounding box around all the bounding boxes, and dividing the bounding box into a plurality of regions from top to bottom according to the template, each region including at least one key-value pair.
[0031] Based on the above embodiments, Figure 3 As shown, the bounding box is divided into multiple key-value pairs from top to bottom, and the key-value pairs of each area in the bounding box are identified and extracted. After identification, the key-value pairs of each area are extracted, and each area is set as a line for easy distinction. The extraction results are: "name" and specific name are in one line, "gender" and specific gender, and "ethnicity" and specific ethnicity are in one line, and so on, extracted and stored in computer text.
[0032] On the basis of the above embodiments, this embodiment provides a text correction system based on a bounding box. According to some embodiments, it includes: a positioning module, which is used to receive an image, find text information in the image using a text detection algorithm DB, and form a bounding box around the text information; a fixed anchor point generation module, which finds keyword information in the image according to a pre-entered template, and calculates a fixed anchor point according to the keyword information and the coordinates of the bounding box; a variable anchor point calculation module, which calculates a variable anchor point according to the bounding box coordinates and the fixed anchor point; a correction module, which performs projection transformation correction on the image according to the fixed anchor point, the variable anchor point and the template; and an information extraction module, which identifies and extracts the text information in the corrected bounding box.
[0033] Based on the above embodiment, an image is received, and text information is found in the image through the text detection algorithm DB and a bounding box is formed around the text information, and keyword information is found in the text of the bounding box. The bounding box is a rectangular bounding box, and the coordinates of the four points of the bounding box are extracted, and the coordinate position of the fixed anchor point is calculated around the keyword information according to the coordinates of the bounding box. Figure 2 As shown in the figure, the fixed anchor point is a black dot set on the left side of the keyword information and is set at the midpoint of the short side of the bounding box. It is set on the left side and will not be affected by the length of the key information, such as the length of the specific name and the specific place name. The white dot on the right is a variable anchor point, which is calculated and positioned by the coordinates of the fixed anchor point and the coordinates of the bounding box, and is used to increase the number of anchor points and improve the accuracy of the projection transformation. The variable anchor point will change its coordinates according to specific key information, such as the length of the specific name and the specific place name. Through the fixed anchor point and the variable anchor point, the projection transformation matrix is calculated, and the bounding box of the original image is mapped to the template image for correction. After the image is corrected, the text information in the bounding box is identified. After the recognition is completed, if the text information has an incorrect expression, such as missing text, unrecognizable, and text recognition errors, it can be corrected manually and automatically through regular expressions and collected information in this article.
[0034] According to some embodiments, an input module is further included, and the input module includes an input or deletion template, and the template includes key layout and keyword information.
[0035] Based on the above embodiments, when there is unimportant text information in the image, such as unimportant text information handwritten on a photocopy, after taking a photo of the photocopy image with a high-definition camera, the keyword information can be more accurately located based on the keyword information and key layout, and the key information can be found based on the keyword information, where the keyword information is "name", "gender", "birth", "year", "month", "day", etc., and the key information is the specific names of people and places before and after the keyword information.
[0036] According to some embodiments, the fixed anchor point generation module further includes calculating the coordinates of the fixed anchor point according to a regular expression.
[0037] According to some embodiments, the fixed anchor point generation module further includes customizing a function of the fixed anchor point to optimize the coordinates of the fixed anchor point according to the coordinates of the bounding box.
[0038] Based on the above embodiment, the coordinate calculation method of the fixed anchor point also includes calculation according to a regular expression, and the user can also customize the function of the fixed anchor point to calculate the coordinates of the fixed anchor point through the coordinates of the four points of the bounding box. Obtaining the coordinates of the anchor point in multiple ways is more practical.
[0039] According to some embodiments, a key-value pair partitioning module is further included, which includes forming a bounding box around all the bounding boxes, and dividing the bounding box into several areas from top to bottom according to the template, each area including at least one key-value pair.
[0040] Based on the above embodiments, Figure 3 As shown, the bounding box is divided into multiple key-value pairs from top to bottom, and the key-value pairs of each area in the bounding box are identified and extracted. After identification, the key-value pairs of each area are extracted, and each area is set as a line for easy distinction. The extraction results are: "name" and specific name are in one line, "gender" and specific gender, and "ethnicity" and specific ethnicity are in one line, and so on, extracted and stored in computer text.
[0041] It is obvious to those skilled in the art that the present invention is not limited to the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the basic features of the present invention. Therefore, the embodiments should be regarded as exemplary and non-restrictive.
Claims
1. A template-based key-value pair extraction method, characterized in that: include: S10, receiving an image, finding text information in the image using a text detection algorithm DB, and forming a bounding box around the text information; S20, finding keyword information in the image according to a pre-entered template, and calculating a fixed anchor point according to the keyword information and the coordinates of the bounding box; S30, calculating a variable anchor point according to the bounding box coordinates and the fixed anchor point; S40, performing projection transformation correction on the image according to the fixed anchor point, the variable anchor point and the template; S50, identifying and extracting text information within the corrected bounding box; The step S10 also includes a step S1, wherein the step S1 includes inputting or deleting a template, wherein the template includes key layout and keyword information; The step S20 also includes calculating the coordinates of the fixed anchor point according to the regular expression; The step S20 further includes customizing a function of the fixed anchor point to optimize the coordinates of the fixed anchor point according to the coordinates of the bounding box; The step S40 further includes a step S41, wherein the step S41 includes forming a bounding box around all the bounding boxes, and dividing the bounding box into a plurality of regions from top to bottom according to the template, each region including at least one key-value pair; The fixed anchor point is set on the left side of the keyword information and at the midpoint of the short side of the bounding box. It is set on the left side and will not be affected by the length of the key information. The variable anchor point is calculated and positioned through the coordinates of the fixed anchor point and the coordinates of the bounding box, and is used to increase the number of anchor points and improve the accuracy of the projection transformation. The variable anchor point transforms its coordinates according to the length of the specific key information. The projection transformation matrix is calculated through the fixed anchor point and the variable anchor point, and the bounding box of the original image is mapped to the template image for correction.
2. A template-based key-value pair extraction system, characterized in that: include: A positioning module, used for receiving an image, finding text information in the image using a text detection algorithm DB, and forming a bounding box around the text information; A fixed anchor point generation module, which finds keyword information in the image according to a pre-entered template, and calculates a fixed anchor point according to the keyword information and the coordinates of the bounding box; A variable anchor point calculation module, which calculates a variable anchor point according to the bounding box coordinates and the fixed anchor point; A correction module, performing projection transformation correction on the image according to the fixed anchor point, the variable anchor point and the template; An information extraction module, identifying and extracting text information within the corrected bounding box; It also includes an input module, which includes an input or deletion template, and the template includes key layout and keyword information; The fixed anchor point generation module also includes calculating the coordinates of the fixed anchor point according to a regular expression; The fixed anchor point generation module further includes a function of customizing the fixed anchor point to optimize the coordinates of the fixed anchor point according to the coordinates of the bounding box; It also includes a key-value pair division module, which includes forming a bounding box around all the bounding boxes, and dividing the bounding box into a plurality of regions from top to bottom according to the template, each region including at least one key-value pair; The fixed anchor point is set on the left side of the keyword information and at the midpoint of the short side of the bounding box. It is set on the left side and will not be affected by the length of the key information. The variable anchor point is calculated and positioned through the coordinates of the fixed anchor point and the coordinates of the bounding box, and is used to increase the number of anchor points and improve the accuracy of the projection transformation. The variable anchor point transforms its coordinates according to the length of the specific key information. The projection transformation matrix is calculated through the fixed anchor point and the variable anchor point, and the bounding box of the original image is mapped to the template image for correction.
Citation Information
Patent Citations
Target text extraction method and device in certificate, equipment and readable storage medium
CN111126125A
Image recognition and information extraction method and device for standardized document
CN111353492A
Image correction method and device and computer readable storage medium
CN111783770A
Key information extraction method based on coordinate transformation
CN113269126A
Automatic key / value pair extraction from document images using deep learning
US10896357B1
