A form content extraction method and device based on layout information and constraint programming
Through the method based on layout information and constraint planning, the form content extraction is optimized, and the extraction difficulties caused by image quality and tilt are solved, the extraction accuracy and continuity are improved, and automated processing is realized.
Patent Information
- Application Number
- CN202410358945.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-27
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-03-27
AI Technical Summary
In the process of converting paper documents into electronic data, poor picture quality and tilt of pictures lead to difficulty in extracting form content, especially in the recognition of multi-line text, which leads to inaccurate and incoherent extraction.
Using a method based on layout information and constraint planning, the text block is identified through OCR, the anchor word and target area are determined, column division and row position constraint planning are carried out, and the text block branching process is optimized to improve the extraction accuracy. and continuity.
It greatly improves the accuracy of form content extraction and the consistency of text block branches, achieves the goal of automated processing of form content, and solves the problem of extraction difficulties under the influence of image quality and skew.
Smart Images

Figure CN118196816B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of form image processing, and in particular to a form content extraction method and device based on layout information and constraint planning. Background Art
[0002] With the development of science and technology, the electronicization and intelligentization of paper documents are becoming an increasingly important part of modern society. The development of the electronicization and intelligentization of paper documents has played a vital role in improving work efficiency, strengthening information security and promoting knowledge sharing.
[0003] However, in the process of converting paper documents into electronic data, due to data quality limitations or shooting technology, angle selection, and the tilt and preservation status of the paper document itself, the scanned images may be deformed and distorted, which will cause difficulties for the subsequent extraction of image content. Taking the common image form extraction as an example, first of all, it is necessary to select the form area in the image. Secondly, each line of form data may actually contain multiple text blocks. During the recognition process, the text blocks recognized by OCR need to be divided into lines. The commonly used line division method is to sort and traverse from left to right and from top to bottom according to the x-coordinate and y-coordinate of each text block. When the distance between the new text block and the latest line exceeds a certain threshold, it is a new line, otherwise it is the current line. All OCR-recognized text blocks are traversed in turn to divide the lines.
[0004] However, in the absence of obvious table lines or when the image is tilted or distorted, the positions of these text blocks may be mistakenly recognized as multiple lines of text, or the text originally in the same line may be recognized as the content of adjacent lines. This will cause the content before and after the text block to be incoherent, which will bring great difficulties to the subsequent form content extraction. Summary of the invention
[0005] The embodiment of the present application provides a form content extraction method and device based on layout information and constraint planning, which is used to solve the following technical problems: the existing extraction of image form content is easily affected by the quality of the collected images, and it is easy to misjudge and recognize multiple lines of text, making it difficult to accurately and coherently extract the form content.
[0006] The present application embodiment adopts the following technical solutions:
[0007] On the one hand, an embodiment of the present application provides a form content extraction method based on layout information and constraint planning, including: performing OCR recognition on a form image to be processed to determine a number of text blocks; according to the anchor words of the form image to be processed and based on the text blocks, performing area delineation and recognition on the form image to be processed to determine a target area containing text blocks; dividing the target area into columns related to the text blocks to determine the column position information of each text block; according to the layout information of the image to be processed and based on the column position information of the text blocks, performing matrix constraints on each text block under the row position to obtain a constraint planning result; wherein the constraint planning result is a constraint planning calculation result of the row position; through a preset objective function, scoring the constraint planning result to determine the row number information to which the text block belongs; based on the column position information and the row number information, extracting the form content information of the text block.
[0008] The embodiment of the present application considers the layout information of the entire form, and based on the description of the layout information of the entire form, the text block branch problem is modeled as a constraint programming problem. Then the form layout information is described by using the objective function and constraints, and it is integrated into the process of text block branching, solving the technical problem of inaccurate text block branching from a global perspective. At the same time, the objective function and constraints are introduced to describe the layout information of the form, which greatly improves the accuracy of form content extraction, improves the consistency and continuity of text block branching, and also achieves the goal of automatically processing form content.
[0009] In a feasible implementation, OCR recognition is performed on the form image to be processed to determine a number of text blocks, which specifically includes: obtaining the layout information of the form to be processed; wherein the layout information at least includes: anchor words, the number of form columns, the form single-line mode, the form multi-line mode and the form alignment; through the layout information, the text box of the form to be processed is marked to obtain the text box position information; OCR recognition is performed on the text box and the text content corresponding to the text box position information to obtain the several text blocks.
[0010] In a feasible implementation manner, according to the anchor word of the form image to be processed and based on the text block, the form image to be processed is region-delimited and identified to determine the target area containing the text block, specifically comprising: according to the position information of the anchor word, determining a text block set corresponding to the anchor word; wherein the anchor word includes header information in the form to be processed; region-delimiting and dividing the form area where the text block set is located to obtain a text block area to be divided into lines; and determining the text block area to be divided into lines as the target area of the form image to be processed.
[0011] In a feasible implementation, the target area is divided into columns related to the text blocks to determine the column position information of each of the text blocks, specifically including: identifying the anchor column position information of the anchor words in each target area; performing an approximation evaluation calculation on the center position information of each text block in the target area based on the anchor column position information to obtain an evaluation value between the anchor column position information and the center position information; if the evaluation value is greater than a first preset threshold, determining the text block under the center position information as a text block under the anchor column position information; and determining the anchor column position information as the column position information of the text block.
[0012] In a feasible implementation, before performing matrix constraints on each of the text blocks under the row position and obtaining the constraint planning result, the method also includes: taking the layout information as a reference benchmark, and through a preset first constraint coefficient matrix, performing classification constraints on the text blocks in each row of the column position information to determine basic constraints; wherein, the classification constraints are: there is at least one text block in each row and each text block can only be located in one row; through a preset second constraint coefficient matrix, performing row number value sorting constraints on the text blocks in the same column of the column position information to determine dynamic constraints; wherein, the sorting constraints are upper and lower dependency constraints on row number values; wherein, the matrix constraints are composed of the basic constraints and the dynamic constraints.
[0013] In a feasible implementation, according to the layout information of the image to be processed and based on the column position information of the text block, matrix constraints are performed on each text block under the relevant row position to obtain a constraint planning result, which specifically includes: through the basic constraints and the dynamic constraints, iterative optimization calculations are performed on each text block under the relevant constraint planning to obtain a constraint output result; if the constraint output result is a feasible constraint, the constraint output result is determined as the constraint planning result; if the constraint output result is an infeasible constraint, the dynamic constraint condition is updated and adjusted until the constraint output result is a feasible constraint.
[0014] In a feasible implementation, the constraint planning result is scored and evaluated through a preset objective function to determine the row number information to which the text block belongs, specifically including: constructing a dynamic objective function based on the layout information of the form image to be processed; through the dynamic objective function, the constraint planning result is scored and evaluated for the overall layout of the text block to obtain an overall layout score; wherein the overall layout of the text block is the distribution of the text block in the entire row and column; through the dynamic objective function, the constraint planning result is scored and evaluated for the local connection position between each text block to obtain a local connection score; wherein the local connection position between each text block is the position between adjacent text blocks; according to the overall layout score and the local connection score, the branch result information of the target area in the form image to be processed is output; and the numerical information in the branch result information is determined as the row number information to which the text block belongs.
[0015] In a feasible implementation, based on the overall layout score and the local connection score, the branch result information of the target area in the form image to be processed is output, specifically including: performing mean clustering analysis calculation on the overall layout score and the local connection score to obtain the maximum clustering output information; and determining the maximum clustering output information as the branch result information of the target area.
[0016] In a feasible implementation, based on the column position information and the row number information, the form content information of the text block is extracted, specifically including: based on the column position information and the row number information of each text block, traversing all text blocks in the form image to be processed; extracting all the text blocks one by one, and extracting the form content information corresponding to each row in the form image to be processed.
[0017] On the other hand, an embodiment of the present application also provides a form content extraction device based on layout information and constraint planning, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, so that the at least one processor can execute a form content extraction method based on layout information and constraint planning described in any of the above embodiments.
[0018] The present application provides a form content extraction method and device based on layout information and constraint planning. Compared with the prior art, the embodiments of the present application have the following beneficial technical effects:
[0019] The embodiment of the present application considers the layout information of the entire form, and based on the description of the layout information of the entire form, the text block branch problem is modeled as a constraint programming problem. Then the form layout information is described by using the objective function and constraints, and it is integrated into the process of text block branching, solving the technical problem of inaccurate text block branching from a global perspective. At the same time, the objective function and constraints are introduced to describe the layout information of the form, which greatly improves the accuracy of form content extraction, improves the consistency and continuity of text block branching, and also achieves the goal of automatically processing form content. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:
[0021] Figure 1 A flow chart of a form content extraction method based on layout information and constraint planning provided in an embodiment of the present application;
[0022] Figure 2 An overall flow chart of form image content extraction based on constraint programming provided in an embodiment of the present application;
[0023] Figure 3 A structural schematic diagram of a form content extraction device based on layout information and constraint planning provided in an embodiment of the present application. DETAILED DESCRIPTION
[0024] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.
[0025] It should be noted that form content extraction in the prior art is mainly divided into two categories:
[0026] 1) The image itself is corrected through perspective transformation and other methods, but the success rate of automated correction is low, the calculation efficiency is low, and the effect is not ideal.
[0027] 2) Reorder the text blocks recognized by OCR and reposition the recognized text blocks to restore the form content to the greatest extent. This is a popular method, such as iterative clustering or vertical merging of text boxes to separate text blocks and then extract the form content. However, the existing methods only use local position information, that is, only use the information between adjacent text blocks on the left and right or close to each other, and use greedy algorithms to separate or extract text blocks without considering the layout information of the entire form. As a result, the extraction result may be correct locally, but the separation result of the entire bill is wrong.
[0028] Furthermore, if there are no obvious table lines or the image itself is tilted or distorted, the text block position in the OCR recognition result may be mistakenly recognized as multiple lines of text, or the text in the same line may be recognized as the content of adjacent lines. This causes the content before and after the text block to be incoherent, making it difficult to extract the subsequent form content.
[0029] In addition, the existing secondary processing method using OCR results only uses local position information and does not consider the layout information of the entire form, resulting in partial correct extraction results but errors in the entire form. This limits the accuracy and continuity of form content extraction.
[0030] The present application embodiment provides a form content extraction method based on layout information and constraint planning, such as Figure 1 As shown, the form content extraction method based on layout information and constraint programming specifically includes steps S101-S106:
[0031] S101, performing OCR recognition on the form image to be processed to determine a number of text blocks.
[0032] Specifically, the layout information of a type of form can be defined according to the commonly used form layout information table, and then the layout information of the form to be processed can be obtained. The layout information at least includes: anchor words, form column number, form single-line mode, form multi-line mode and form alignment.
[0033] Furthermore, the text box of the form to be processed is marked by the anchor words in the layout information, that is, the header information in the form, and the text box position information is obtained to locate the form content that needs to be framed.
[0034] Furthermore, the text box and the text content corresponding to the text box position information are subjected to OCR recognition to obtain a number of text blocks. That is, the selected text box and the corresponding text content are uniformly extracted using OCR recognition technology to generate a number of incoherent text blocks.
[0035] In one embodiment, the layout information of a form is described as follows: a specific type of form layout information is described, specifically, the common header information (anchor words) of this type of form, the number of columns of the form, the mode of each row of the form (e.g., in a row, which column must have a value, which columns can be empty, which two columns must have a value or be empty at the same time, etc.), and the upper and lower row mode (e.g., whether there is a blank row between the upper and lower rows, whether the upper and lower rows can be merged into one row, etc.).
[0036] S102: According to the anchor words of the form image to be processed and based on the text block, the form image to be processed is region-defined and identified to determine a target region containing the text block.
[0037] Specifically, it is necessary to first determine the text block set corresponding to the anchor word based on the position information of the anchor word. That is, under the reference standard of the anchor word, identify several text blocks corresponding to each anchor word, that is, the text block set. Among them, the anchor words include the header information in the form to be processed. The anchor words include project name, quantity, unit, amount, remarks, etc., and can also be non-mandatory content in the quantity / unit column and remarks column.
[0038] Furthermore, the form area where the text block set is located is divided and delineated, that is, the form position needs to be determined based on the anchor words in the form layout information and the text block set recognized by OCR, and the text block to be divided is automatically delineated to obtain the text block area to be divided. Then, the text block area to be divided is determined as the target area of the form image to be processed.
[0039] In one embodiment, for example, in an outpatient expense list, the form area can be demarcated and divided according to anchor words such as outpatient items and amounts, and the corresponding text blocks therein are the set of text blocks in the text block area to be divided, and the divided text block area is determined as the target area of the form image to be processed.
[0040] S103: Divide the target area into columns of relevant text blocks, and determine the column position information of each text block.
[0041] Specifically, Figure 2 The overall flow chart of form image content extraction based on constraint programming provided in the embodiment of the present application is as follows: Figure 2 As shown, the anchor column position information of the anchor words in each target area is first identified.
[0042] Furthermore, based on the anchor column position information, the center position information of each text block in the target area is evaluated for proximity, and an evaluation value between the anchor column position information and the center position information is obtained. That is, the center position information of each text block is judged as a nearby point with the anchor column position information, and based on the calculated distance evaluation value, it is judged whether the text block belongs to the column where the anchor point is located.
[0043] In a feasible implementation, if the evaluation value is greater than the first preset threshold, the text block under the center position information is determined as the text block under the anchor column position information. Finally, the anchor column position information is determined as the column position information of the text block.
[0044] S104: According to the layout information of the image to be processed and based on the column position information of the text block, matrix constraints are applied to each text block under the row position to obtain constraint planning results, wherein the constraint planning results are constraint planning calculation results of the row position.
[0045] Specifically, it is necessary to first determine the number of rows in the target area of the image to be processed by the elbow rule of the K-means algorithm, and then use the layout information as a reference, and use the preset first constraint coefficient matrix to perform a classification constraint on the text blocks in each row of the column position information, that is, in the form of a constraint coefficient matrix, perform a classification constraint on the text blocks in each row of the column position information, and then determine the basic constraint conditions. Among them, the classification constraint is: each row has at least one text block and each text block can only be located in one row.
[0046] In one embodiment, Figure 2 As shown, an N*M matrix A can be constructed, where N is the number of rows and M is the number of text blocks to be divided into rows. If the mth text block is located in the nth row, then A[n,m]=1, otherwise A[n,m]=0.
[0047] Furthermore, the second constraint coefficient matrix is used to perform a sorting constraint on the row number values of the text blocks in the same column of the column position information to determine the dynamic constraint condition, wherein the sorting constraint is a top-bottom dependency constraint on the row number values.
[0048] As a feasible implementation method, matrix constraints are applied to each text block regarding the row position: they are divided into two categories: one is basic constraints: there is at least one text block in each row; the other is dynamic constraints: they are dynamically generated according to the properties of the text blocks to be divided into rows: for example, if there are multiple text blocks in the same column, their row numbers have a vertical dependency relationship and restrict each other.
[0049] Furthermore, through the basic constraints and dynamic constraints, each text block is subjected to iterative optimization calculation under the constraint programming to obtain the constraint output result. If the constraint output result is a feasible constraint, the constraint output result is determined as the constraint programming result. If the constraint output result is an infeasible constraint, the dynamic constraint condition is updated and adjusted until the constraint output result is a feasible constraint.
[0050] In one embodiment, Figure 2 As shown in the figure, after iterative optimization calculation under constraint programming, a feasible solution is obtained, that is, a feasible constraint. At the same time, there may be multiple or zero possibilities. Due to the generation of dynamic constraints in the process of setting constraints, there may be no solution. In this case, the constraint output result is an infeasible constraint, and the dynamic constraints need to be updated and adjusted, and the dynamic constraints need to be updated and solved again. After obtaining multiple feasible solutions, the objective function is used to select the optimal solution to obtain the constraint output result.
[0051] S105 . Score evaluation is performed on the constraint programming result through a preset objective function to determine the line number information to which the text block belongs.
[0052] Specifically, we first need to build a dynamic objective function based on the layout information of the form image to be processed.
[0053] Furthermore, the constraint programming result is evaluated by the dynamic objective function in terms of the overall layout of the text blocks, and the overall layout score is obtained. The overall layout of the text blocks refers to the distribution of the text blocks in the entire row and column.
[0054] Furthermore, the constrained planning result is evaluated by the dynamic objective function for the local connection position between each text block, and a local connection score is obtained, wherein the local connection position between each text block is the position between adjacent text blocks.
[0055] Furthermore, according to the overall layout score and the local connection score, the branch result information of the target area in the form image to be processed is output.
[0056] In one embodiment, the dynamic objective function is determined based on the layout information, and the objective functions of different layout forms can be adjusted. The dynamic objective function includes two parts: the overall layout score and the local connection score. The overall layout score is a score that considers the overall layout (whole row and whole column level): if there is (no) text block in a certain column of a certain row, add 5 points; or if the upper and lower rows are considered at the same time, there is a text block in a certain column of the upper row, and there is (no) text block in a certain column of the lower row, add 10 points, etc.; the local connection score is a consideration of the text block level. If two text blocks are adjacent and close (only the position information of the OCR recognition result can be used to determine that the two text blocks are in the same row), and the constraint programming is used to obtain that the two text blocks are also in the same row, then add 1 point. After the evaluation scores of the above two parts are formed in this way, the score evaluation score of each row in the target area can be output, that is, the corresponding branch result information.
[0057] Furthermore, the numerical information in the line branch result information is determined as the line number information to which the text block belongs: it is necessary to first perform mean clustering analysis and calculation on the overall layout score and the local connection score to obtain the maximum clustering output information. Then, the maximum clustering output information is determined as the line branch result information of the target area.
[0058] In one embodiment, it is necessary to use a dynamic objective function and based on a clustering algorithm, find the feasible constraint (feasible solution) with the highest score in the branch result information after constraint planning, and then use the branch output result with the highest score as the actual branch result of the final text block. Then, the number in the upper left corner of the branch text block is used as the line number to which the text block belongs, that is, the line number information to which the text block belongs.
[0059] S106: Extracting form content information of the text block based on the column position information and the row number information.
[0060] Specifically, based on the column position information and row number information of each text block, all text blocks in the form image to be processed are traversed.
[0061] Furthermore, all text blocks are extracted one by one, and the form content information corresponding to each row in the form image to be processed is further extracted, such as: project name, quantity / unit, amount, remarks, etc., to finally complete the content extraction of the entire form.
[0062] As a feasible implementation method, the invention aims to improve the accuracy of form content extraction by considering the layout information of the entire form and integrating it into the process of text block branching. Compared with the method of using only local position information, considering the layout information of the entire form can better restore the structure of the form, thereby improving the accuracy of the extraction results. This will provide a more reliable basis for subsequent form content analysis and processing. At the same time, it only needs to provide the description information of the form layout, redefine the objective function and constraints, and it can be quickly applied to the extraction of form content in the new layout.
[0063] In addition, the embodiment of the present application also provides a form content extraction device based on layout information and constraint planning, such as Figure 3 As shown, the form content extraction device 300 based on layout information and constraint programming specifically includes:
[0064] At least one processor 301. And a memory 302 in communication with the at least one processor 301. The memory 302 stores instructions that can be executed by the at least one processor 301, so that the at least one processor 301 can execute:
[0065] Perform OCR recognition on the form image to be processed and determine several text blocks;
[0066] According to the anchor words of the form image to be processed and based on the text block, the form image to be processed is region-delineated and identified to determine the target area containing the text block;
[0067] Divide the target area into columns of relevant text blocks and determine the column position information of each text block;
[0068] According to the layout information of the image to be processed and based on the column position information of the text block, matrix constraints are applied to each text block under the row position to obtain the constraint planning result. Among them, the constraint planning result is the constraint planning calculation result of the row position;
[0069] Through the preset objective function, the constraint programming results are scored and evaluated to determine the line number information to which the text block belongs;
[0070] Based on the column position information and the row number information, the form content information of the text block is extracted.
[0071] The example of this application considers the layout information of the entire form, and based on the description of the layout information of the entire form, the text block branch problem is modeled as a constraint programming problem. The form layout information is then described by using an objective function and constraints, and integrated into the process of text block branching, solving the technical problem of inaccurate text block branching from a global perspective. At the same time, the objective function and constraints are introduced to describe the layout information of the form, which greatly improves the accuracy of form content extraction, improves the consistency and continuity of text block branching, and also achieves the goal of automatically processing form content.
[0072] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0073] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0074] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the embodiments of the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present application should be included in the scope of the claims of the present application.
Claims
1. A form content extraction method based on layout information and constraint programming, characterized in that: The method comprises: Perform OCR recognition on the form image to be processed and determine several text blocks; According to the anchor word of the form image to be processed and based on the text block, the form image to be processed is region-delineated and identified to determine a target region containing the text block; Dividing the target area into columns of the text blocks and determining column position information of each text block specifically includes: Identify the anchor column position information of the anchor word in each target area; According to the anchor point column position information, an evaluation calculation is performed on the center position information of each text block in the target area to obtain an evaluation value between the anchor point column position information and the center position information; If the evaluation value is greater than a first preset threshold, the text block under the center position information is determined as a text block under the anchor column position information; Determine the anchor point column position information as the column position information of the text block; According to the layout information of the form image to be processed and based on the column position information of the text block, matrix constraints are performed on each text block under the row position to obtain a constraint planning result; wherein the constraint planning result is a constraint planning calculation result of the row position; The constraint programming result is scored and evaluated by a preset objective function to determine the line number information to which the text block belongs, specifically including: Based on the layout information of the form image to be processed, construct a dynamic objective function; Through the dynamic objective function, the constraint programming result is evaluated with respect to the overall layout of the text block to obtain an overall layout score; wherein the overall layout of the text block is the distribution of the text block in the entire row and column; The constraint programming result is evaluated by the dynamic objective function for the local connection position between each text block to obtain a local connection score; wherein the local connection position between each text block is the position between adjacent text blocks; Outputting the branch result information of the target area in the form image to be processed according to the overall layout score and the local connection score, specifically including: Performing mean clustering analysis calculation on the overall layout score and the local connection score to obtain maximum clustering output information; Determine the maximum information of the cluster output as the branch result information of the target area; Determine the numerical information in the line branch result information as the line number information to which the text block belongs; Based on the column position information and the row number information, form content information of the text block is extracted.
2. A form content extraction method based on layout information and constraint programming according to claim 1, characterized in that: Perform OCR recognition on the form image to be processed and identify several text blocks, including: Acquire the layout information of the form to be processed; wherein the layout information at least includes: anchor words, number of form columns, form single-line mode, form multi-line mode and form alignment; Using the layout information, the text box of the form to be processed is marked to obtain the text box position information; Perform OCR recognition on the text box and text content corresponding to the text box position information to obtain a plurality of text blocks.
3. The form content extraction method based on layout information and constraint programming according to claim 1, characterized in that: According to the anchor word of the form image to be processed and based on the text block, the form image to be processed is region-delineated and identified to determine the target region containing the text block, specifically including: According to the position information of the anchor word, a text block set corresponding to the anchor word is determined; wherein the anchor word includes header information in the form to be processed; Delimiting and dividing the form area where the text block set is located to obtain the text block area to be divided into lines; The text block area to be divided into lines is determined as the target area of the form image to be processed.
4. The form content extraction method based on layout information and constraint programming according to claim 1, characterized in that: Before performing matrix constraints on each text block under the row position to obtain the constraint programming result, the method further includes: Taking the layout information as a reference, and using a preset first constraint coefficient matrix, the text blocks in each row of the column position information are subjected to a classification constraint to determine a basic constraint condition; wherein the classification constraint is: there is at least one text block in each row and each text block can only be located in one row; By using a preset second constraint coefficient matrix, the text blocks in the same column in the column position information are sorted by row number values to determine a dynamic constraint condition; wherein the sorting constraint is a top-bottom dependency constraint of the row number values; The matrix constraint consists of the basic constraint condition and the dynamic constraint condition.
5. A form content extraction method based on layout information and constraint programming according to claim 4, characterized in that: According to the layout information of the form image to be processed and based on the column position information of the text block, matrix constraints are performed on each text block under the row position to obtain constraint planning results, which specifically include: Through the basic constraint conditions and the dynamic constraint conditions, an iterative optimization calculation under the constraint programming is performed on each text block to obtain a constraint output result; If the constraint output result is a feasible constraint, determining the constraint output result as the constraint planning result; If the constraint output result is an infeasible constraint, the dynamic constraint condition is updated and adjusted until the constraint output result is a feasible constraint.
6. The form content extraction method based on layout information and constraint programming according to claim 1, characterized in that: Extracting the form content information of the text block based on the column position information and the row number information specifically includes: Based on the column position information and row number information of each text block, traverse all text blocks in the form image to be processed; All the text blocks are extracted one by one, and the form content information corresponding to each row in the form image to be processed is extracted.
7. A form content extraction device based on layout information and constraint programming, characterized in that: The device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, so that the at least one processor can execute the form content extraction method based on layout information and constraint programming according to any one of claims 1-6.
