Streaming table identification method and system

Through the DBNet+CRNN and YOLOv7 model combining text semantics and character features, the accuracy of multi-layer bank statement table recognition is solved, the accurate identification of bank statement tables is realized and information restoration is restored, and verification and operation are improved. Convenience.

CN118968529AInactive Publication Date: 2024-11-15BEIJING RONGDAKEJI CO LTD

Patent Information

Application Number
CN202411038842.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing technology cannot effectively process bank statement forms of multiple layouts, with low accuracy and the inability to completely restore the information in the bank statement forms, resulting in difficulty in verification and operation.

Method used

The DBNet+CRNN algorithm is used to detect text information, and the YOLOv7 model locates the table, combines text semantics and character features for classification, and uses json format to output the row and column positions of text blocks to achieve accurate identification of multi-type bank statement tables.

Benefits of technology

It realizes accurate identification of bank statement forms in various types of bank statement forms, improves the accuracy and universality of bank statement forms identification, and can completely restore form information and simplify subsequent operations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968529A_ABST
    Figure CN118968529A_ABST
Patent Text Reader

Abstract

The invention discloses a flow table identification method and system, and belongs to the technical field of table identification, and the method specifically comprises the steps: obtaining a PDF file with a table, converting the PDF file with the table into an image, carrying out the correction of the image, respectively detecting the text, table and date information in the image, completing and grouping the detected date information, and carrying out the recognition of the flow table. The method comprises the following steps of: judging specific positions of a table in an image, including a header position and a mark body position, performing row division according to date information in the image, acquiring candidate column lines by using a column division technology, performing column division according to the candidate column lines in combination with text semantics and character features, and identifying a text block according to the row and the column to which the identified text block belongs; according to the method, each text block and the corresponding row and column positions are output in a json format, table recognition can be carried out for various formats, and table contents of various formats can be accurately recognized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of table recognition, and in particular to a method and system for recognizing a running table. Background Art

[0002] In the company's IPO process, bank statement verification is an important part. Generally, bank statements of the issuer, its subsidiaries, branches, and individuals are required. Since these bank statements are usually scanned from paper versions, it is impossible to directly obtain the bank statement information, which makes subsequent verification difficult. Therefore, before verification, it is necessary to perform text recognition on the bank statement, use table recognition technology to reconstruct the table, and obtain editable bank statement table information.

[0003] Since bank statements come from banks across the country, the formats of bank statements are diverse, including wired forms, wireless forms, inconsistent number of header rows, inconsistent number of content rows, etc., and the basic information of bank statements exported by each bank is also inconsistent.

[0004] For example, a Chinese patent with publication number CN114463763A discloses a bank statement table extraction method, device, computer equipment and storage medium, the method comprising: obtaining a bank statement image, detecting the text area in the bank statement image; merging the text area to construct a text line, and obtaining a text line set; traversing each text line in the text line set, extracting a statement header line and a statement body line set; dividing each text area in the statement body line set into columns according to the distance between the statement header line and each text area in the statement body line set, and determining a statement body column set; based on the consistency of the statement header and the statement body, verifying and updating the statement header line and the statement body column set, and saving the verified and updated statement header line and the statement body column set as the table extraction result. The use of this method can improve the accuracy of bank statement table extraction.

[0005] For example, a Chinese patent with publication number CN114187599A discloses an image text recognition method, device and electronic device suitable for bank statements, wherein the method includes: performing layout analysis on borderless flow templates in different bank statement templates to determine the width and length of the borderless flow template; determining the number of longitude and latitude lines to be added according to the width and length of the borderless flow template and the latitude and longitude values ​​between different rows and columns; generating a text box with the same number of longitude and latitude lines, and adding the text box to the borderless flow template to obtain a bordered flow template; and recognizing the image of the bordered flow template. The invention performs layout analysis on the borderless flow template, uses the latitude and longitude value projection conversion method to generate a chart text box, and effectively frames the bank statement text, thereby achieving a technical effect of avoiding manual labeling, and solving the problem of wasting manpower and time in bank statement recognition due to manual labeling training of a large amount of data.

[0006] The defects of the above patent are: 1) Using a borderless statement template to convert the borderless statement into a bordered statement for identification. This technology is limited by the borderless statement template and cannot meet the needs of multi-format bank statements. A customized template is required for each format; 2) Reconstructing the table only by distance cannot meet the needs of multi-format bank statements and the accuracy is not high. Summary of the invention

[0007] In view of the deficiencies in the prior art, the present invention proposes a bank statement form recognition method and system, which can process bank statements of various formats, such as wired forms, wireless forms, single-line headers, multi-line headers, single-line content, multi-line content and other formats. In addition, bank statements may have column adhesion problems. To address this problem, text semantics and character features are combined to assist in column separation, which can improve the accuracy of column separation and can completely restore all information in the bank statement form, which greatly facilitates subsequent verification and other operations.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] A method for identifying a flow table, comprising:

[0010] Get a PDF file with a table, convert the PDF file with a table into an image, and correct the image;

[0011] Detect text, table and date information in images respectively;

[0012] Improve and group the detected date information;

[0013] Determine the specific position of the table in the image, including the position of the table header and the position of the body;

[0014] Split the image into rows based on the date information in the image;

[0015] Use column separation technology to obtain candidate column lines, and then separate the columns according to the candidate column lines combined with text semantics and character features;

[0016] According to the row and column to which the identified text block belongs, each text block and the corresponding row and column position are output in JSON format.

[0017] Specifically, the steps of respectively detecting text, table and date information in an image include:

[0018] Use DBNet+CRNN algorithm to detect the image and obtain all text information and coordinate information of the text information in the image and the position information set of each character in the text, x min The minimum horizontal coordinate representing the text information, x max Indicates the maximum horizontal coordinate of the text information, y min Represents the minimum ordinate of text information, y max Indicates the maximum vertical coordinate of the text information;

[0019] Locate the table in the image and use the YOLOv7 model to locate the table part;

[0020] The date information in the image is detected to obtain the date information in the image.

[0021] Specifically, the step of improving and grouping the detected date information includes:

[0022] Regularly add or delete the date information in the image;

[0023] According to the date information after regular addition or deletion in the image and the detected table, the date information after regular addition or deletion in the image is grouped using the overlap rate.

[0024] Specifically, dividing the images into rows according to date information in the images includes:

[0025] When the content of a text line in the image is a single line, the text line is a table row;

[0026] When the content of a text line in an image is larger than one line, the logic for dividing the text line in the image is:

[0027] 1) Top-aligned branch logic: branch between two adjacent dates, with the minimum y coordinate of the first date 1min and the minimum ordinate y of the second date 2min The content between is a table row.

[0028] 2) Bottom-aligned branch logic: branch between two adjacent dates, with the maximum y coordinate of the first date 1max and the maximum ordinate y of the second date 2max The content between is a table row.

[0029] 3) Center-aligned branch logic: branch between two adjacent dates, and define the median ordinate of the first date as cur_y center , the median ordinate of the second date next_y center , half the distance between the first and second dates is half_dis = (cur_y center +next_y center ) / 2, half_dis represents half of the distance between the first and second dates, then in cur_y center +half_dis and next_y center +The content between half_dis is a table row;

[0030] The text lines in the image are divided into rows to obtain the table rows of the table in the image.

[0031] Specifically, the step of sorting the candidate columns according to the combination of text semantics and character features includes:

[0032] Get candidate column lines and determine the number of header rows in the current layout;

[0033] Candidate column lines are used, and text semantics and character features are combined for column segmentation. Each row is segmented using candidate column lines. If the candidate column line is between two text blocks, it is divided into left and right parts according to the candidate column line. If the candidate column line is in the middle of a text block, it is segmented according to the character position.

[0034] Specifically, if the candidate column line is in the middle of a text block, segmentation is performed according to character positions, including:

[0035] The scores of adjacent characters of the segmented text block, score = score_line + score_blank + score_dis, score_line represents the score of the candidate line position, score_blank represents the score of the space between adjacent characters, and score_blank represents the score of the space between adjacent characters;

[0036] Select the largest score and split between adjacent characters when score>0.5.

[0037] A flow table recognition system, which is used to implement the flow table recognition method, is characterized by comprising: an image acquisition module, an image detection module, a grouping improvement module, a position judgment module, a row division module, a column division module and a table output module;

[0038] The image acquisition module is used to acquire a PDF file with a table, convert the PDF file with a table into an image, and correct the image;

[0039] The image detection module is used to detect text, table and date information in the image respectively;

[0040] The perfecting and grouping module is used to perfect and group the detected date information;

[0041] The position determination module is used to determine the specific position of the table in the image, including the position of the table header and the position of the body;

[0042] The branching module is used to branch the images according to the date information in the images;

[0043] The column separation module is used to obtain candidate column lines using the column separation technology, and then perform column separation based on the candidate column lines in combination with text semantics and character features;

[0044] The table output module is used to output each text block and the corresponding row and column position using the JSON format according to the row and column to which the identified text block belongs.

[0045] Specifically, the branch module includes: a content single-line branch unit and a content multi-line branch unit.

[0046] The content single-line branching unit is used to branch the table rows when the content in the text row in the image is a single line;

[0047] The content multi-row branching unit is used to perform table row branching when the content in a text row in the image is larger than one row.

[0048] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a transaction table recognition method when executing the computer program.

[0049] A computer-readable storage medium stores computer instructions, which execute the steps of a flow table recognition method when the computer instructions are executed.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] 1. The present invention proposes a method for recognizing a running table, which has strong versatility and is not limited to the format, and can perform table recognition for various formats.

[0052] 2. The present invention proposes a bank statement form recognition method, which can accurately recognize the contents of forms of various formats, greatly improving the accuracy of bank statement form recognition.

[0053] 3. The present invention proposes a flow table recognition method, which can recognize all the information in the table in the image, facilitating subsequent operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A flow chart of a method for identifying a flow table is provided for the present invention;

[0055] Figure 2 Provide a coordinate definition diagram of a text box for the present invention;

[0056] Figure 3 Provides a rendering of the implementation effect of format 1 for the present invention;

[0057] Figure 4 Provides a rendering of the implementation effect of format 2 for the present invention;

[0058] Figure 5 Provides a rendering of the implementation effect of format 3 for the present invention;

[0059] Figure 6 Provides a rendering of the implementation effect of format 4 for the present invention;

[0060] Figure 7 Provides a rendering of the implementation effect of format 5 for the present invention;

[0061] Figure 8 Provides a rendering of the implementation effect of format 6 for the present invention;

[0062] Fig. 9 Provides a rendering of the implementation effect of format 7 for the present invention;

[0063] Fig.10 A diagram of the structure of a flow table recognition system provided by the present invention;

[0064] Fig.11 A schematic diagram of an electronic device provided by the present invention;

[0065] Fig.12 A schematic diagram of the computer-readable storage medium structure provided by the present invention. DETAILED DESCRIPTION

[0066] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, in the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate the orientation or position relationship based on the orientation or position relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "No. 1", "No. 2" and "No. 3" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. The present invention is further explained below in conjunction with specific embodiments.

[0067] Example 1

[0068] See also Figure 1-9 The present invention provides an embodiment: a method for identifying a flow table, comprising the following specific steps:

[0069] The bank statement form in the present invention is a bank statement form;

[0070] Step S1: obtaining a PDF file with a table, converting the PDF file with a table into an image, and correcting the image;

[0071] Because the image may be tilted, correction is beneficial for OCR and table recognition;

[0072] Step S2: Detecting text, table and date information in the image respectively;

[0073] The specific steps of step S2 are:

[0074] Step S201: Use DBNet+CRNN algorithm to detect the image and obtain all text information and coordinate information of the text information in the image and the position information set of each character in the text, x min The minimum horizontal coordinate representing the text information, x max Indicates the maximum horizontal coordinate of the text information, y min Represents the minimum ordinate of text information, y max Indicates the maximum vertical coordinate of the text information, divides the text into lines (from top to bottom, from left to right), and calculates the information required for the subsequent steps, including: calculating the average line height of the text lines in the image, text semantics, and character features. The calculation formula for the average line height of the text lines in the image is: Hg represents the average height of text lines in the image, n represents the number of text lines in the image, and y i,max Indicates the maximum vertical coordinate of the i-th line of text in the image, y i+1,minIndicates the minimum vertical coordinate value of the i-th line of text in the image;

[0075] Text semantics uses the BERT model to determine the segmentation point of a sentence. The model performs binary classification on each character of a sentence. If the classification is 0, it means that the character should not be segmented, and if the classification is 1, it means that the character can be segmented. After the segmentation is completed, the character features are formed;

[0076] Step S202: Locate the table in the image and use the YOLOv7 model to locate the bank statement table for subsequent table recognition. Although most of an image is a table, it is also necessary to find the specific location of the table to provide a complete table for subsequent row and column analysis;

[0077] Step S203: Detect the date information in the image to obtain the date information in the image, and then perform a line division operation based on the detected date information;

[0078] Step S3: perfecting and grouping the detected date information;

[0079] The specific steps of step S3 are:

[0080] Step S301: Regularly supplement or delete the date information in the image;

[0081] In this embodiment, since the detected date may be incomplete or misdetected, it is necessary to combine regular expressions to improve the date information; this part of the regular expression is designed by ourselves, which can meet the complete matching and fuzzy matching of basically all date formats, and can ensure the integrity of date search;

[0082] Step S302: according to the date information after regular supplementation or deletion in the image and the detected table, the date information after regular supplementation or deletion in the image is grouped using the overlap rate to solve the problem of multiple tables in a single image.

[0083] Step S4: determining the specific position of the table in the image, including the position of the table header and the position of the table body;

[0084] In this embodiment, if the header exists, the header is checked according to the header library. Since the OCR part may detect multiple headers as one text block or one header as multiple text blocks, the header needs to be correctly divided into blocks here, which can be used for subsequent column verification. However, if the header is not in the header library or does not exist, the header does not need to be checked. This part of the header verification is auxiliary to subsequent operations. It does not matter whether it exists or not. If it exists, the column division effect will be better for some structures and complex layouts. If it does not exist, the impact will not be too great.

[0085] Step S5: dividing the image into rows according to the date information in the image;

[0086] In this embodiment, the line here is a table row, not a text row. Each table row can have multiple text rows. For single-line content, the row where the date is located is a table row; for multi-line content, there are three types according to the position of the row where the date is located: the date row is at the top row of the multi-line content, which is called top alignment; the date row is at the bottom row of the multi-line content, which is called bottom alignment; the date row is in the middle row of the multi-line content, which is called center alignment;

[0087] The specific steps of step S5 are:

[0088] Step S501: when the content of a text line in the image is a single line, the text line is a table row;

[0089] Step S502: When the content of a text line in the image is larger than one line, the logic for dividing the text line in the image into lines is:

[0090] 1) Top-aligned branch logic: branch between two adjacent dates, with the minimum y coordinate of the first date 1min and the minimum ordinate y of the second date 2min The content between is a table row.

[0091] 2) Bottom-aligned branch logic: branch between two adjacent dates, with the maximum y coordinate of the first date 1max and the maximum ordinate y of the second date 2max The content between is a table row.

[0092] 3) Center-aligned branch logic: branch between two adjacent dates, and define the median ordinate of the first date as cur_y center , the median ordinate of the second date next_y center , half the distance between the first and second dates is half_dis = (cur_y center +next_y center ) / 2, half_dis represents half of the distance between the first and second dates, then in cur_y center +half_dis and next_y center +The content between half_dis is a table row;

[0093] The median ordinate of a date is (the maximum ordinate of a date + the minimum ordinate of a date) / 2;

[0094] Step S503: dividing the text lines in the image into rows to obtain table rows of the table in the image.

[0095] Judge each row of the results of the three branching methods according to the following rules, and analyze which branching method has the least unreasonable rows. The one with the least unreasonable rows is the most reasonable branching method, and this branching method should be selected.

[0096] In this embodiment, each table row is first divided into columns using horizontal projection (here is a simple column division, used to determine the rationality of the division, and compared with the subsequent column division, the logic here is very simple), and the main row is determined according to the number of blocks in the table row (the layout of multi-line content needs to be judged, and the single-line content has only one row, which is the main row). Each column is judged: if the number of text blocks in the current column is 1 and the block is not in the same row as the main row, it is necessary to judge whether the block is reasonable. If the previous column of the block has a text block in the same row as the block, it is considered reasonable, otherwise it is considered unreasonable, and the unreasonable number of this alignment method is +1; if the number of text blocks in the current column is greater than 1, it is necessary to judge the distance between each block in the current column and the average distance. If the distance between the blocks minus the minimum distance between the blocks in the current column is greater than half of the average row height (the average row height Hg calculated by the division after text recognition), it is considered unreasonable, and the unreasonable number of this alignment method is +1.

[0097] Then, for each table row of each alignment method, determine whether it complies with the current alignment rules. For example, each row of the result of top alignment should comply with the top alignment rules, otherwise it is considered unreasonable. Top alignment should comply with the main row being the top row, otherwise it is considered unreasonable, and the unreasonable number of top alignments is increased by 1; bottom alignment should comply with the main row being the bottom row, otherwise it is considered unreasonable, and the unreasonable number of top alignments is increased by 1; center alignment should comply with the main row being the middle row, otherwise it is considered unreasonable, and the unreasonable number of center alignments is increased by 1.

[0098] In summary, the alignment method with the least unreasonable number is selected.

[0099] Step S6: using the column separation technology to obtain candidate column lines, and then performing column separation based on the candidate column lines combined with text semantics and character features;

[0100] The specific steps of step S6 are:

[0101] Step S601: Obtain candidate column lines and determine the number of table header rows in the current layout, because each row of table headers needs to correspond to each text row content. For example, if there are two rows of table headers in the layout, each table row content also has two rows. The first row of table headers needs to correspond to the first text row, and so on.

[0102] In this embodiment, the logic for determining the number of rows in the table header. If the table header is found in step S4, the number of rows in the found table header is used; otherwise, it is necessary to determine based on the table content: if the row parsing uses a centered layout, the number of rows in the table header must be 1; if the dates in the table row are in separate columns, the number of rows in the table header is 1; if the dates in the table row are not in separate columns, it is necessary to determine the number of text rows between adjacent dates, count the number of text rows between every two adjacent dates, and take the number of text rows with the most occurrences as the number of rows in the table header. Perform the following column splitting logic for the content corresponding to each row of the table header:

[0103] First, use horizontal projection to find the candidate column lines. If there is a table header and the number of columns of the candidate column line and the table header is the same, stop searching for the candidate column line, otherwise continue searching;

[0104] Then use the determined candidate column line to check the amount column for each column to ensure that the amount column is correct. If the amount column is stuck, it needs to be split. Use the amount regularity to count all the amounts in the current column. Use the overlap rate to group all the amounts. If the overlap rate is greater than 0.5, it is considered to be a group. If there is only one group, the amount column is correctly split. Otherwise, the amount column is divided into multiple columns according to the grouping information. The overlap rate calculation formula is:

[0105] iou=(min(x_max1,x_max2)-max(x_min1,x_min2)) / min(x_max1-x_min1,x_max2-x_min2), where iou represents the overlap rate, min represents the minimum value function, max represents the maximum value function, x_max1 represents the maximum horizontal coordinate value of the first amount column, x_min1 represents the minimum horizontal coordinate value of the first amount column, x_max2 represents the maximum horizontal coordinate value of the second amount column, x_min2 represents the minimum horizontal coordinate value of the second amount column, y_max1 represents the maximum vertical coordinate value of the first amount column, y_min1 represents the minimum vertical coordinate value of the first amount column, y_max2 represents the maximum vertical coordinate value of the second amount column, y_min2 represents the minimum vertical coordinate value of the second amount column,

[0106] Finally, use the logic of text block alignment to check whether each column can be further split. First, group all the text blocks in each column according to the overlap rate and alignment strategy: create an initial column using each block in the first row, traverse each block in each row, if the average value of the minimum abscissa values of all text blocks in a certain column and the minimum abscissa value of the current text block is less than the average line height, and the overlap rate is greater than 0.5, then it belongs to this column, and so on. Group all the text blocks, and the candidate column lines corresponding to each column can be obtained after grouping. Then, judge whether the candidate column lines of each column after grouping meet the splitting conditions: Condition 1, the standard deviation of the minimum abscissa values of all blocks in the column is less than the average line height; Condition 2, combine the candidate column lines and text semantics to judge the text blocks that should be split, and the proportion of the number of rows of the text blocks that should be split is greater than 65% of the total number of rows; Condition 3, all blocks in the current column have a common prefix or suffix, and the coordinate overlap rate of the prefix and suffix is greater than 0.7. As long as [Condition 1 and (Condition 2 or Condition 3)] is met, it meets the splitting conditions.

[0107] Step S602: Use the candidate column lines and combine text semantics and character features to perform splitting. Use the candidate column lines to split each row. If the candidate column line is between two text blocks, it can be divided into left and right parts according to the candidate column line. If the candidate column line is in the middle of a text block, it is necessary to decide from which character to start the split.

[0108] In this embodiment, first, determine from which character to perform the split according to the semantics of the text. If it is impossible to determine according to the semantics,

[0109] Then, split according to the character features in combination with the position of the candidate line. Calculate the score of adjacent characters of the text block to be split, score = score_line + score_blank + score_dis. Define the candidate line position as line_x and the average line height as avg_line_height. Then the calculation of each score is as follows:

[0110] score_line represents the score at the candidate line position. If char_x_min <= line_x < char_x_max, score_line is 1, otherwise score_line is:

[0111] max(0, avg_line_height * 3 - min(abs(char_x_min - line_x), abs(char_x_max - line_x)) / (avg_line_height * 3)), abs represents the absolute value function, char_x_min represents the minimum abscissa of the candidate line position, and char_x_max represents the maximum abscissa of the candidate line position.

[0112] score_blank represents the score of the space between characters. The distance between two adjacent characters is the minimum horizontal coordinate x2_min of the next character minus the maximum horizontal coordinate x1_max of the current character, that is,

[0113] score_blank=x2_min-x1_max,

[0114] score_dis represents the distance score between characters. The center distance between two adjacent characters is char_center_dis. char_center_dis = next_char_x_center - cur_char_x_center. next_char_x_center represents the central horizontal coordinate of the next character, and cur_char_x_center represents the central horizontal coordinate of the current character.

[0115] score_dis=min(avg_line_height*0.5*1.1,char_center_dis) / (avg_line_height*0.5),

[0116] After calculating the score, select the largest score and split when the score>0.5, otherwise do not split;

[0117] Step S7: Output each text block and the corresponding row and column position using the json format according to the row and column to which the identified text block belongs.

[0118] In this embodiment, Figure 3 The layout is a single-line header and a single-line content. Figure 4 The format is a single-line header and multiple lines of content, with the date centered. Figure 5 The format is a single-line header and multiple lines of content, with the date aligned at the top. Figure 6 The layout is a single-line header and multiple lines of content. Figure 7 The layout is no header, single-line content and column glue. Figure 8 The layout is no header, multiple lines of content and amounts are glued together. Fig. 9 The layout is multi-table.

[0119] Example 2

[0120] See also Fig.10 , another embodiment provided by the present invention: a flow table recognition system, comprising: an image acquisition module, an image detection module, a grouping improvement module, a position determination module, a row division module, a column division module and a table output module;

[0121] The image acquisition module is used to acquire a PDF file with a table, convert the PDF file with a table into an image, and correct the image;

[0122] The image detection module is used to detect text, table and date information in the image respectively;

[0123] The perfecting and grouping module is used to perfect and group the detected date information;

[0124] The position determination module is used to determine the specific position of the table in the image, including the position of the table header and the position of the body;

[0125] The branching module is used to branch the images according to the date information in the images;

[0126] The column separation module is used to obtain candidate column lines using the column separation technology, and then perform column separation based on the candidate column lines in combination with text semantics and character features;

[0127] The table output module is used to output each text block and the corresponding row and column position using the JSON format according to the row and column to which the identified text block belongs.

[0128] The branch module includes: content single-line branch unit and content multi-line branch unit.

[0129] The content single-line branching unit is used to branch the table rows when the content in the text row in the image is a single line;

[0130] The content multi-row branching unit is used to perform table row branching when the content in a text row in the image is larger than one row.

[0131] Example 3

[0132] Fig.11 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Fig.11 As shown, according to another aspect of the present application, an electronic device 500 is provided. The electronic device 500 may include one or more processors and one or more memories. The memories store computer readable codes, and when the computer readable codes are executed by one or more processors, a method for identifying a running table can be executed.

[0133] The method or system according to the embodiment of the present application can also be used by Fig.11 The electronic device architecture shown in FIG. Fig.11As shown, the electronic device 500 may include a bus 501, one or more CPUs 502, ROM 503, RAM 504, a communication port 505 connected to a network, an input / output component 506, a hard disk 507, and the like. The storage device in the electronic device 500, such as ROM 503 or hard disk 507, may store a method for recognizing a running table provided in the present application. A method for recognizing a running table may, for example, include: obtaining a PDF file with a table, converting the PDF file with a table into an image, and correcting the image, respectively detecting the text, table, and date information in the image, improving and grouping the detected date information, determining the specific position of the table in the image, including the header position and the body position, dividing the rows according to the date information in the image, using column division technology to obtain candidate column lines, and then dividing the columns according to the candidate column lines in combination with text semantics and character features, and outputting each text block and the corresponding row and column positions in json format according to the rows and columns to which the identified text blocks belong. Furthermore, the electronic device 500 may also include a user interface 508. Of course, Fig.11 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Fig.11 One or more components of an electronic device are shown.

[0134] Example 4

[0135] Fig.12 Schematic diagram of a computer-readable storage medium structure provided by an embodiment of the present application. Fig.12 As shown, a computer-readable storage medium 600 according to an embodiment of the present application is shown. Computer-readable instructions are stored on the computer-readable storage medium 600. When the computer-readable instructions are executed by the processor, a method for identifying a flow table according to an embodiment of the present application described with reference to the above figures can be executed. The storage medium 600 includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0136] It should be understood that the methods, apparatuses, and devices of the present application can be implemented in many ways. For example, the methods, apparatuses, and devices of the present application can be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is only for illustration, and the steps of the method of the present application are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present application can also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers a recording medium storing a program for executing the method according to the present application.

[0137] In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0138] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for identifying a flow table, characterized in that: include: Get a PDF file with a table, convert the PDF file with a table into an image, and correct the image; Detect text, table and date information in images respectively; Improve and group the detected date information; Determine the specific position of the table in the image, including the position of the table header and the position of the body; Split the image into rows based on the date information in the image; Use the column separation technology to obtain candidate column lines, and then separate the columns according to the candidate column lines combined with text semantics and character features; According to the row and column to which the identified text block belongs, each text block and the corresponding row and column position are output in JSON format.

2. A method for identifying a flow table as claimed in claim 1, characterized in that: The method of respectively detecting text, table and date information in an image comprises: Use DBNet+CRNN algorithm to detect the image and obtain all text information and coordinate information of the text information in the image and the position information set of each character in the text, x min The minimum horizontal coordinate representing the text information, x max Indicates the maximum horizontal coordinate of the text information, y min Represents the minimum vertical coordinate of text information, y max Indicates the maximum vertical coordinate of the text information; Locate the table in the image and use the YOLOv7 model to locate the table part; The date information in the image is detected to obtain the date information in the image.

3. A method for identifying a flow table as claimed in claim 2, characterized in that: The step of improving and grouping the detected date information includes: Regularly add or delete the date information in the image; According to the date information after regular addition or deletion in the image and the detected table, the date information after regular addition or deletion in the image is grouped using the overlap rate.

4. A method for identifying a flow table as claimed in claim 3, characterized in that: The step of dividing the images into rows according to the date information in the images comprises: When the content of a text line in the image is a single line, the text line is a table row; When the content of a text line in an image is larger than one line, the logic for dividing the text line in the image is: 1) Top-aligned branch logic: branch between two adjacent dates, with the minimum y coordinate of the first date 1min and the minimum y coordinate of the second date 2min The content between is a table row. 2) Bottom-aligned branch logic: branch between two adjacent dates, with the maximum y coordinate of the first date 1max and the maximum ordinate y of the second date 2max The content between is a table row. 3) Center-aligned branch logic: branch between two adjacent dates, and define the median ordinate of the first date as cur_y center , the median ordinate of the second date next_y center , half the distance between the first and second dates is half_dis = (cur_y center +next_y center ) / 2, half_dis represents half of the distance between the first and second dates, then in cur_y center +half_dis and next_y center +The content between half_dis is a table row; The text lines in the image are divided into rows to obtain the table rows of the table in the image.

5. A method for identifying a flow table as claimed in claim 4, characterized in that: The step of sorting the candidate columns according to the text semantics and character features includes: Get candidate column lines and determine the number of header rows in the current layout; Candidate column lines are used, and text semantics and character features are combined for column segmentation. Each row is segmented using candidate column lines. If the candidate column line is between two text blocks, it is divided into left and right parts according to the candidate column line. If the candidate column line is in the middle of a text block, it is segmented according to the character position.

6. A method for identifying a flow table as claimed in claim 5, characterized in that: If the candidate column line is in the middle of a text block, segmentation is performed according to character positions, including: The scores of adjacent characters of the segmented text block, score = score_line + score_blank + score_dis, score_line represents the score of the candidate line position, score_blank represents the score of the space between adjacent characters, and score_blank represents the score of the space between adjacent characters; Select the largest score and split between adjacent characters when score>0.

5.

7. A transaction table recognition system, used to implement a transaction table recognition method according to any one of claims 1 to 6, characterized in that: include: Image acquisition module, image detection module, perfect grouping module, position judgment module, row division module, column division module and table output module; The image acquisition module is used to acquire a PDF file with a table, convert the PDF file with a table into an image, and correct the image; The image detection module is used to detect text, table and date information in the image respectively; The perfecting and grouping module is used to perfect and group the detected date information; The position determination module is used to determine the specific position of the table in the image, including the position of the table header and the position of the body; The branching module is used to branch the images according to the date information in the images; The column separation module is used to obtain candidate column lines using the column separation technology, and then perform column separation based on the candidate column lines in combination with text semantics and character features; The table output module is used to output each text block and the corresponding row and column position using the JSON format according to the row and column to which the identified text block belongs.

8. A flow table recognition system as claimed in claim 7, characterized in that: The branch module includes: a content single-line branch unit and a content multi-line branch unit. The content single-line branching unit is used to branch the table rows when the content in the text row in the image is a single line; The content multi-row branching unit is used to perform table row branching when the content in a text row in the image is larger than one row.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of a transaction table recognition method described in any one of claims 1-6 are implemented.

10. A computer-readable storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed, the steps of a transaction table recognition method described in any one of claims 1-6 are executed.

Citation Information

Patent Citations

  • Table identification method and device

    CN111062259A

  • An imported cargo customs declaration OCR branch processing method and device

    CN113723301A

  • Bank statement table extraction method and device, computer equipment and storage medium

    CN114463763A

  • Bank account statement identification method

    CN117037198A

  • Table information identification method

    CN117558017A

Cited By

  • Table automatic identification and structured conversion method and device

    CN121259859A

  • A method and apparatus for table automatic recognition and structural conversion

    CN121259859B