Bank statement form extraction method and device, computer device and storage medium

Abstract

The present application relates to the technical field of computer image processing, and provides a bank flow table extraction method and device, computer equipment and storage medium, the method comprises: obtaining a bank flow image, detecting the text area in the bank flow image; merging the text area to construct a text line, obtaining a text line set; traversing each text line in the text line set, extracting a flow table header row and a flow table body row set; according to the distance of each text area in the flow table header row and the flow table body row set, column division is carried out on each text area in the flow table body row set, and the flow table body column set is determined; based on the consistency of the table header and the table body row and column, the flow table header row and the flow table body column set are checked and updated, and the checked and updated flow table header row, flow table body row and flow table body column set are saved as a table extraction result. The present application can improve the accuracy of bank flow table extraction.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer image processing, and in particular relates to a bank flow form extraction method, device, computer equipment and storage medium. Background Art

[0002] When a company conducts an IPO (Initial Public Offering), the China Banking Regulatory Commission usually requires the company and its directors and senior executives to provide all bank transaction records from recent years and to go to a securities firm to complete a bank statement verification to prevent the company from various potential risks such as financial fraud.

[0003] However, currently, corporate bank statements are primarily printed on paper, requiring business personnel to manually enter bank transaction records into the bank statement verification system, significantly reducing bank statement entry efficiency. Furthermore, due to the complexity and variety of bank statement forms, missing lines, and susceptibility to interference from various factors such as stamps and stains, conventional form extraction methods are unable to handle these complex and diverse forms, resulting in reduced form extraction accuracy. Summary of the Invention

[0004] Based on this, it is necessary to provide a bank statement form extraction method, device, computer equipment and storage medium that can improve the accuracy of bank statement form extraction in response to the above technical problems.

[0005] The present invention provides a method for extracting a bank statement form, comprising:

[0006] Acquire a bank statement image, and detect a text area in the bank statement image;

[0007] Merging the text regions to construct text lines, and obtaining a text line set;

[0008] Traversing each text line in the text line set, extracting a flow table header line and a flow table body line set; the flow table header line is a header line of a table on a bank flow table image, and the flow table body line set is a set including all flow table body lines, and the flow table body lines are other text lines except the header line;

[0009] Dividing each text area in the water meter body row set into columns according to the distance between the water meter header row and each text area in the water meter body row set to determine a water meter body column set;

[0010] Based on the consistency of the table header and table body rows and columns, the flow table header row and the flow table body column set are verified and updated, and the verified and updated flow table header row, flow table body row and flow table body column set are saved as the table extraction result; specifically including:

[0011] Determine the total number of text areas in the flow meter body column set and the number of text areas corresponding to each flow meter body column in the flow meter body column set;

[0012] When the number of the text areas exceeds a preset percentage of the total number of the text areas, constructing a header text block corresponding to the flow table body column;

[0013] Adding the header text block whose area overlaps with the text area in the flow table header row is less than a threshold to the flow table header row to obtain an updated flow table header row;

[0014] Re-dividing the columns based on the updated flow table header row to determine the updated flow table body column set;

[0015] The updated flow table header row, flow table body row, and flow table body column set are saved as the table extraction result.

[0016] In one embodiment, merging the text regions to construct text lines and obtaining a text line set includes:

[0017] Calculating a vertical overlap ratio between the text regions according to the region positions of the text regions;

[0018] Determining a group of text regions that can be merged according to the vertical overlap ratio;

[0019] The text region groups are merged using a union-find algorithm to obtain a text line set.

[0020] In one embodiment, traversing each text line in the text line set to extract a flow table header line and a flow table body line set includes:

[0021] Traversing the text areas of the text lines in the text line set, matching the text content corresponding to the text areas of the text lines with the header text trigger words, and determining the number of trigger word matches of the text lines;

[0022] Determining a flow table header row from each of the text rows according to the number of trigger word matches;

[0023] Retraverse the non-flowing header rows in the text row set, match the flow elements of each non-flowing header row using the flow element regular expression, and determine the number of flow element matches;

[0024] Table body rows are extracted according to the matching quantity of the flow elements to obtain a flow table body row set.

[0025] In one embodiment, dividing each text area in the water meter body row set into columns according to the distance between the water meter header row and each text area in the water meter body row set to determine the water meter body column set includes:

[0026] Constructing a two-dimensional array of table body cells according to the number of cells in the flow table body row and the number of elements in the flow table body row set;

[0027] Traversing the set of flow meter body rows, determining the position of the header row where the distance between each text area in each flow meter body row and the flow meter header row is the smallest;

[0028] According to the position of the table header row, each text area in the flow table body row is added to the two-dimensional array of table body cells;

[0029] Based on the horizontal overlap ratio between the text areas in the two-dimensional array of table body cells, a flow table body column is obtained, and a flow table body column set is obtained.

[0030] In one embodiment, determining the position of the header row where the distance between each text area in each flow meter body row and the flow meter header row is the smallest includes:

[0031] Calculating the distance between each text area in the flow meter body row and each text area in the flow meter header row based on the center point coordinates and width of the text area;

[0032] The distances are compared to determine a minimum distance, and a text area position corresponding to the minimum distance is determined from each text area of ​​the flow table header row as the table header row position.

[0033] In one embodiment, the verifying and updating the flow table header row and the flow table body column set based on the consistency of the table header and table body rows and columns, and saving the verified and updated flow table body row and flow table body column set as the table extraction result, includes:

[0034] Determine the total number of text areas in the flow meter body column set and the number of text areas corresponding to each flow meter body column in the flow meter body column set;

[0035] When the number of the text areas exceeds a preset percentage of the total number of the text areas, constructing a header text block corresponding to the flow table body column;

[0036] Adding the header text block whose area overlaps with the text area in the flow table header row is less than a threshold to the flow table header row to obtain an updated flow table header row;

[0037] Re-divide the columns based on the updated flow table header row to determine the updated table body column set;

[0038] Save the updated flow table body rows and flow table body column sets as the table extraction result.

[0039] In one embodiment, saving the updated flow meter body row and the updated flow meter body column set as the table extraction result includes:

[0040] Construct a two-dimensional array of bank flow cells;

[0041] The updated transaction table body row and the updated transaction table body column set are assigned to the bank transaction cell two-dimensional array and saved in a preset file to obtain a table extraction result.

[0042] A bank statement form extraction device, comprising:

[0043] A detection module, configured to acquire a bank statement image and detect a text area in the bank statement image;

[0044] A merging module, configured to merge the text regions to construct text lines and obtain a text line set;

[0045] an extraction module, configured to traverse each text line in the text line set and extract a flow table header line and a flow table body line set; the flow table header line is a header line of a table on a bank flow table image; the flow table body line set is a set including all flow table body lines, and the flow table body lines are text lines other than the header line;

[0046] a determination module, configured to divide each text area in the water meter body row set into columns according to the distance between the water meter header row and each text area in the water meter body row set, and determine a water meter body column set;

[0047] A verification module is used to verify and update the water table header row and the water table body column set based on the consistency of the table header and table body rows and columns, and save the verified and updated water table header row, water table body row and water table body column set as a table extraction result, specifically used to: determine the total number of text areas in the water table body column set and the number of text areas corresponding to each water table body column in the water table body column set; when the number of text areas exceeds a preset percentage of the total number of text areas, construct a table header text block corresponding to the water table body column; add the table header text block whose area overlaps with the text area in the water table header row is less than a threshold to the water table header row to obtain an updated water table header row; re-divide the columns based on the updated water table header row to determine an updated water table body column set; save the updated water table header row, water table body row and water table body column set as a table extraction result.

[0048] The present invention also provides a computer device, which includes a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the bank flow form extraction method described in any one of the above are implemented.

[0049] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of any one of the above-mentioned methods for extracting bank statement forms are implemented.

[0050] The bank statement table extraction method, device, computer device, and storage medium described above first detect the text regions of the bank statement image and then merge the text regions to construct text rows, obtaining a text row set. The method then traverses each text row in the text row set to extract the statement header row and statement body row set. The method then divides each text region in the statement body row set into columns based on the distance between the statement header row and each text region in the statement body row set, determining a statement body column set. The method further verifies and updates the statement header row and statement body column set based on the consistency of the statement header and statement body rows and columns, and saves the verified and updated statement header row and statement body column set as the table extraction result. This method extracts and verifies each row and column in the table through the rows and columns themselves, fully utilizing the characteristics of the consistency of the table rows and columns and the alignment of the table header and table body to extract the bank statement table. This method can uniformly process various tables using a spatial alignment strategy and effectively handle missing elements, thereby greatly improving the accuracy and robustness of bank statement table extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a diagram of the application environment of a bank statement form extraction method in one embodiment.

[0052] Figure 2 The figure is a flow chart of a method for extracting bank statement forms in one embodiment.

[0053] Figure 3 A schematic diagram of a bank statement image in one embodiment.

[0054] Figure 4 FIG. 4 is a schematic diagram of a text area in an embodiment.

[0055] Figure 5 Schematic diagram of a text line in one embodiment.

[0056] Figure 6 Schematic diagram of a collection of flow meter header rows and flow meter body rows in one embodiment.

[0057] Figure 7 Schematic diagram of a water meter column in one embodiment.

[0058] Figure 8 In one embodiment, based on Figure 7 Schematic diagram of the updated transaction table.

[0059] Figure 9 A schematic diagram of table extraction results in one embodiment.

[0060] Figure 10 1 is a structural block diagram of a bank statement extraction device in one embodiment. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0062] The bank flow form extraction method provided in this application can be applied to Figure 1 In the illustrated application environment, the application environment involves a terminal 102 and a server 104. The terminal 102 communicates with the server 104 via a network. The terminal 102 may be, but is not limited to, various personal computers, laptops, smartphones, tablet computers, and portable wearable devices. The server 104 may be implemented as a standalone server or a server cluster consisting of multiple servers.

[0063] When terminal 102 receives a bank statement table extraction instruction, the above-mentioned bank statement table extraction method can be implemented by terminal 102 alone. Alternatively, terminal 102 can send the bank statement table extraction instruction to the communicating server 104, and server 104 can implement the above-mentioned bank statement table extraction method. Taking server 104 as an example, specifically, server 104 obtains a bank statement image and detects text areas in the bank statement image; server 104 merges text areas to construct text rows and obtains a text row set; server 104 traverses each text row in the text row set and extracts a statement header row and a statement body row set; server 104 divides each text area in the statement body row set into columns based on the distance between the statement header row and each text area in the statement body row set, and determines a statement body column set; server 104 verifies and updates the statement header row and statement body column set based on the consistency of the statement header and statement body rows and columns, and saves the verified and updated statement header row and statement body column set as the table extraction result.

[0064] In one embodiment, Figure 2 As shown, a method for extracting bank flow forms is provided, which is described by taking the method applied to a server as an example, and includes the following steps:

[0065] Step S201: Acquire a bank statement image and detect text areas in the bank statement image.

[0066] The bank statement image refers to an image obtained by capturing an image of a paper bank statement, and the text area refers to an area in the bank statement containing text content.

[0067] Specifically, after receiving the instruction, the server first obtains the bank statement image obtained by scanning the paper bank statement by the scanner, such as Figure 3 Then, the text detection model is used to extract the text area in the bank flow image and the text recognition model is used to recognize the text in the text area to obtain the text recognition result of the bank flow image. It can be recorded as C = {C1, C2, ..., C n |C i =(r i , t i )}, where n is the number of text regions, i = {1, 2, ..., n}, C i Used to represent each text area, the text area includes the area position r i And the text recognition result t i .like Figure 4 As shown, a schematic diagram of a text area is provided. Figure 4 The rectangle in the text area is the text area.

[0068] Step S202: Merge text regions to construct text lines, and obtain a text line set.

[0069] Specifically, after obtaining the text area, in order to accurately obtain the rows in the table, the server merges the text areas belonging to the same row to obtain text rows, and the collection of all text rows is the text row set L. Figure 5 Said, a schematic diagram of a text line is provided, Figure 5 The horizontal and straight lines in the text represent a text line, and the text content on a straight line belongs to the text content of the same line.

[0070] In one embodiment, step S202 includes: calculating the vertical overlap ratio between text regions based on the region positions of the text regions; determining a mergeable text region group based on the vertical overlap ratio; and merging the text region groups using a union-find algorithm to obtain a text line set.

[0071] Specifically, when two text regions belong to the same line, there should be some overlap in the vertical direction. Therefore, when the server merges text regions to determine the text line, it calculates the vertical overlap ratio between the two text regions based on their area positions. Based on the vertical overlap ratio, it determines whether the two text regions belong to the same line and whether they can be merged into a text line.l and the rth text region c r For example, the vertical overlap ratio V iou The calculation formula is as follows:

[0072]

[0073] Among them, (x l ,y l ,w l ,h l ) is the lth text area c l The regional position, (x r ,y r ,w r ,h r ) is the rth text area c r Region position, x and y represent coordinates, w and h represent width and height. In this embodiment, when the vertical overlap ratio of two text regions is greater than 0.7, the two text regions are considered to be mergeable, and are referred to as their mergeable text region group. 0.7 is set based on the text position in the actual bank flow, and can be adjusted based on actual conditions. Then, the determined text region group is merged using a union-find algorithm to obtain a series of text lines. According to the region position of the text lines, the text lines are sorted from top to bottom to obtain an ordered set of text lines, denoted as L = {l1, l2, ..., l m |l i =(C1,……,C i )}, where m is the total number of text lines, and each text line l i Contains more than one text area C i .

[0074] Step S203 , traverse each text line in the text line set, and extract the flow table header line and the flow table body line set.

[0075] The flow table header row is the table header row on the bank flow image, and the flow table body row set refers to the set of all flow table body rows. The flow table body rows are other text rows except the header row. Figure 6 As shown, a schematic diagram of a flow meter header row and a flow meter body row is provided, from top to bottom. Figure 6 The first horizontal line corresponds to the flow table header row, and each horizontal line below the flow table header row is a flow table body row, and their total is the flow table body row set.

[0076] Specifically, after the server merges text regions to obtain text lines, it identifies the text lines that may belong to header lines by identifying the text content corresponding to each text line and considers them as flow table header lines. Once a flow table header line is determined, generally speaking, the remaining text lines below the flow table header line are likely to be flow table body lines. However, because this embodiment combines text lines for the entire bank flow, there may be text lines that are neither header lines nor body lines. Therefore, to accurately obtain a set of flow table body lines, the remaining non-flow table header lines are still determined to be flow table body lines by judging the text content.

[0077] In one embodiment, step S203 includes: traversing the text area of ​​each text line in the text line set, matching the text content corresponding to the text area of ​​the text line with the header text trigger word, and determining the number of trigger word matches of the text line; determining the flow header row from each text line based on the number of trigger word matches; re-traversing the non-flow header rows in the text line set, using the flow element regular expression to match the flow elements of each non-flow header row respectively, and determining the number of flow element matches; extracting the body rows based on the number of flow element matches to obtain a flow body row set.

[0078] Specifically, the server first obtains the header text trigger words WH of each bank. The header text trigger words can be collected in advance or collected by the server in real time according to the given flow sample image, which is recorded as WH = {W1, W2, ..., W m}, where m is the number of trigger words in the header text, W i Represents the i-th trigger word, where i = {1, 2, ..., m}. The header text trigger words in this embodiment can be understood as text content that often appears in the bank flow header, such as transaction date, transaction amount, account balance, account information, etc. Then, since the text line set is an ordered set arranged from top to bottom, and the headers are all text lines at the top. Therefore, in order to save traversal time, the server traverses the text lines in the text line set in order from top to bottom. Every time a text line is traversed, the text content corresponding to the text area of ​​the currently traversed text line is matched with the header text trigger word. Every time a header text line trigger word is matched, the number of trigger word matches is increased by 1, until the text content of all text areas of the currently traversed text line is matched, and the number of trigger word matches for the text line is obtained. This matching operation is performed on each text line in the text line set, and then the flow header line is determined according to the number of trigger word matches corresponding to each text line in the text line set. For example, let the current number of trigger word matches be t l , traverse the text line l in turn i Any text area C i If the text area C iIf the text content in the header text is consistent with any trigger word in WH, then t l The quantity is increased by 1. Finally, t l Make a judgment, if t l Greater than l i 1 / 2 of the total number of text areas in the text area is considered as text line l i It is the flow table header line, recorded as l t , the current position in the corresponding text line set L is recorded as idx t .

[0079] After getting the log header line, since the text line before the log header line must not belong to the table body line, in order to save the time of traversal judgment, the server starts from the log header line l t Position idx t Starting from the next line, traverse the non-flowing header lines in the text line set L from top to bottom, and perform table body judgment for each non-flowing header line in the text line set L. For a normal bank statement, each line of statement has three necessary elements, namely transaction date, transaction amount and balance. Therefore, traverse the non-flowing header lines l in sequence. i Any text area C i , match the date and amount through the preset regular expression of the flow element, and determine the non-flow header line l i The number of flow element matching t b If the non-daily header line l i The number of matches t b ≥2, then the non-flowing table header row l i For the flow meter body row l b , the flow meter body line l b The current position in the corresponding text set L is recorded as idx b In order to distinguish the final row of the table, find the last flow table row l from the text set L b , record its location as idx b_max Then extract [idx t +1, idx b-max The text lines in ] form the flow table body line set L b .

[0080] Step S204 : dividing each text area in the flow meter body row set into columns according to the distance between the flow meter header row and each text area in the flow meter body row set, and determining the flow meter body column set.

[0081] Specifically, after the server determines the bank statement table's flow header row and flow body row set, based on the consistency of the table's rows and columns, the flow body columns are divided according to the distance between the flow header row and the text area in the flow body row set to obtain a flow body column set. The flow body column set is a collection of flow body columns, such as Figure 7 As shown, a schematic diagram of a water meter column is provided. Figure 7 The rectangle in the middle is the flow meter body column.

[0082] In one embodiment, step S204 includes: constructing a two-dimensional array of body cells based on the number of cells in the flow body row and the number of elements in the flow body row set; traversing the flow body row set to determine the position of the header row where the distance between each text area in each flow body row and the flow header row is the smallest; adding each text area in the flow body row to the two-dimensional array of body cells based on the horizontal overlap ratio between each text area in the two-dimensional array of body cells to obtain a flow body column, and obtaining a flow body column set.

[0083] Specifically, the comprehensive flow table header line l t and the flow table body row set L b , construct a two-dimensional array M of table cells b [row][col], col is the flow table header row l t The number of cells in the row is the flow meter body set L b The number of elements in the array, each cell can contain 0 or more text areas. The number of cells refers to the number of rows in the flow table body. t The number of text areas included, the number of elements refers to the flow table body line set L b Then, the server traverses the set of flow meter body rows L b , determine the text area in each flow table body row and the flow table header row l t The header row position with the smallest distance. Each text area in the flow-through body row is added to the corresponding position in the two-dimensional array of the body cells based on the header position. At this point, the text areas included in each column are divided. The header row position is the position in the flow-through body row with the smallest distance to the text area in the flow-through body row. For example, if text area A in a flow-through header row has the smallest distance to text area B in a flow-through body row, then the position of text area A is the header row position with the smallest distance to text area B.

[0084] In another embodiment, determining the position of the header row where the distance between each text area in each flow meter body row and the flow meter header row is the smallest includes: calculating the distance between each text area in the flow meter body row and each text area in the flow meter header row based on the center point coordinates and width of the text area; comparing each distance to determine the minimum distance, and determining the text area position corresponding to the minimum distance from each text area in the flow meter header row as the header row position.

[0085] Specifically, when traversing the flow table body line L b When determining the position of the header row, take out the text area C included in the current traversal of the flow table body row in turn i , according to the text area C i The x-axis coordinate of the center point and the width w l , respectively calculate and water table header line l t The distance between each text area in the flow table is compared with the distance between each text area in the flow table. t The position of the text area with the smallest distance is found as the header row position, which represents the text area C in the current traversal flow table body row. i The corresponding column position. In this embodiment, based on the table body cell two-dimensional array M b [row][col] The process of dividing the text area in the flow table row into columns is expressed in code as follows:

[0086] for(i=0;i <row;i++){

[0087] l i =L b [i]; / / From L b Take out the i-th flow meter body row l i ;

[0088] for(int j = 0; j <len(l);j++){

[0089] C i =l i [j]; / / from l i Take out the jth text area C i ;

[0090] idx_col=findNearestHeaderIdx(C i ,l t ) / / According to text area C i The x-axis coordinate of the center point, from the head line l t Find the header row position with the smallest distance;

[0091] M b[i][idx_col].append(Ci) / / Set text area C i Add to M bIn the corresponding cell in the array;

[0092] }

[0093] }

[0094] findNearestHeaderIdx is defined as follows:

[0095]

[0096] Among them, x l and w l Represents the text area C i The x coordinate and width of Z can be understood as the serial header line l t The text area.

[0097] Then, in order to obtain a complete column, the server adds the text area to the table body cell two-dimensional array M b Each column M in [row][col] b_idx_col Merge the columns in the same position. Column merging is also based on the overlap ratio. That is, if the table body cell two-dimensional array M b If the horizontal overlap ratio of any two text regions in [row][col] is greater than 0.7, then the two text regions are considered to be mergeable. The same union-find algorithm is used based on the column M. b_idx_col The merged areas of all cells in the table are merged to obtain a series of text columns, which are the flow table body columns. l and the rth text region c r For example, the horizontal overlap ratio H iou The calculation formula is as follows:

[0098]

[0099] After merging to obtain a series of flow meter body columns, the series of flow meter body columns are sorted from left to right according to the regional position of the flow meter body columns to obtain an ordered set of flow meter body columns, which is recorded as N = {n1, n2, ..., n m |n i =(C1,……,C i )}, where m represents the total number of flow table body columns, i = {1, 2, ..., m}, and each flow table body column contains one or more text areas C i .

[0100] Step S205 , based on the consistency of table header and table body rows and columns, verify and update the water table header row, water table body row and water table body column set, and save the verified and updated water table header row, water table body row and water table body column set as the table extraction result.

[0101] Specifically, after the server has determined the bank statement header row, statement body row, and statement body column in the above steps, in order to avoid errors in header and body division and extraction caused by missing elements in the header, the server further verifies and updates the statement header row, statement body row, and statement body column set based on the column alignment property of the header and body.

[0102] In one embodiment, step S205 includes: determining the total number of text areas in the flow meter body column set and the number of text areas corresponding to each flow meter body column in the flow meter body column set; when the number of text areas exceeds a preset percentage of the total number of text areas, constructing a header text block corresponding to the flow meter body column; adding the header text block whose area overlaps with the text area in the flow meter header row is less than a threshold to the flow meter header row to obtain an updated flow meter header row; re-dividing columns based on the updated flow meter header row to determine an updated flow meter body column set; and saving the updated flow meter header row, flow meter body row, and flow meter body column set as a table extraction result.

[0103] Specifically, for the current column M b_idx_col Perform column splitting: If the number of elements in the flow meter body column set N (the number of flow meter body columns) is 1, it is considered that the current column does not need to be split. Otherwise, the flow meter body columns n in N are split in turn. i Make a judgment. If n i If the number of text areas in N is greater than 25% (1 / 4) of the total number of text areas in N, then a flow table column n is constructed. i Corresponding header text block C ni =(r ni , t ni ), where r ni The corresponding y-axis coordinate and matrix height are equal to the flow table header row l t The mean of the y-axis coordinates of all text areas and the mean of the matrix height, r ni The corresponding x-axis coordinate and matrix height are equal to the flow meter column n i The mean of the x-axis coordinates of all text areas and the mean of the matrix height, t ni The corresponding value is set to "unknown".

[0104] Traverse the table body column set N to obtain the result C of the header text block n ={C n1 ,C n2 ,……,C nn |

[0105] C ni =(r ni , t ni )}, i={1,2,……,n}. Then for C nAny header text block C ni Judgment and flow table header line l t The area overlap of the corresponding text area is calculated. If the area overlap is less than the threshold value 0.3, the header text block C ni Add to the journal header line t In the above example, we get the updated flow table header row l t . Because the flow table header line l t If there is an update, the previous row is based on the flow table header line l t The divided and determined flow table body columns should be updated synchronously. Therefore, based on the updated flow table header row l t Re-determine the flow table body column to obtain the updated flow table body column set, and then save the updated flow table header row and flow table body column set as the table extraction result. It should be understood that if the flow table header row l t If there is no update in the table, then there is no need to update the water flow table body column set, which means that the determined water flow table header row and water flow table body column set have passed the verification and can be saved as the table extraction result. To update the water flow table body column set, return to step S204, based on the distance between the updated water flow table header row and each text area in the water flow table body row set, divide the text area in the water flow table body row set into columns, and determine the water flow table body column set. The principle is the same and will not be repeated here. Figure 8 As shown, a method based on Figure 7 Schematic diagram of the transaction table after verification and update, with the newly added "unknown" column as Figure 7 The column that intersects with the circular seal.

[0106] In another embodiment, saving the updated bank flow table row and the updated bank flow table column set as the table extraction result includes: constructing a two-dimensional array of bank flow table cells; assigning the updated bank flow table row and the updated bank flow table column set to the two-dimensional array of bank flow table cells and saving them to a preset file to obtain the table extraction result.

[0107] Specifically, construct a two-dimensional array of bank flow cells M[row+1][col], and convert the bank flow header row l t Assign to M[0][col], and update M b [row][col], that is, the updated bank statement row set is assigned to M[1 to row+1][col]. Then, the assigned bank statement cell two-dimensional array M is updated and saved to the Excel file. At the same time, the cells corresponding to the column containing the text content "unknown" are marked in red, which can be further manually verified by the user. The result is as follows Figure 9 shown.

[0108] The above-mentioned bank statement table extraction method first detects the text regions of the bank statement image and then merges the text regions to construct text rows, obtaining a text row set. It then traverses each text row in the text row set to extract the statement header row and statement body row set. It then divides each text region in the statement body row set into columns based on the distance between the statement header row and each text region in the statement body row set, determining the statement body column set. Furthermore, based on the consistency of the header and body rows and columns, the statement header row and statement body column set are verified and updated, and the verified and updated statement header row and statement body column set are saved as the table extraction result. This method extracts and verifies each row and column in the table through the rows and columns themselves, fully utilizing the characteristics of the table row and column consistency and the alignment of the header and body to achieve bank statement table extraction. This method can use spatial alignment strategies to uniformly process various tables and effectively handle missing elements, thereby greatly improving the accuracy and robustness of bank statement table extraction.

[0109] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2 At least part of the steps in the method may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The order of execution of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps. In addition, Figure 3-Figure 9 The schematic diagrams are mainly used for schematic diagrams of the status of each row and column in the bank flow table when the bank flow image is extracted from the table in each embodiment. Figure 3-Figure 9 The specific bank transaction information content is not important information in each embodiment. Even if the bank transaction information content is unclear, it will not affect the schematic effect and understanding of the embodiment.

[0110] In one embodiment, Figure 10 As shown, a bank statement form extraction device is provided, comprising:

[0111] The detection module 1001 is used to obtain a bank statement image and detect text areas in the bank statement image.

[0112] The merging module 1002 is used to merge text regions to construct text lines and obtain a text line set.

[0113] The extraction module 1003 is configured to traverse each text line in the text line set and extract a flow table header line and a flow table body line set.

[0114] The determination module 1004 is configured to divide each text area in the flow meter body row set into columns according to the distance between the flow meter header row and each text area in the flow meter body row set, and determine the flow meter body column set.

[0115] The verification module 1005 is used to verify and update the flow header row and flow body column set based on the consistency of the header and body rows and columns, and save the verified and updated flow header row, flow body row and flow body column set as the table extraction result.

[0116] In one embodiment, the merging module 1002 is further configured to calculate the vertical overlap ratio between text regions based on the region positions of the text regions; determine a group of text regions that can be merged based on the vertical overlap ratio; and merge the text region groups using a union-find algorithm to obtain a set of text lines.

[0117] In one embodiment, the extraction module 1003 is also used to traverse the text area of ​​each text line in the text line set, match the text content corresponding to the text area of ​​the text line with the header text trigger word, and determine the number of trigger word matches of the text line; determine the flow header row from each text line based on the number of trigger word matches; re-traverse the non-flow header rows in the text line set, use the flow element regular expression to match the flow elements of each non-flow header row respectively, and determine the number of flow element matches; extract the body rows based on the number of flow element matches to obtain a flow body row set.

[0118] In one embodiment, the determination module 1004 is further used to construct a two-dimensional array of body cells based on the number of cells in the flow body row and the number of elements in the flow body row set; traverse the flow body row set to determine the header row position where the distance between each text area in each flow body row and the flow header row is the smallest; add each text area in the flow body row to the two-dimensional array of body cells based on the horizontal overlap ratio between each text area in the two-dimensional array of body cells to obtain a flow body column, and obtain a flow body column set.

[0119] In one embodiment, the determination module 1004 is also used to calculate the distance between each text area in the flow meter body row and each text area in the flow meter header row based on the center point coordinates and width of the text area; compare the distances to determine the minimum distance, and determine the text area position corresponding to the minimum distance from each text area in the flow meter header row as the header row position.

[0120] In one embodiment, the verification module 1005 is also used to determine the total number of text areas in the flow meter body column set and the number of text areas corresponding to each flow meter body column in the flow meter body column set; when the number of text areas exceeds a preset percentage of the total number of text areas, construct a header text block corresponding to the flow meter body column; add the header text block whose area overlaps with the text area in the flow meter header row is less than a threshold to the flow meter header row to obtain an updated flow meter header row; re-divide the columns based on the updated flow meter header row to determine the updated body column set; save the updated flow meter body row and the flow meter body column set as the table extraction result.

[0121] In one embodiment, the verification module 1005 is further used to construct a two-dimensional array of bank flow cells; assign the updated flow table body row and the updated flow table body column set to the two-dimensional array of bank flow cells and save them to a preset file to obtain a table extraction result.

[0122] For the specific definition of the bank statement form extraction device, please refer to the definition of the bank statement form extraction method mentioned above, which will not be repeated here. The various modules in the above-mentioned bank statement form extraction device can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned bank statement form extraction method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form.

[0123] In one embodiment, a computer device is provided, which may be a server and includes a processor, a memory, and a network interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for extracting bank flow forms is implemented. Exemplarily, the computer program can be divided into one or more modules, one or more modules are stored in the memory, and are executed by the processor to complete the present invention. One or more modules can be a series of computer program instruction segments that can perform specific functions, and the instruction segments are used to describe the execution process of the computer program in the computer device.

[0124] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor. The processor is the control center of the computer device and connects various parts of the entire computer device using various interfaces and lines.

[0125] The memory can be used to store the computer programs and / or modules, and the processor implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created based on the use of the mobile phone (such as audio data, a phone book, etc.). In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0126] Those skilled in the art will understand that the computer device structure shown in this embodiment is only a partial structure related to the solution of the present invention and does not constitute a limitation on the computer device to which the present invention is applied. The specific computer device may include more or fewer components, or combine certain components, or have a different component arrangement.

[0127] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0128] Obtain a bank statement image and detect text areas in the bank statement image;

[0129] Merge text regions to construct text lines and obtain a text line set;

[0130] Traverse each text line in the text line set and extract the flow table header line and flow table body line set;

[0131] Divide each text area in the flow meter body row set into columns according to the distance between the flow meter header row and each text area in the flow meter body row set, and determine the flow meter body column set;

[0132] Based on the consistency of the table header and table body rows and columns, the set of transaction header rows and transaction body columns is verified and updated, and the verified and updated set of transaction header rows and transaction body columns is saved as the table extraction result.

[0133] In one embodiment, when the processor executes the computer program, the processor further implements the following steps: calculating the vertical overlap ratio between text areas based on the area positions of the text areas; determining a group of text areas that can be merged based on the vertical overlap ratio; and merging the text area groups using a union-find algorithm to obtain a set of text lines.

[0134] In one embodiment, when the processor executes the computer program, it also implements the following steps: traversing the text area of ​​each text line in the text line set, matching the text content corresponding to the text area of ​​the text line with the header text trigger words, and determining the number of trigger word matches of the text line; determining the flow header row from each text line based on the number of trigger word matches; re-traversing the non-flow header rows in the text line set, using the flow element regular expression to match the flow elements of each non-flow header row respectively, and determining the number of flow element matches; extracting the body rows based on the number of flow element matches to obtain a flow body row set.

[0135] In one embodiment, when the processor executes the computer program, it further implements the following steps: constructing a two-dimensional array of body cells based on the number of cells in the flow body row and the number of elements in the flow body row set; traversing the flow body row set to determine the position of the header row where the distance between each text area in each flow body row and the flow header row is the smallest; adding each text area in the flow body row to the two-dimensional array of body cells based on the horizontal overlap ratio between each text area in the two-dimensional array of body cells to obtain flow body columns, thereby obtaining a flow body column set.

[0136] In one embodiment, when the processor executes the computer program, it also implements the following steps: based on the center point coordinates and width of the text area, calculate the distance between each text area in the flow meter body row and each text area in the flow meter header row; compare each distance to determine the minimum distance, and determine the text area position corresponding to the minimum distance from each text area in the flow meter header row as the header row position.

[0137] In one embodiment, when the processor executes the computer program, the following steps are further implemented: determining the total number of text areas in the flow meter body column set and the number of text areas corresponding to each flow meter body column in the flow meter body column set; when the number of text areas exceeds a preset percentage of the total number of text areas, constructing a header text block corresponding to the flow meter body column; adding the header text block whose area overlaps with the text area in the flow meter header row is less than a threshold to the flow meter header row to obtain an updated flow meter header row; re-dividing the columns based on the updated flow meter header row to determine an updated body column set; and saving the updated flow meter body row and the flow meter body column set as a table extraction result.

[0138] In one embodiment, when the processor executes the computer program, it further implements the following steps: constructing a two-dimensional array of bank flow cells; assigning the updated flow table body rows and the updated flow table body column sets to the two-dimensional array of bank flow cells and saving them to a preset file to obtain a table extraction result.

[0139] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0140] Obtain a bank statement image and detect text areas in the bank statement image;

[0141] Merge text regions to construct text lines and obtain a text line set;

[0142] Traverse each text line in the text line set and extract the flow table header line and flow table body line set;

[0143] Divide each text area in the flow meter body row set into columns according to the distance between the flow meter header row and each text area in the flow meter body row set, and determine the flow meter body column set;

[0144] Based on the consistency of the table header and table body rows and columns, the set of transaction header rows and transaction body columns is verified and updated, and the verified and updated set of transaction header rows and transaction body columns is saved as the table extraction result.

[0145] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: calculating the vertical overlap ratio between text areas based on the area positions of the text areas; determining a group of text areas that can be merged based on the vertical overlap ratio; and merging the text area groups using a union-find algorithm to obtain a set of text lines.

[0146] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: traversing the text area of ​​each text line in the text line set, matching the text content corresponding to the text area of ​​the text line with the header text trigger words, and determining the number of trigger word matches of the text line; determining the flow header row from each text line based on the number of trigger word matches; re-traversing the non-flow header rows in the text line set, matching the flow elements of each non-flow header row using the flow element regular expression, and determining the number of flow element matches; extracting the body rows based on the number of flow element matches to obtain a flow body row set.

[0147] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: constructing a two-dimensional array of body cells based on the number of cells in the flow body row and the number of elements in the flow body row set; traversing the flow body row set to determine the position of the header row where the distance between each text area in each flow body row and the flow header row is the smallest; adding each text area in the flow body row to the two-dimensional array of body cells based on the horizontal overlap ratio between each text area in the two-dimensional array of body cells to obtain flow body columns, thereby obtaining a flow body column set.

[0148] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on the center point coordinates and width of the text area, the distance between each text area in the flow meter body row and each text area in the flow meter header row is calculated; each distance is compared to determine the minimum distance, and the text area position corresponding to the minimum distance is determined from each text area in the flow meter header row as the header row position.

[0149] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: determining the total number of text areas in the flow meter body column set and the number of text areas corresponding to each flow meter body column in the flow meter body column set; when the number of text areas exceeds a preset percentage of the total number of text areas, constructing a header text block corresponding to the flow meter body column; adding the header text block whose area overlaps with the text area in the flow meter header row is less than a threshold to the flow meter header row to obtain an updated flow meter header row; re-dividing columns based on the updated flow meter header row to determine an updated body column set; and saving the updated flow meter body row and flow meter body column set as a table extraction result.

[0150] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: constructing a two-dimensional array of bank transaction cells; assigning the updated transaction table body rows and the updated transaction table body column sets to the two-dimensional array of bank transaction cells and saving them to a preset file to obtain a table extraction result.

[0151] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0152] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0153] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for extracting bank statement forms, characterized in that: include: Acquire a bank statement image, and detect a text area in the bank statement image; Merging the text regions to construct text lines, and obtaining a text line set; Traversing each text line in the text line set, extracting a flow table header line and a flow table body line set; the flow table header line is a header line of a table on a bank flow table image, and the flow table body line set is a set including all flow table body lines, and the flow table body lines are other text lines except the header line; Dividing each text area in the water meter body row set into columns according to the distance between the water meter header row and each text area in the water meter body row set to determine a water meter body column set; Based on the consistency of the table header and table body rows and columns, verify and update the water flow table header row and the water flow table body column set, and save the verified and updated water flow table header row, water flow table body row and water flow table body column set as the table extraction result, specifically including: Determine the total number of text areas in the flow meter body column set and the number of text areas corresponding to each flow meter body column in the flow meter body column set; When the number of the text areas exceeds a preset percentage of the total number of the text areas, constructing a header text block corresponding to the flow table body column; Adding the header text block whose area overlaps with the text area in the flow table header row is less than a threshold to the flow table header row to obtain an updated flow table header row; Re-dividing the columns based on the updated flow table header row to determine the updated flow table body column set; The updated flow table header row, flow table body row, and flow table body column set are saved as the table extraction result.

2. The method according to claim 1, characterized in that The merging of the text regions to construct text lines to obtain a text line set includes: Calculating a vertical overlap ratio between the text regions according to the region positions of the text regions; Determining a group of text regions that can be merged according to the vertical overlap ratio; The text region groups are merged using a union-find algorithm to obtain a text line set.

3. The method according to claim 1, characterized in that The step of traversing each text line in the text line set and extracting a flow table header line and a flow table body line set includes: Traversing the text areas of the text lines in the text line set, matching the text content corresponding to the text areas of the text lines with the header text trigger words, and determining the number of trigger word matches of the text lines; Determining a flow table header row from each of the text rows according to the number of trigger word matches; Retraverse the non-flowing header rows in the text row set, match the flow elements of each non-flowing header row using the flow element regular expression, and determine the number of flow element matches; Table body rows are extracted according to the matching quantity of the flow elements to obtain a flow table body row set.

4. The method according to claim 1, wherein The dividing each text area in the water meter body row set into columns according to the distance between the water meter header row and each text area in the water meter body row set to determine the water meter body column set includes: Constructing a two-dimensional array of table body cells according to the number of cells in the flow table body row and the number of elements in the flow table body row set; Traversing the set of flow meter body rows, determining the position of the header row where the distance between each text area in each flow meter body row and the flow meter header row is the smallest; According to the position of the table header row, each text area in the flow table body row is added to the two-dimensional array of table body cells; Based on the horizontal overlap ratio between the text areas in the two-dimensional array of table body cells, a flow table body column is obtained, and a flow table body column set is obtained.

5. The method according to claim 4, characterized in that The determining of the position of the header row where the distance between each text area in each flow table body row and the flow table header row is the smallest includes: Calculating the distance between each text area in the flow meter body row and each text area in the flow meter header row based on the center point coordinates and width of the text area; The distances are compared to determine a minimum distance, and a text area position corresponding to the minimum distance is determined from each text area of ​​the flow table header row as the table header row position.

6. The method according to claim 1, characterized in that Saving the updated flow table body row and the updated flow table body column set as the table extraction result includes: Construct a two-dimensional array of bank flow cells; The updated transaction table body row and the updated transaction table body column set are assigned to the bank transaction cell two-dimensional array and saved in a preset file to obtain a table extraction result.

7. A bank statement form extraction device, characterized in that: include: A detection module, configured to acquire a bank statement image and detect a text area in the bank statement image; A merging module, configured to merge the text regions to construct text lines and obtain a text line set; an extraction module, configured to traverse each text line in the text line set and extract a flow table header line and a flow table body line set; the flow table header line is a header line of a table on a bank flow table image; the flow table body line set is a set including all flow table body lines, and the flow table body lines are text lines other than the header line; a determination module, configured to divide each text area in the water meter body row set into columns according to the distance between the water meter header row and each text area in the water meter body row set, and determine a water meter body column set; A verification module is used to verify and update the water table header row and the water table body column set based on the consistency of the table header and table body rows and columns, and save the verified and updated water table header row, water table body row and water table body column set as a table extraction result, specifically used to: determine the total number of text areas in the water table body column set and the number of text areas corresponding to each water table body column in the water table body column set; when the number of text areas exceeds a preset percentage of the total number of text areas, construct a table header text block corresponding to the water table body column; add the table header text block whose area overlaps with the text area in the water table header row is less than a threshold to the water table header row to obtain an updated water table header row; re-divide the columns based on the updated water table header row to determine an updated water table body column set; save the updated water table header row, water table body row and water table body column set as a table extraction result.

8. A computer device comprising a processor and a memory, wherein the memory stores a computer program, wherein: The processor is used to implement the bank statement form extraction method described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the bank statement form extraction method according to any one of claims 1 to 6 is implemented.

Images