Table area positioning correction method based on image edge detection

Abstract

The invention relates to the technical field of scanning image processing, in particular to a table area positioning correction method based on image edge detection. The method comprises the following specific implementation steps: acquiring an original image of a financial statement, constructing a collaborative feedback link based on an instance segmentation network and an edge detection algorithm, extracting a semantic mask through the instance segmentation network, performing pattern recognition by using the edge detection algorithm, extracting a geometric structure, and forming a collaborative feedback and accurate recognition table through an attention mechanism; establishing a table structure topological graph, carrying out graph matching on the table structure topological graph and a typical grid graph selected from a database, calculating attitude correction parameters, and correcting the image; and generating a multi-channel repair guide graph containing lines, contents and defect positions, and guiding the table repair network to repair table defects.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of scanning image processing, in particular to a table region positioning correction method based on image edge detection. BACKGROUND

[0002] With the global digital wave, the financial industry urgently needs to convert the important information carried by the massive paper documents into structured data that computers can understand and calculate. In the core business processes such as bank credit approval, insurance claims, and enterprise tax automation, the system needs to process tens of thousands of scanned financial statements, bank statements and other documents every day. The table in these documents is the carrier of key data, so accurate identification and extraction of the structure of the table are the core prerequisite for realizing business process automation, big data risk control and business intelligence decision-making.

[0003] However, in actual digitalization collection, physical defects and collection noise cause serious degradation of the visual pattern of the table, and the automatic recognition faces serious challenges. The broken lines destroy the continuity of the table pattern, leading to ambiguity in OCR recognition; the loss of small frame lines causes the loss of structural information, making it impossible to reason about the complete layout of the layout; and document tilt distortion fundamentally challenges the pose positioning and pattern matching algorithm.

[0004] The existing method is limited to direct analysis of visible pixel features, and lacks the ability to reason and complete the inherent grid structure pattern of the table. This makes it difficult for existing methods to reliably identify the complete topological relationship and correct geometric pose of the table from incomplete visual information, thereby introducing errors and risks in the automated data extraction process.

[0005] Therefore, a table region positioning correction method based on image edge detection is proposed. SUMMARY

[0006] The purpose of the present application is to provide a table region positioning correction method based on image edge detection to solve the problems raised in the background.

[0007] To achieve the above purpose, the present application provides the following technical solution: a table region positioning correction method based on image edge detection, the specific implementation method comprising: Obtain the original image edge detection algorithm of the financial report; Perform table semantic recognition on the original image through an instance segmentation network to generate a basic mask; within the limited range of the basic mask, perform geometric pattern recognition to generate a geometric structure skeleton; feed back the geometric structure skeleton to the instance segmentation network; form a collaborative feedback link based on the instance segmentation network and the edge detection algorithm, and iteratively converge to generate a single cell structure mask and edge structure information; establish a table structure topology graph based on the cell structure mask and the edge structure information; perform graph matching on the table structure topology graph and a highest adaptation grid graph in a financial report database, and then calculate a pose correction parameter; based on the pose correction parameter, perform geometric correction on the original image and the structure mask to generate a pose correction image and a corrected mask; generate a multi-channel repair guide image according to the corrected mask and the edge structure information; train a table repair network by using the financial report database; based on the multi-channel repair guide image, perform table repair on the pose correction image by using the table repair network to generate a final correction image.

[0008] Preferably, the collaborative feedback link based on the instance segmentation network and the edge detection algorithm specifically includes: input the original image into the instance segmentation network based on MaskR-CNN for initial forward reasoning; the instance segmentation network based on MaskR-CNN identifies and generates a basic mask through context semantic understanding of the original image; based on the basic mask, use an edge detection algorithm to extract structural features of the original image to generate a geometric structure skeleton; encode the geometric structure skeleton into a spatial attention map through an attention encoder; fuse the spatial attention map as a feedback signal with the original image to generate guide parameters required for the next round of iteration, forming a collaborative feedback link; the collaborative feedback link is iterated cyclically until the geometric structure skeleton converges, generating a cell structure mask and edge structure information.

[0009] Preferably, the process of generating a geometric structure skeleton based on a basic mask and using an edge detection algorithm for geometric pattern recognition specifically includes: limit the image area based on the basic mask, and use a Canny operator to extract candidate edge pixel points constituting a table outline in the limited area to generate an edge feature image; use Hough transform to extract linear structures from the edge feature image, and integrate all extracted linear structures to form the geometric structure skeleton.

[0010] Preferably, the process of establishing a table structure topology graph based on a cell structure mask and edge structure information specifically includes: perform instance analysis on the cell structure mask, identify the cell structure mask boundary and geometric center point, and abstract them into a topology graph node representing a cell instance; record the positioning coordinates of the cell instance in the image in the topology graph node; based on the edge structure information, identify the connection relationship between the generated nodes, extract a table line segment connecting two adjacent nodes, and abstract the table line segment into an edge of the topology graph; all identified nodes and edges representing the topology relationship together constitute a table structure topology graph.

[0011] Preferably, the table structure topology graph is matched with the highest adaptation grid graph in the financial report database, and then the posture correction parameter is calculated, and the specific implementation process comprises: Various types of financial reports are collected, classified and stored according to types, and a financial report database is established; The table structure topology graph is subjected to mode analysis, and key topological features including the number of rows and columns and the relative spacing of cells are extracted; the key topological features are identified in the financial report database, and a typical financial report grid graph is extracted according to a maximum matching algorithm; the table structure topology graph is matched with the typical financial report grid graph through a global optimization algorithm, and an affine transformation matrix capable of minimizing the position deviation of all corresponding nodes and the angle deviation of corresponding edges between the two graphs is iteratively solved as the posture correction parameter.

[0012] Preferably, the multi-channel repair guide graph generated according to the corrected mask and the edge structure information specifically comprises: A binary mask reflecting a complete table frame is generated based on the edge structure information as a line structure channel; a binary mask identifying all non-edge regions is generated by performing a logical set operation on all cell masks in the corrected mask as a content area channel; the line structure channel and the image after posture correction are compared and analyzed to identify specific regions with pixel loss and line break, and a binary mask marking the repair position is generated as a defect marking channel; the line structure channel, the content area channel and the defect marking channel are combined to form a multi-channel repair guide graph.

[0013] Preferably, the table repair network is trained using the financial report database, and the specific implementation process comprises: Complete financial reports are extracted from the financial report database; the complete financial reports are processed into paired training samples, the complete financial reports are randomly damaged to generate damaged table images; line structure channels, content area channels and defect marking channels are extracted based on the complete financial reports; three channels are combined to generate sample repair guide graphs; complete financial reports, damaged table images and sample repair guide graphs are combined to construct paired training samples; The damaged table image and the sample repair guide graph are spliced in the channel dimension, input into the table repair network, and a predicted repair image is generated by inference; an overall error between the predicted repair image and the complete financial report is calculated by using an error function; the overall error is combined, and the internal weight parameters of the table repair network are iteratively updated by using a back propagation algorithm based on gradient descent until the model converges, thereby generating a trained table repair network.

[0014] Compared with the prior art, the present application has the following advantages: 1. The invention designs a cooperative feedback link that combines the powerful semantic understanding ability of the deep learning model recognition unit with the precise geometric analysis ability of the edge detection algorithm to extract straight skeletons. Through iterative feedback, the semantic recognition results provide the initial area for geometric extraction, while the extracted geometric structure is reversely guided and corrected in the form of spatial attention map, both of which work together and refine each other. This mechanism fundamentally overcomes the poor stability of traditional algorithms in the face of image noise and distortion, and solves the problem that existing deep learning models cannot understand and complete logical structures, thereby greatly improving the robustness of table positioning and the accuracy of cell segmentation when dealing with financial statements of varying quality in the real world.

[0015] 2. Unlike the existing technology that relies on bounding boxes or corner points for rough correction, the invention constructs a table structure topology graph that reflects the internal logical relationship of the table and matches it with a typical financial statement grid graph generated based on a large amount of data learning. This global topology structure optimization-based method can calculate the precise pose parameters, i.e., affine transformation matrix, effectively dealing with complex nonlinear distortion caused by shooting angle, paper curling, etc., and achieving flat processing of the table. This provides an ideal, distortion-free image basis for subsequent structure repair and OCR recognition.

[0016] 3. The invention proposes a scheme to generate a multi-channel repair guide map, providing the repair network with unprecedented rich prior information. The guide map not only marks the defect location, but also clearly defines the complete line structure of the table, the content area that must be protected, and the content area channel. This makes table repair no longer a blind pixel-level filling, but an intelligent reasoning based on the logical structure of the table. The repair network can understand where to draw straight lines and where to preserve text, thereby accurately restoring broken frame lines, removing stains, and protecting key financial data, outputting a complete structure and clear content table image.

[0017] 4. The generation of the typical financial statement grid graph and the training of the table repair network in the invention both rely on large-scale, diverse financial statement data sets. This makes the method not limited to a specific version of the report, but has good adaptability and generalization ability for various financial statements. In the massive document processing scenario of the financial industry, the method can complete the task stably and adaptively, significantly reducing the dependence on manual correction, and providing key technical support for realizing end-to-end automation of financial business processes. BRIEF DESCRIPTION OF DRAWINGS

[0018] Fig. 1 A flowchart of a table region positioning and correction method based on image edge detection is proposed for the embodiments of the invention. Fig. 2 The structural schematic diagram of the cooperative feedback link proposed for the embodiment of the application is shown in the figure. Fig. 3 The structural schematic diagram of the multi-channel repair guide map proposed for the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0019] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described in detail below in conjunction with specific embodiments. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0020] Please refer to Figs. 1 to 3 The present application relates to a table region positioning correction method based on image edge detection, and the specific implementation steps are as follows: Obtain the original image of the financial statement; Extract the basic mask from the original image by using the instance segmentation network based on MaskR-CNN; in the region indicated by the basic mask, extract the geometric structure skeleton from the original image by using the edge detection algorithm; feed back the geometric structure skeleton to the instance segmentation network based on MaskR-CNN to form a cooperative feedback link based on the instance segmentation network and the edge detection algorithm, and iteratively converge to generate a single cell structure mask and edge structure information; Establish a table structure topology graph based on the single cell structure mask and the edge structure information; perform graph matching on the table structure topology graph and the highest adaptive financial statement grid graph in the financial statement database, and then calculate the pose correction parameters; Based on the pose correction parameters, perform affine transformation on the original image and the single cell structure mask to obtain a pose correction image and a corrected mask; Generate a multi-channel repair guide map according to the corrected mask and the edge structure information; train a table repair network by using the financial statement database; repair the table defects by using the table repair network based on the multi-channel repair guide map to generate a final correction image.

[0021] The technical solutions of the present application will be further described in detail below in conjunction with specific embodiments.

[0022] Embodiment one The embodiment of the present application discloses a table region positioning correction method based on image edge detection, which corrects the table region positioning of the balance sheet A, and refers to Fig. 1 The specific implementation steps of the method proposed by the present application include: S1, obtain the original image of the financial statement; S2, extracting a basic mask from the original image by using an instance segmentation network based on MaskR-CNN; within the region indicated by the basic mask, extracting a geometric structure skeleton from the original image by using an edge detection algorithm; feeding back the geometric structure skeleton to the instance segmentation network to form a collaborative feedback link based on the instance segmentation network and the edge detection algorithm, and iteratively converging to generate a single cell structure mask and edge structure information; S3, establishing a table structure topology graph based on the single cell structure mask and the edge structure information; performing graph matching on the table structure topology graph and a highest adaptation financial statement grid graph in a financial statement database, and then calculating a pose correction parameter; S4, performing affine transformation on the original image and the single cell structure mask based on the pose correction parameter to obtain a pose correction image and a corrected mask; S5, generating a multi-channel repair guide image according to the corrected mask and the edge structure information; training a table repair network by using a financial statement database; and repairing a table defect by using the table repair network based on the multi-channel repair guide image to generate a final correction image.

[0023] Further, an original image of the balance sheet A uploaded is obtained; and corresponding to the above S1, the specific process is as follows: Specifically, the processing object is a scanned image of an A4 paper balance sheet, and the size is pixels, and there is an approximately 3-degree counterclockwise tilt. There is a mark circled with a red pen in the "liabilities and owner's equity" part, which covers the border of two cells.

[0024] Further, a basic mask is extracted from the original image of the balance sheet A by using an instance segmentation network based on MaskR-CNN; within the region indicated by the basic mask, a geometric structure skeleton is extracted from the original image by using an edge detection algorithm; the geometric structure skeleton is fed back to the instance segmentation network to form a collaborative feedback link based on the instance segmentation network and the edge detection algorithm, and iteratively converging to generate a single cell structure mask and edge structure information; and corresponding to the above S2, the specific implementation process includes: Specifically, the original image is input into a pre-trained instance segmentation network based on MaskR-CNN for preliminary identification. Due to the strong feature interference of the red pen mark, the model incorrectly identifies the two covered cells as one large and irregular mask. The generated mask set with defects in this stage is called a basic mask. The network is pre-trained on more than ten thousand various financial statement image datasets, and can preliminarily identify the text region and table cell in the image.

[0025] Using edge detection algorithm, within the image region defined by the base mask, the Canny operator is applied to extract edge pixel points, forming a preliminary edge map, wherein the low threshold is set to 50 and the high threshold is 15. Then the Hough transform is applied to the edge map to identify straight line segments with a length exceeding 50 pixels, which constitute the geometric structure skeleton.

[0026] The geometric structure skeleton, i.e. a data set containing line coordinates and angles, is converted into a spatial attention map consistent with the internal feature map size of the instance segmentation network based on MaskR-CNN through an attention encoder. The weight of the real table line position on the map is higher, and the weight of the non-table line area is lower. The spatial attention map is weighted and fused with the deep feature map of MR-CNN to guide the network to pay more attention to the identified geometric structure in the next round of reasoning. In the second reasoning, the network will pay more attention to the area enhanced by the spatial attention map, thereby generating a cell mask that is more consistent with the actual table line. This process is repeated until the straight line segment difference between the new and old skeletons meets the multi-dimensional convergence judgment mechanism.

[0027] Specifically, the multi-dimensional convergence judgment mechanism is: After each iteration, four stability indicators are calculated, including the number stability, i.e. the difference in the total number of straight line segments in the new and old structures; the position stability, i.e. the average deviation distance of the straight line segments in the new and old structures; the angle stability, i.e. the average angle change amplitude of the straight line segments in the new and old structures; and the topological stability, i.e. the number change of key intersection points in the geometric structure skeleton, including cross and T-shaped intersection points. Then the state difference vector is calculated, and the convergence inertia, i.e. the difference change rate, is calculated according to the state difference vector, with the formula being ; wherein, is the convergence inertia of the kth round, is the state difference vector of the kth round, represents the calculation of the Euclidean distance. When the convergence inertia for three consecutive periods and is less than the threshold value 0.05, it is determined that the geometric structure skeleton has reached a stable state, i.e. the geometric structure skeleton converges.

[0028] In this embodiment, after 5 iterations, the geometric structure skeleton extracted by the cooperative feedback link tends to be stable, and at this time the cell structure mask and the structural positioning information of each table line are obtained.

[0029] The cooperative feedback link can significantly improve the accuracy and reliability of cell segmentation. The traditional single inference network is prone to errors, while the present scheme uses semantic guidance geometry and geometric refinement semantic cycle to enable the model to self-correct and optimize, so that the cell mask and the structural positioning information converge to a better solution in the iteration, solving the problem of inaccurate initial segmentation.

[0030] The edge detection algorithm ensures that the geometric skeleton of the table is stably and reliably extracted from the imperfect mask. Through the Canny algorithm and the Hough transform, noise can be effectively filtered out, and the extraction of linear features is focused on. The extracted structural positioning information provides high-quality geometric feedback information for the collaborative feedback, ensuring that the entire feedback system can work effectively.

[0031] Further, a table structure topology graph is established based on the cell structure mask and the edge structure information; the table structure topology graph is graph-matched with a highest-adaptation financial statement grid graph in a financial statement database, and then a pose correction parameter is calculated; corresponding to the S3 step, the specific implementation process includes: Each independent cell mask instance in the cell structure mask is traversed, and the first moment and the zero moment of each mask are calculated according to the partition distribution, and the accurate coordinates of the geometric center point are further obtained. The geometric center point is abstracted as a topology graph node. The unit boundary and the geometric center point coordinates of the cell instance grid are stored in the corresponding topology node attribute as the positioning coordinates. After all the nodes are generated, the connection relationship between the generated nodes is identified according to the edge structure information. Each line segment in the edge structure information is traversed, and it is searched whether there are two topology nodes within a baseline distance threshold around it, and the baseline distance threshold is set to 5 pixels in this embodiment. After traversing all the line segments and completing the above matching, all the identified nodes and the edges representing the topology relationship together constitute a table structure topology graph that can accurately and completely reflect the structure of the balance sheet.

[0032] The above method of establishing a table structure topology graph realizes the abstraction and reconstruction representation of the table image structure. Compared with directly processing pixels or boundaries, abstracting the table into a topology graph containing nodes and edges can more essentially describe the row-column relationship and sparsity of the table. This representation method is more robust for subsequent global matching and local correction, and is not easily affected by local, contour thickness changes or sparse noise.

[0033] Further, the table structure topology graph is graph-matched with a highest-adaptation financial statement grid graph in a financial statement database, and then a pose correction parameter is calculated; corresponding to the S3 step, the specific implementation process includes: First, more than five thousand standard asset balance sheets, profit statements and other financial statements are collected, classified according to types, and a financial statement database is established. Then, the table structure topology graph generated in this embodiment is analyzed, and key topology features such as the number of rows and columns and the relative spacing of the cells are extracted. According to the key topology features, the highest-matching algorithm is used to obtain the typical financial statement with the highest matching degree in the financial statement database.

[0034] In particular, the maximum matching algorithm is: Extract key topological features, including the total number of rows and columns of the table, and the merged cell pattern. Extract auxiliary features, including node adjacency, and cell span. Through key topological features, a first round of screening and filtering is performed from the entire financial statement database to frame a highly similar candidate set of financial statements. Use auxiliary features to construct a matching degree function. The higher the overlap with auxiliary features, the higher the matching degree score. Select the financial statement with the highest matching degree score as the typical financial statement.

[0035] Through a random sample consensus algorithm, an affine transformation matrix that minimizes the topological structure error between the balance sheet A and the typical financial statement is iteratively solved. This matrix accurately describes the 3-degree tilt of the original image.

[0036] The above graph matching and parameter calculation method provides a matching benchmark that is expected to be strong and typical. By establishing a financial grid graph database and using an algorithm to screen, the graph matching can better adapt to various versions and specifications of the report, enhancing the universality of the method. In addition, by converting the table topography into a topological graph and performing graph matching with the standard template, a more accurate affine transformation matrix can be calculated, successfully flattening the deformed table and creating ideal conditions for subsequent repair and OCR.

[0037] Further, based on the pose correction parameters, affine transformation is performed on the original image and the cell structure mask to obtain a pose correction image and a corrected mask; corresponding to the above S4 step, the specific implementation process includes: Based on the solved affine transformation matrix, affine transformation is performed on the original image and the cell structure mask to obtain a completely horizontal pose correction image and a corrected mask.

[0038] Further, a multi-channel repair guide image for repairing the asset deposit table A is generated according to the corrected mask and the edge structure information; corresponding to the above S5 step, the specific implementation process includes: Specifically, a binary mask reflecting the complete table framework, i.e. the line structure channel, is generated according to the edge structure information; a binary mask identifying all non-edge content carrying areas, i.e. the content area channel, is generated by performing a logical union operation on all cell masks in the corrected mask; the line structure channel and the pose-corrected image are compared and analyzed to identify specific areas with pixel missing and crack breaks due to red pen markings, and a binary mask marking the repair location, i.e. the defect marking channel, is generated. Finally, the line structure channel, the content area channel and the defect marking channel are combined to form a multi-channel repair guide image.

[0039] The multi-channel repair guide image provides comprehensive prior information for the table repair network, not only indicating the image defects, but also indicating the ideal state of the table structure and the content area that needs to be protected. The shaped multi-channel repair guide image can make the repair task more clear, thereby achieving the precision and fidelity far exceeding the traditional repair image method.

[0040] Further, the table repair network is trained using a financial statement database; based on the multi-channel repair guide image, the table repair network is used to repair the table defects to generate a final corrected image; corresponding to the S5 step, the specific implementation process includes: More than 10,000 high-definition balance sheet images are selected from the financial statement database and processed into paired training samples. For each report image, random defects are processed to simulate various image defects that may be encountered in actual financial business scenarios to generate a defective table image. Based on the original complete financial statement, the line structure channel, the content area channel and the defect marking channel are extracted, and the sample repair guide image is generated in combination with the three channels. The complete financial statement image, the defective table image and the sample repair guide image are combined to construct a paired training sample.

[0041] A U-Net with an attention mechanism is used as the basic architecture of the table repair network. The architecture includes an encoder path to capture the context features of the image and a symmetric decoder path. The attention module is integrated into the skip connection between the encoder and the decoder, allowing the network to pay more attention to important structural areas indicated by the guide image.

[0042] In each training batch, the defective table image and the corresponding sample repair guide image are spliced in the channel dimension to form a 4-channel tensor as the input of the repair network.

[0043] In order to ensure the accuracy and authenticity of the repair, a composite loss function is used to calculate the overall error between the predicted repair image and the ideal complete table image. The composite loss function is composed of three parts: The structural similarity loss is used to measure the similarity of two images in brightness, contrast and structure. This loss function ensures that the repaired table is consistent with the original table in macro structure, avoiding structural distortion or blurring.

[0044] The mask weighted reconstruction loss uses the content area channel and the defect marking channel of the sample repair guide image as a weight mask to weight the traditional L1 pixel loss, focusing the loss calculation on the most critical pixel subset, achieving strong protection of the content and key supervision of the repair area.

[0045] The structure gradient loss uses a Laplacian operator to extract the edges of the image, and calculates the difference between the predicted image and the real image on the edge map. By focusing on the rate of change between pixels, the repair network is effectively constrained to generate a repair result that conforms to the line characteristics, improving the repair quality of the lines.

[0046] The three loss structures are weighted to form a composite loss function .

[0047] An Adam optimizer based on gradient descent is used to calculate the overall error The gradient of the network internal weight parameter is calculated, and the iterative update is performed in the opposite direction of the gradient. The training process is carried out on a high-performance GPU cluster, with a batch size of 16 and an initial learning rate of 0.0002. By monitoring the loss value on the independent validation set, when the validation loss does not decrease for several consecutive periods, the network model is considered to have converged. After training is completed, the weight parameters of the network model are saved. Finally, the table repair network is obtained.

[0048] Finally, the posture corrected image of the balance sheet A is input into the trained table repair network together with the corresponding multi-channel repair guide image, and the corrected image is output by the table repair network, wherein the table lines below the red pen mark are completely repaired.

[0049] A deep learning model specifically for table repair tasks is obtained. By constructing specific training sample pairs containing incomplete images, ideal images and guide images, the network learns how to use structured prior information for intelligent repair, and its repair capability is far superior to that of general image repair networks, especially in restoring the coherence of table lines and protecting the integrity of text content The overall technical scheme of the present application proposes a comprehensive solution combining traditional image recognition and neural networks, which overcomes the limitations of single methods in processing severely distorted or low-quality images through instance segmentation network and edge detection algorithm cooperative feedback, significantly improving the accuracy of table edge structure recognition. Based on graph matching rather than simple edge detection, the posture calculation can obtain globally optimal geometric correction parameters, effectively dealing with complex nonlinear distortion. A multi-channel repair guide image is designed to provide rich structural prior knowledge for the table repair network, so that the repair is no longer blind pixel filling, but targeted, structured repair conforming to the table logic. The repaired image has clear lines and complete content, greatly facilitating subsequent OCR processing.

[0050] Embodiment two This embodiment aims to illustrate the specific workflow of the present application in processing a bank statement scan with part of the table lines broken due to printing quality problems, and in particular to show how the cooperative feedback link corrects the preliminary identification errors, please refer toFig. 2 .

[0051] The processing object is a partial image of a bank statement taken from the enterprise credit application materials, with a size of pixels. The image contains columns such as "transaction date", "abstract", "income", "expense", etc., but one of the key horizontal lines separating two transactions is broken in the middle due to printing quality problems.

[0052] The pixel original image is input into the pre-trained instance segmentation network. Due to the 40-pixel break in the horizontal line, the network cannot visually find a clear closed boundary, so it incorrectly merges two cells that should be independent, in this embodiment, the expense amount cells belonging to two transactions, into one irregular cell mask , and other clear lines are correctly identified.

[0053] Based on the preliminary mask, the geometric skeleton is extracted. The Canny operator is applied in the image region defined by the preliminary cell mask, including the error . The low threshold is set to 50 and the high threshold is set to 150. The operator calculates the gradient intensity of the pixel neighborhood, and uses the double threshold method to filter and connect edge points. A preliminary binary edge feature map is generated. In the figure, all clear table lines are successfully detected as continuous edge pixel chains. For that broken horizontal line, the unbroken parts on the left and right are also detected, but there is a clear pixel break in the middle. The Hough transform is applied to the above edge feature map to identify straight line segments. Since the gap at the break is greater than the maximum gap set by the Hough transform, the Hough transform identifies the broken horizontal line as two independent, discontinuous line segments.

[0054] All extracted line segments are passed through an attention encoder to generate a 800x600 spatial attention map. In this map, the pixel positions corresponding to the identified line segments are assigned high weight values, in this case 1.0, and other regions are assigned low weight values, in this case 0.1. This attention map is fed back to the instance segmentation network and fused with the backbone network in the deep feature map.

[0055] The instance segmentation network uses the enhanced feature data fused with the attention map for a second inference. Although there is a visual break in the original image, the context information provided by the attention map guides the network to tend to segment at that location. The network successfully separates the originally incorrect cell mask The correction to two independent, more regular boundary cell masks correctly separates the two transaction records. In the third iteration, the more complete attention map further guides the segmentation network to generate nearly perfect cell masks. When the geometric skeleton is extracted again, the result is exactly the same as the skeleton data of the second iteration, and it is judged that the geometric structure skeleton has converged.

[0056] After the above-mentioned cooperative feedback iteration process, the embodiment finally outputs two key information, the cell mask, a set of masks that accurately correspond to the actual table cells and have clear boundaries; and the structure positioning information, a geometric structure skeleton data containing all table lines.

[0057] Embodiment three The embodiment describes in detail the process of extracting a table structure topology from an inclined profit table image, matching a highly adaptive financial report image, and calculating pose correction parameters for geometric correction through image matching.

[0058] The processing object is a profit table image with about 15 degrees of inclination and slight perspective distortion. The exact mask and edge information of all cells in the image have been extracted.

[0059] The original image is quickly pre-processed to extract its rough topological features, including the number of rows and columns, and quickly located to the approximate positions of several key anchor cell such as "total profit" and "revenue" through keyword-based OCR. The extracted rough features are matched with each typical financial report image in the database to calculate the matching degree. The matching degree score reflects the degree of agreement between the image and the typical financial report image structure. Through comparison, the matching degree score of image A in the database is the highest, which is 0.93. Therefore, it is determined that it is the financial report image with the highest adaptation degree for the original image.

[0060] The geometric center point coordinates of each cell mask in the image are calculated. The center point of the "revenue" cell mask is calculated as , and the center point of the "cost of sales" cell mask is calculated as Based on the edge structure information, a nearly vertical table line connecting the "revenue" cell and the "cost of sales" cell is identified, and its geometric properties such as direction and length are recorded as an edge connecting the two nodes. A table structure topology graph containing all cell center points and connecting lines is constructed. An affine transformation matrix M with the smallest topology error is solved, an error function is defined, and in this example, the mean square error of the coordinates of all matched node pairs after affine transformation is used, and the least squares method is used to solve it. A 2x3 affine transformation matrix M is calculated.

[0061] ; This matrix precisely describes the composite transformation, i.e. pose correction parameters, required to correct the image, including rotation, scaling and translation.

[0062] Embodiment Four This embodiment describes the process of generating the multi-channel guide map for table repair in detail, referring to Fig. 3 , and the process of training and applying the table repair network. This embodiment takes an image of a corrected but still defective cash flow statement as an example for illustration.

[0063] An image of an enterprise cash flow statement that has been accurately pose-corrected through the previous steps, with a size of pixels. Although the geometry of the image is standard, it still has the following defects: a horizontal dividing line below the "Net cash flow from operating activities" is broken into a 60-pixel gap due to the fold in the original document; there is a pixel stain on the corresponding amount of "Net cash flow from investing activities", which obscures part of the number.

[0064] Create a pixel black background. According to the existing complete geometric structure skeleton information, draw all the table line segments that should exist on the background as white, with a line width of 2 pixels, to generate a binary mask, which is the line structure channel.

[0065] Compare the line structure channel with the pose-corrected image at the pixel level to find the pixel points that exist on the ideal line but are missing in the actual image. Apply a blob detection algorithm to the pose-corrected image to find areas that do not belong to text or lines, which meet the size characteristics of stains, such as connected domains with an area of 100 to 2000 pixels in this example. Merge the two defect areas to generate the final defect marking channel.

[0066] Combine all known cell masks logically, i.e. combine all masks into a single binary image that precisely tells the repair network which areas are content that needs to be protected, to generate the content area channel.

[0067] Stack the above three channels to form a three-channel guide map.

[0068] The table repair network uses a U-Net neural network, which consists of an encoder part for feature extraction, a decoder part for upsampling and detail restoration, and an output part that outputs the repaired image.

[0069] A training set containing 50,000 pairs of financial statement samples is established. The incomplete table image and the multi-channel repair guide image are spliced in the channel dimension to form a 4-channel tensor as the input of the U-Net, and the network outputs a predicted repair image. The overall error between the predicted image and the ideal complete table image is calculated using a composite loss function, and the internal weight parameters of the U-Net are iteratively updated according to the overall error by using the Adam optimizer based on gradient descent and the back propagation algorithm.

[0070] After about 100 cycles of iterative training, when the loss of the network on the validation set no longer decreases, the model converges. Finally, a trained table repair network capable of intelligently repairing table defects according to structured guide information is obtained.

[0071] The posture-corrected image of the cash flow statement in this embodiment and the corresponding multi-channel repair guide image are spliced to form an input tensor of The tensor is input into the trained table repair network for a forward inference, and the table repair network outputs a final corrected image of .

[0072] Although embodiments of the present application have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A table area positioning and correction method based on image edge detection, characterized in that: The following stages are included: Obtain an original image of a financial report; perform table semantic recognition on the original image using an instance segmentation network to generate a basic mask; perform geometric pattern recognition using an edge detection algorithm within the range defined by the basic mask to generate a geometric structure skeleton; Feed the geometric structure skeleton into the instance segmentation network; A collaborative feedback link is formed based on the instance segmentation network and the edge detection algorithm, and the cell structure mask and edge structure information are generated through iterative convergence; Establishing a table structure topology map based on the cell structure mask and edge structure information; performing graph matching on the table structure topology map and the highest-fitting grid map in the financial report database, and then calculating the posture correction parameters; Based on the posture correction parameters, geometrically correct the original image and the structure mask to generate a posture-corrected image and a corrected mask; A multi-channel restoration guide map is generated according to the corrected mask and edge structure information; a table restoration network is trained using a financial report database; and based on the multi-channel restoration guide map, the table restoration network is used to perform table restoration on the posture-corrected image to generate a final corrected image.

2. The table area positioning and correction method based on image edge detection according to claim 1, characterized in that: The collaborative feedback link based on the instance segmentation network and the edge detection algorithm is specifically: Inputting the original image into an instance segmentation network based on Mask R-CNN for initial forward reasoning; the instance segmentation network recognizes and generates a basic mask by understanding the contextual semantics of the original image; Based on the basic mask, the edge detection algorithm is used to perform geometric pattern recognition on the original image to generate a geometric structure skeleton; Encoding the geometric structure skeleton into a spatial attention map through an attention encoder; The spatial attention map is used as a feedback signal and fused with the original image to generate the guidance parameters required for the next round of iteration, forming a collaborative feedback link; The collaborative feedback link iterates cyclically until the geometric structure skeleton converges, generating a cell structure mask and edge structure information.

3. The table area positioning and correction method based on image edge detection according to claim 2, characterized in that: The specific implementation process of performing geometric pattern recognition on the original image based on the basic mask and generating the geometric structure skeleton by using the edge detection algorithm includes: The image area is limited based on the basic mask, and the Canny operator is used to extract candidate edge pixel points constituting the table outline in the limited area to generate an edge feature image; the linear structure of the edge feature image is extracted using the Hough transform, and all the extracted linear structures are integrated to form the geometric structure skeleton.

4. The table area positioning and correction method based on image edge detection according to claim 1, characterized in that: The specific implementation process of establishing a table structure topology map based on the cell structure mask and edge structure information includes: An instance analysis is performed on the cell structure mask to identify the boundaries and geometric center points of the cell structure mask and abstract them into topological map nodes representing the cell instances; the positioning coordinates of the cell instances in the image are recorded in the topological map nodes; based on the edge structure information, the connection relationship is identified between the generated nodes, and the table line segments connecting two adjacent nodes are extracted, and the table line segments are abstracted into the edges of the topological map; all the identified nodes and the edges representing the topological relationships together constitute the table structure topological map.

5. The table area positioning and correction method based on image edge detection according to claim 4, characterized in that: The table structure topology graph is matched with the highest-fit grid graph in the financial report database, and then the posture correction parameters are calculated. The specific implementation process includes: Collect various types of financial statements, classify and store them by type, and establish a financial statement database; The table structure topology graph is subjected to pattern analysis to extract key topological features, including the number of rows and columns and the relative spacing between cells; identification is performed in the financial report database based on the key topological features, and a highest-fit financial report grid graph is extracted based on a maximum matching algorithm; graph matching is performed between the table structure topology graph and the highest-fit financial report grid graph using a global optimization algorithm, and an affine transformation matrix that can minimize the position deviation of all corresponding nodes and the angular deviation of corresponding edges between the two graphs is iteratively solved as a posture correction parameter.

6. The table area positioning and correction method based on image edge detection according to claim 1, characterized in that: The method of generating a multi-channel restoration guide map based on the corrected mask and edge structure information is as follows: Based on the edge structure information, a binary mask reflecting the complete table frame is generated as a line structure channel; a logical union operation is performed on all cell masks in the corrected mask to generate a binary mask that identifies all non-edge areas as a content area channel; the line structure channel is compared and analyzed with the posture-corrected image to identify specific areas where the table is missing, and a binary mask marking the location to be repaired is generated as a defect marking channel; The line structure channel, content area channel and defect marking channel are combined to form a multi-channel repair guidance map.

7. The table area positioning and correction method based on image edge detection according to claim 1, characterized in that: The specific implementation process of using the financial statement database training table to repair the network includes: Extracting complete financial statements from a financial statement database; processing the complete financial statements into paired training samples, performing random incomplete processing on the complete financial statements to generate incomplete table images; extracting a line structure channel, a content area channel, and a defect mark channel based on the complete financial statements; combining the three channels to generate a sample repair guidance map; and constructing paired training samples by combining the complete financial statements, the incomplete table images, and the sample repair guidance map. The incomplete table image and the sample repair guidance map are spliced ​​in the channel dimension and input into the table repair network to generate a predicted repair image through inference; the overall error between the predicted repair image and the complete financial statement is calculated using an error function; based on the overall error, the internal weight parameters of the table repair network are iteratively updated using a back-propagation algorithm based on gradient descent until the model converges, thereby generating a trained table repair network.

Images