Bank report processing method and system, electronic equipment and storage medium

Abstract

The invention provides a bank report processing method and system, electronic equipment and a storage medium, binarization processing is performed on a report scanning copy uploaded by a user to obtain a series of point sets, and downsampling processing is performed on the series of point sets to obtain a target point set; judging whether the report sheet meets a principal component analysis condition or not according to the target point set; if the report sheet meets the principal component analysis condition, carrying out principal component analysis on the report sheet to obtain a rotation document image; positioning the coordinate of each character in the rotated document image based on a character space network identification method, and converting the coordinate of each character in the rotated document image to obtain a corresponding numerical matrix; performing form extraction by using the numerical matrix through a pre-trained form extraction model to obtain a form segmentation model diagram of the report form; wherein the pre-trained form extraction model is obtained by training a to-be-trained full convolutional neural network by using a historical numerical matrix, and the historical numerical matrix is obtained by processing a historical report.

Description

Technical Field

[0001] The present invention relates to the technical field of report form processing, and more specifically, to a method, system, electronic equipment and storage medium for processing bank report forms. Background Art

[0002] With the rapid transformation and development of major banks towards industrial informatization and digitalization, the openness of banks has also emerged in the public eye. In banks, as an internal business management system for network financial departments, the importance of the operation and control platform is self-evident. Business personnel use this management system to connect with and manage a large number of partner users, approve and review business processes in various open banking systems, help partners quickly access the bank's financial product ecosystem, accelerate the iterative innovation of financial products, and achieve a win-win situation.

[0003] Currently, business personnel will ask partner users to provide various formalities documents, upload them through the management platform and enter them into the database, and transform them into a form that can be used and processed later. However, various forms will be received in this process. Among them, the form has a high-density information volume and a regular two-dimensional space correspondence. On the one hand, these forms may be in various forms, such as full-border forms, classic three-line forms, and even descriptive forms without lines. Due to the different forms received, business personnel have various fallacies in the process of manual entry.

[0004] Based on the above problems, in the prior art, the problem of manual input by business personnel can be solved by scanning forms. However, since the documents submitted by users may be scanned copies that cannot be directly parsed, and the user's scanning operation may be interfered with by various factors, resulting in the submitted finished product being rotated, miscut and deformed, it adds difficulty to form recognition.

[0005] Therefore, providing a method for quickly and clearly identifying and extracting forms to avoid various errors made by business personnel during manual entry, and solving the problem of rotation, misalignment and deformation of submitted finished products due to interference from various factors in user scanning operations, resulting in difficulty in form recognition, are issues that urgently need to be addressed in this application. Summary of the invention

[0006] In view of this, the present invention provides a method, system, electronic device and storage medium for processing bank statements to avoid various errors made by business personnel during manual entry, and to solve the problem that the submitted finished product is rotated, sheared and deformed due to interference from various factors in the user's scanning operation, resulting in difficulty in form recognition.

[0007] The first aspect of the present application provides a method for processing a bank statement, the method comprising:

[0008] Binarize the scanned report uploaded by the user to obtain a series of point sets, and downsample the series of point sets to obtain a target point set;

[0009] Judging whether the report sheet meets the principal component analysis condition according to the target point set;

[0010] If the report meets the principal component analysis condition, perform principal component analysis on the report to obtain a rotated document image;

[0011] Locating the coordinates of each character in the rotated document image based on a character space network identification method, and converting the coordinates of each character in the rotated document image to obtain a corresponding numerical matrix;

[0012] The form is extracted using the numerical matrix through a pre-trained form extraction model to obtain a table segmentation model diagram of the report; wherein the pre-trained form extraction model is obtained by training a fully convolutional neural network to be trained using a historical numerical matrix, and the historical numerical matrix is ​​obtained by processing historical reports.

[0013] Optionally, judging whether the report sheet satisfies a principal component analysis condition according to the target point set includes:

[0014] For each target point in the target point set, construct a binary group of the target point; wherein the binary group of the target point includes two components of the target point;

[0015] Calculate the mean of each component of the target point;

[0016] Constructing a covariance matrix of the target point set according to the mean of each component of each target point;

[0017] Determine the corresponding first eigenvalue and second eigenvalue according to the covariance matrix of the target point set, and perform vector extraction on the first eigenvalue and the second eigenvalue respectively to obtain a first eigenvector and a second eigenvector;

[0018] Determine the argument of the principal component vector according to the principal component direction, and judge whether the argument of the principal component vector is greater than a threshold angle; wherein the first eigenvector is the principal component direction;

[0019] If the principal component vector argument is greater than a threshold angle, it is determined that the report meets the principal component analysis condition;

[0020] If the vector argument of the principal component direction is not greater than a threshold angle, it is determined that the report does not meet the principal component analysis condition.

[0021] Optionally, if the target point set meets the principal component analysis condition, principal component analysis is performed on the report form to obtain a rotated document image, including:

[0022] If the target point set meets the principal component analysis condition, the report form is rotated using the principal component direction to complete the principal component analysis of the report form and obtain a corresponding rotated document image.

[0023] Optionally, the pre-trained form extraction model includes a first encoding / decoding module, a second encoding / decoding module, a third encoding / decoding module and an angular attention regression layer;

[0024] The form extraction is performed using the numerical matrix through the pre-trained form extraction model to obtain the form segmentation model diagram of the report form, including:

[0025] Inputting the numerical matrix into a pre-trained form extraction model;

[0026] Perform entity segmentation extraction of the header area on the numerical matrix by the first encoding / decoding module to obtain a header entity segmentation mask;

[0027] The second encoding / decoding module performs segmentation and extraction of each row of the form body on the numerical matrix to obtain a row entity segmentation mask;

[0028] The header entity segmentation mask and the row entity segmentation mask are merged, and the corresponding header-form field entity segmentation mask is outputted through the third encoding / decoding module according to the fusion result, the header entity segmentation mask and the row entity segmentation mask;

[0029] The header entity segmentation mask and the row entity segmentation mask are processed respectively by the angular attention regression layer to obtain a form field entity prediction graph and an entity link prediction graph, and the form field entity prediction graph and the entity link prediction graph are associated to obtain a corresponding entity association graph;

[0030] The table header-form field entity segmentation mask and the entity association graph are combined to obtain a table segmentation model graph of the report form.

[0031] Optionally, the fully convolutional neural network to be trained includes a first encoding / decoding module to be trained, a second encoding / decoding module to be trained, a third encoding / decoding module to be trained, and an angular attention regression layer to be trained;

[0032] The method of using the historical numerical matrix to train the fully convolutional neural network to be trained to obtain the pre-trained form extraction model includes:

[0033] Inputting the historical numerical matrix into the fully convolutional neural network to be trained; wherein the historical numerical matrix is ​​obtained by processing the historical report form;

[0034] Perform entity segmentation extraction of the header area on the numerical matrix by using the first encoding / decoding module to be trained to obtain a predicted header entity segmentation mask;

[0035] Segment and extract each row of the form body of the numerical matrix through the second encoding / decoding module to be trained to obtain a predicted row entity segmentation mask;

[0036] The predicted header entity segmentation mask and the predicted row entity segmentation mask are fused, and a predicted header-form field entity segmentation mask is output through the third encoding / decoding module to be trained according to the fusion result, the predicted header entity segmentation mask and the predicted row entity segmentation mask;

[0037] The predicted header entity segmentation mask and the predicted row entity segmentation mask are processed respectively by the angular attention regression layer to be trained to obtain a predicted form field entity prediction graph and a predicted entity link prediction graph, and the predicted form field entity prediction graph and the predicted entity link prediction graph are associated to obtain a corresponding predicted entity association graph;

[0038] The predicted header-form field entity segmentation mask and the predicted entity association graph are combined to obtain a predicted table segmentation model graph. The predicted table segmentation model graph is close to the standard table segmentation model graph of historical reports as a training model, and the fully convolutional neural network to be trained is trained until the fully convolutional neural network to be trained reaches convergence to obtain a pre-trained form extraction model.

[0039] A second aspect of the present application provides a bank statement processing system, the system comprising:

[0040] A processing unit, used for performing binarization processing on the scanned copy of the report uploaded by the user to obtain a series of point sets, and performing downsampling processing on the series of point sets to obtain a target point set;

[0041] A judging unit, used for judging whether the report sheet satisfies the principal component analysis condition according to the target point set;

[0042] A principal component analysis unit, configured to perform principal component analysis on the report form to obtain a rotated document image if the report form meets the principal component analysis conditions;

[0043] A conversion unit, used for locating the coordinates of each character in the rotated document image based on a character space network identification method, and converting the coordinates of each character in the rotated document image to obtain a corresponding numerical matrix;

[0044] An identification and extraction unit is used to extract the form using the numerical matrix through a pre-trained form extraction model to obtain a table segmentation model diagram of the report; wherein the pre-trained form extraction model is obtained by the training unit using a historical numerical matrix to train a fully convolutional neural network to be trained, and the historical numerical matrix is ​​obtained by processing historical reports.

[0045] Optionally, the judging unit includes:

[0046] A two-tuple construction unit is used to construct a two-tuple of the target point for each target point in the target point set; wherein the two-tuple of the target point includes two components of the target point;

[0047] A mean value calculation unit, used for calculating the mean value of each component of the target point;

[0048] A covariance matrix construction unit, used to construct a covariance matrix of the target point set according to the mean of each component of each target point;

[0049] A vector extraction unit, used to determine the corresponding first eigenvalue and second eigenvalue according to the covariance matrix of the target point set, and perform vector extraction on the first eigenvalue and the second eigenvalue respectively to obtain a first eigenvector and a second eigenvector;

[0050] A judging subunit, used to determine the argument of the principal component vector according to the principal component direction, and to judge whether the argument of the principal component vector is greater than a threshold angle; wherein the eigenvector of the eigenvalue is the principal component direction;

[0051] A first determination unit, configured to determine that the report meets a principal component analysis condition if the vector argument of the principal component direction is greater than a threshold angle;

[0052] The second determination unit is used to determine that the report does not meet the principal component analysis condition if the vector argument of the principal component direction is not greater than a threshold angle.

[0053] Optionally, the principal component analysis unit includes:

[0054] The principal component analysis subunit is used to rotate the report form using the principal component direction if the target point set meets the principal component analysis conditions, so as to complete the principal component analysis of the report form and obtain the corresponding rotated document image.

[0055] The third aspect of the present application provides an electronic device, comprising: a processor and a memory, wherein the processor and the memory are connected via a communication bus; wherein the processor is used to call and execute a program stored in the memory; and the memory is used to store a program, wherein the program is used to implement the bank statement processing method provided in the first aspect of the present application.

[0056] The fourth aspect of the present application provides a computer-readable storage medium, in which a computer-executable instruction is stored, and the computer-executable instruction is used to execute the processing method of the bank report form provided in the first aspect of the present application. The embodiment of the present application provides a processing method, system, electronic device and storage medium for a bank report form, by binarizing the scanned copy of the report form uploaded by the user to obtain a series of point sets, and downsampling the series of point sets to obtain a target point set; judging whether the report form meets the principal component analysis condition according to the target point set; and if the report form meets the principal component analysis condition, performing principal component analysis on the report form to obtain a rotation document image; locating the coordinates of each character in the rotation document image based on the character space network identification method, and transforming the coordinates of each character in the rotation document image to obtain a corresponding numerical matrix; extracting the form using the numerical matrix through a pre-trained form extraction model to obtain a table segmentation model diagram of the report form; wherein the pre-trained form extraction model is obtained by training the fully convolutional neural network to be trained using a historical numerical matrix, and the historical numerical matrix is ​​obtained by processing historical report forms. The technical solution provided by the present application performs principal component analysis on the report form to correct the report form, then locates and rotates each character in the document image through the character space network identification method, and then converts it into a numerical matrix using the character space representation method, and then downsamples it to reduce the resolution. Finally, it is sent to the pre-trained form extraction model, and the form extraction model predicts the entity area of ​​the table header, table sidebar, and the entity area of ​​the form field cell based on the input numerical matrix. After that, the form extraction model will predict and score which table header entity and table sidebar entity each cell belongs to, and select the one with the highest confidence score, thereby completing the entity connection task and obtaining the final table segmentation model diagram, thereby avoiding various errors made by business personnel in the process of manual entry, and solving the problem of rotation, miscutting and deformation of the submitted finished product due to interference from various factors in the user's scanning operation, resulting in difficulty in form recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0058] Figure 1 A flowchart of a method for processing a bank statement provided in an embodiment of the present application;

[0059] Figure 2 An example diagram of a character point set representation provided by an embodiment of the present application when a document is tilted;

[0060] Figure 3 A structural example diagram of a U-Net provided in an embodiment of the present application;

[0061] Figure 4 A structural example diagram of a form extraction model provided in an embodiment of the present application;

[0062] Figure 5 A diagram showing an example structure of a corner regression network layer provided in an embodiment of the present application;

[0063] Figure 6 An example diagram of a vector indication of a link predicted by an entity link prediction module for a form field provided in an embodiment of the present application;

[0064] Figure 7 A schematic diagram of the structure of a bank statement processing system provided by an embodiment of the present invention;

[0065] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] In this application, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0068] In order to better understand this application, the technical names involved in this application are explained below:

[0069] Downsampling / Upsampling: Downsampling is a common operation in the field of deep learning. Its main function is to compress the image and lose some information to obtain a thumbnail of the image so that the image meets the input size requirements of the neural network model. For example, set a convolution mask with a step size of 2 and convolve the entire image. Usually, an image of size N×M is downsampled k times to obtain an image of size N×M / k. 2 In particular, consider an extreme case where the entire image is downsampled to a pixel unit, and its value is the mean of the entire image. In simple terms, downsampling filters out redundant information and retains key features. The opposite operation is called upsampling, which is usually filled using an interpolation algorithm.

[0070] Principal component analysis: Principal component analysis (PCA), also called principal component analysis or principal component regression analysis, essentially maps the current data coordinate space to another coordinate space through linear transformation. In the new space, the projection of the data on the coordinates of the first principal component has the largest variance, in other words, it retains the largest characteristic information, followed by the second principal component, the third principal component, and so on. By selecting the first several principal components and linearly combining them, the new data obtained is the reduced-dimensional representation of the original data, but retains the maximum characteristic information.

[0071] Character space representation: Character space or character grid representation is a character mask-level document representation that presupposes that characters in a document do not overlap and that each character can be represented by a character box. Thus, the entire document can be viewed as a number of character boxes laid out on a 2D plane. Specifically, a document can be represented as a set {(c k ,o x ,o y ,wx ,w y ):c k ∈υ,(x,y)∈N×M}, that is, the coordinates of a character element on the document page can be expressed by w x ,w y The above five-tuple represents, where υ is the vocabulary. For Chinese documents, the vocabulary is a set of discrete characters such as Chinese characters, English, numbers and underscores; the document size is N×M, which can be regarded as a local two-dimensional coordinate plane system; (o x ,o y ) is the coordinate of the upper left corner of the font box in the plane; w x ,w y The width and height of the text box.

[0072] Image semantic segmentation: Image semantic segmentation (ISS) is a pixel-level method that assigns a predefined label representing a semantic category to each pixel in the image. The difference between it and target detection is that target detection uses a positioning box to frame the target, which is a coarse-grained method, while image semantic segmentation draws the outline of the target area at the pixel level, which is a fine-grained method; and the difference between it and instance segmentation is that image semantic segmentation does not distinguish between targets of the same type, while instance segmentation distinguishes different instances under the same class, such as distinguishing different breeds of cats under the same cat class. Therefore, instance segmentation goes one step further than image semantic segmentation.

[0073] Fully Convolutional Neural Network: Fully Convolutional Network (FCN) is improved on the basis of Convolutional Neural Network (CNN). Usually, Convolutional Neural Network is connected to several fully connected layers after the convolution module, so that the feature map output by the convolution module can be mapped to a feature vector of fixed length, thereby achieving classification and regression tasks at the image level. Fully Convolutional Neural Network can complete the image segmentation task (ISS) at the semantic level. It can accept images of any size, perform an upsampling operation on the feature map output by the last convolution layer, and extend the feature map to the original input image size, so that a prediction can be generated for each pixel, that is, the category to which each pixel belongs can be restored from the feature map.

[0074] See also Figure 1 , shows a flow chart of a method for processing a bank statement provided in an embodiment of the present application, and the method for processing a bank statement specifically includes the following steps:

[0075] S101: Binarize the scanned copy of the report uploaded by the user to obtain a series of point sets, and downsample the series of point sets to obtain a target point set.

[0076] In the specific process of executing step S101, it is possible to detect in real time whether there is a report form uploaded by the user; when a report form uploaded by the user is detected, since the report form uploaded by the user is not parseable, the detected report form can be binarized to obtain a series of point sets {d1, d2, ..., n}; however, since the binarization process is performed at the pixel level, this results in a large number of redundant points in the obtained series of point sets, and therefore, the obtained series of point sets can be further downsampled to reduce the number of valid points in the series of point sets, a downsampled target point set χ = ​​{x1, x2, x3, ...}. Among them, the report form uploaded by the user can be a bank report form, which is not limited in this embodiment of the present application.

[0077] Specifically, the process of downsampling a series of point sets can be as follows: set a square window with a width of w and slide it over the entire document (a series of point sets), set a threshold α, and when the number of points in the window is greater than the threshold, generate a target point x in the middle of the window. i , thus obtaining a downsampled target point set χ={x1,x2,x3,...}. The target point set includes multiple target points.

[0078] S102: judging whether the report sheet satisfies the principal component analysis condition according to the target point set; if the report sheet satisfies the principal component analysis condition, executing step S103.

[0079] In the embodiment of the present application, since the report form uploaded by the user is in scanned form, that is, it cannot be parsed, after receiving the report form uploaded by the user, the report form can be further subjected to principal component analysis to obtain the corresponding rotated document image. The principle of principal component analysis is as follows: if the document is regarded as a two-dimensional plane, and the document content is a set of points on the plane. Since most of the current documents are written in the order of left and right, these point sets will show a strong positive correlation as a whole. When the document is tilted, the character point set will behave like this. Figure 2 In this case, if a principal component analysis is performed on it, the axis parallel to the text line direction in the document is the principal component direction, because the variance of the projection of the point set in this direction will reach the maximum. It can be seen that by rotating the document to the principal component direction, the rotation operation can be completed, that is, the principal component analysis of the document is completed, and the corresponding rotated document image is obtained.

[0080] However, in actual application, many documents do not need to be corrected. If the principal component analysis is performed directly on the report, errors may occur, but the error angle will not be too large. Therefore, a threshold angle θ can be set. r , when the principal component vector angle is not greater than θ r, the rotation operation is abandoned, that is, the principal component analysis of the report is abandoned. It can be seen that the corresponding principal component analysis condition can be set according to the threshold angle. Among them, the principal component analysis condition indicates that the principal component vector argument is greater than the threshold angle, and the report is determined to meet the principal component analysis condition, and the principal component analysis is performed on the report; if the principal component vector argument is greater than the threshold angle, it is determined that the report does not meet the principal component analysis condition, and the principal component analysis of the report is abandoned.

[0081] In the specific process of executing step S102, after obtaining the target point set corresponding to the report, the first principal component analysis direction can be further determined according to the obtained target point set, and the corresponding principal component vector argument can be determined according to the principal component direction, so as to judge whether the report meets the principal component analysis conditions according to the determined principal component vector argument; if the report meets the principal component analysis conditions, step S103 is executed. If the report does not meet the principal component analysis conditions, the principal component analysis of the report is abandoned, the report is directly used as the rotated document image, and step S103 is executed.

[0082] Optionally, the process of judging whether a report satisfies the principal component analysis condition according to the target point set may specifically be as follows: for each target point in the target point set, construct a pair of target points; wherein the pair of target points includes two components of the target point; calculate the mean on each component of the target point; construct the covariance matrix of the target point set according to the mean on each component of each target point; determine the corresponding first eigenvalue and second eigenvalue according to the covariance matrix of the target point set, and perform vector extraction on the first eigenvalue and second eigenvalue respectively to obtain the first eigenvector and the second eigenvector; determine the principal component vector angle according to the principal component direction, and judge whether the principal component vector angle is greater than a threshold angle; wherein the eigenvector of the eigenvalue is the principal component direction; if the vector angle of the principal component direction is greater than the threshold angle, it is determined that the report satisfies the principal component analysis condition; if the vector angle of the principal component direction is not greater than the threshold angle, it is determined that the report does not meet the principal component analysis condition.

[0083] In actual application, the upper left corner of the document area (report form) can be selected as the origin to establish a coordinate system. Then each target point x i You can use a binary It indicates that, The target point x i The component on the horizontal axis of the coordinate system, Target point x i The component of the ordinate in the coordinate system; calculate each target point x i The mean of each component of is as shown in formula (1); according to each component and its mean at each target point, the corresponding covariance matrix cov(x (1) ,x (2)), as shown in formula (2); obtain the first eigenvalue λ1 and the second eigenvalue λ2 of the covariance matrix = eig(C), where λ1≥λ2; perform vector extraction on the first eigenvalue and the second eigenvalue to obtain the corresponding first eigenvector υ1 and second eigenvalue υ2; take υ1 as the principal component direction, which can be considered as the correction direction of the document text line, and its angle θ with the horizontal direction is the desired rotation direction, that is, the principal component direction depression angle θ; determine whether the principal component direction angle θ is greater than the threshold angle θ r , when the principal component vector angle is not greater than θ r When the principal component vector angle is greater than θ, the rotation operation is abandoned; r , execute step S103.

[0084]

[0085] Among them, μ (1) The target point x i The component on the horizontal axis of the coordinate system The mean value, μ (2) The target point x i The vertical coordinate in the coordinate system The mean of the components on , n is the number of target points in the target point set.

[0086]

[0087] Where C is the covariance matrix.

[0088] S103: Perform principal component analysis on the report form to obtain a rotated document image.

[0089] In the embodiment of the present application, when it is determined that the report form meets the principal component analysis conditions, the report form can be rotated using the principal component direction to complete the principal component analysis of the report form and obtain the corresponding rotated document image.

[0090] Specifically, the principal component direction may be regarded as a horizontal correction line to rotate the report form to obtain a corresponding rotated document image.

[0091] It should be noted that if the report form is regarded as a two-dimensional plane, the writing order of the report form is horizontal, so if the character set in the report form is regarded as a point set, the two dimensions have a strong linear correlation, and the variance of the point set is maximum in the direction parallel to the row, which is exactly the direction of the principal component. The correction can be completed by rotating the report form in this direction, which is convenient for subsequent character positioning and recognition.

[0092] S104: Locate the coordinates of each character in the rotated document image based on the character space network identification method, and transform the coordinates of each character in the rotated document image to obtain a corresponding numerical matrix.

[0093] In the embodiment of the present application, in order to ensure that the subsequent neural network can work, the obtained rotated document image can be further processed to effectively vectorize the rotated document image and obtain a numerical matrix that meets the neural network input.

[0094] In the specific process of executing step S104, after obtaining the corresponding rotated document image, the coordinates of each character in the document image can be rotated based on the character space network identification method; specifically, a pixel-level grid mask can be used to cover the entire document (rotated document image), and each valid character in the document may occupy multiple pixels, so these pixels point to this character at the same time; further, because the characters are regarded as character frames, the pixels covered by the entire character frame will be mapped to the same character; the remaining invalid characters or blank area pixels point to a preset invalid symbol. Therefore, if the size of the document is N×M, then this grid mask is the corresponding pixel-level matrix κ∈υ N ×υ M . And, κ∈υ N ×υ M A sparse matrix, and the local part is always repeated, representing that a character occupies multiple pixels, so it can be downsampled to compress its size, where υ is the alphabet; and because the document is composed of Chinese, English, numbers and various punctuation marks, its basic unit is character rather than word, so it does not need to be represented by word embedding vector technology. In fact, the task of extracting the report form is more about spatial reasoning, because it has a strict two-dimensional matrix arrangement form. In order to meet the input of the neural network, the matrix can be further converted into a numerical vector form, that is, each character in the alphabet (each character in the rotated document image) is mapped to an integer code to obtain the corresponding numerical matrix.

[0095] In actual application, report extraction depends on the spatial distribution of each table part (this is the same as the logic of human recognition of forms. When a document in the form of a two-dimensional matrix layout is observed, and obvious row and column arrangement trends can be observed - especially for a sparse table without a wireless frame, it can be concluded at the first time that it is most likely a form without further observation of the content. The module of the fully convolutional neural network is designed to imitate this process: the neural network is required to focus on "observing" and learning the spatial distribution of characters to infer the formal characteristics of the table); for this reason, it can be assumed that the size of the document (rotated document image) is N×M, and that the characters in the document (rotated document image) are relatively discrete and independent, and do not overlap with each other. Each character can be calibrated with a character frame, then the document can be represented as D={(c k ,o x ,o y ,w x ,w y )}, where the five components represent the characters c k , the coordinates of the upper left corner of the font box o x ,o y 、Font frame size w x ,w y ; Now use N×M pixel-level mask to "cover" the document. For the pixel point with coordinates (i, j), the pixel point must be in a certain character face frame. Assuming that the pixel point is in the kth character face frame, the character corresponding to the pixel is c k ; Characters can be mapped to a numerical value, considering that the limited characters in the document can form an alphabet υ (characters can be recognized and collected by the front optical character recognition engine), represented by the index of the character in the alphabet, assuming it is τ k , that is, υ[τ k ]=c k , so that the pixel mask constitutes a numerical matrix; as mentioned above, because a character occupies pixels in an area, and there are a large number of blank invalid character areas in the document, the numerical matrix is ​​sparse and partially repeated, and then downsampling can be performed during the period, while reducing the resolution while still retaining the overall character distribution.

[0096] S105: Use a numerical matrix to extract the form through a pre-trained form extraction model to obtain a table segmentation model diagram of the report form, wherein the pre-trained form extraction model is obtained by training a fully convolutional neural network to be trained using a historical numerical matrix, and the historical numerical matrix is ​​obtained by processing historical reports.

[0097] In an embodiment of the present application, a historical report form can be obtained, and the historical report form can be binarized to obtain a series of historical point sets, and the series of historical point sets can be downsampled to obtain a historical target point set; when it is determined that the historical report form meets the principal component analysis conditions according to the historical target point set, the historical report form is subjected to principal component analysis to obtain a historical rotation document image; the coordinates of each character in the historical rotation document image are located based on the character space network identification method, and the coordinates of each character in the historical rotation document image are converted to obtain a corresponding historical numerical matrix. The historical numerical matrix is ​​input into the fully convolutional neural network to be trained, so that the fully convolutional neural network to be trained uses the input historical numerical matrix to extract the form, and obtains a predicted table segmentation model diagram, and the obtained predicted table segmentation model diagram is close to the standard table segmentation model diagram of the historical report form as a training model, and the fully convolutional neural network to be trained is trained until the fully convolutional neural network to be trained reaches convergence to obtain a pre-trained form extraction model.

[0098] It should be noted that fully convolutional neural networks are good at image semantic segmentation (ISS) tasks, especially reasoning at the pixel level, which is highly consistent with the representation of character spatial positions, and the form extraction task can be regarded as an image semantic segmentation task - treating the document as an image and the header and table elements as a semantic unit class.

[0099] In the embodiment of the present application, U-Net can be used as the basic unit of the fully convolutional neural network to be trained involved in the present application, and the fully convolutional neural network to be trained is composed of a combination of several U-Net units. Among them, the basic structure of U-Net is as follows Figure 2 As shown in the figure, the left half from top to bottom is called the "compression path". Each module uses two convolution operations and one maxpooling downsampling. With each layer, the image size is reduced to half of the original size and the number of convolution kernels is doubled. After passing through four modules, a small feature map is finally obtained. The right half from bottom to top is called the "expansion path", which is dual to the compression path. Each module performs deconvolution to upsample the feature map. The image cannot be completely restored during upsampling, so there will be information loss. In order to reduce the loss, as shown in Figure 2 Use the "copy and crop" operation to crop the downsampled feature map corresponding to the left half to the same size and then perform convolution. Figure 2 The 28×28×1024 feature map in the lower right corner is deconvolved with 512 2×2×1024 convolution kernels to get a size of 56×56×512. Then the 56×56×512 downsampled feature map on the left is concatenated to get a size of 56×56×(512+512). Figure 3It can be seen that the overall structure looks like a U-shaped structure, so it is called U-Net. By stacking and splicing U-Net as a basic unit, and dividing different modules according to different functions, the fully convolutional neural network to be trained provided in the embodiment of the present application is finally formed. Among them, the fully convolutional neural network to be trained is composed of a fully convolutional neural network to be trained, including a first encoding / decoding module to be trained, a second encoding / decoding module to be trained, a third encoding / decoding module to be trained, and an angular attention regression layer to be trained.

[0100] Optionally, the process of using the historical numerical matrix to train the fully convolutional neural network to be trained to obtain a pre-trained form extraction model can be specifically as follows: inputting the historical numerical matrix into the fully convolutional neural network to be trained; wherein the historical numerical matrix is ​​obtained by processing historical report forms; performing entity segmentation extraction of the header area on the numerical matrix through the first encoding / decoding module to be trained to obtain a predicted header entity segmentation mask; performing segmentation extraction of each row of the form body on the numerical matrix through the second encoding / decoding module to be trained to obtain a predicted row entity segmentation mask; fusing the predicted header entity segmentation mask and the predicted row entity segmentation mask, and performing the third encoding / decoding module to be trained based on the fusion result, the predicted header entity segmentation mask and the predicted Row entity segmentation mask, output predicted header-form field entity segmentation mask; process the predicted header entity segmentation mask and the predicted row entity segmentation mask respectively through the corner attention regression layer to be trained, and obtain the predicted form field entity prediction graph and the predicted entity link prediction graph, and associate the predicted form field entity prediction graph and the predicted entity link prediction graph to obtain the corresponding predicted entity association graph; combine the predicted header-form field entity segmentation mask and the predicted entity association graph to obtain the predicted table segmentation model graph, and use the predicted table segmentation model graph that is close to the standard table segmentation model graph of the historical report as the training model, and train the fully convolutional neural network to be trained until the fully convolutional neural network to be trained reaches convergence, and obtains the pre-trained form extraction model.

[0101] In the specific process of executing step S105, after obtaining the numerical matrix of the insurance report, the numerical matrix can be input into the pre-trained form extraction model, so that the form extraction model processes the input form matrix to obtain the header-form field entity segmentation mask and the entity association graph, and the header-form field entity segmentation mask and the entity association graph are combined to obtain the table segmentation model graph of the report.

[0102] Optionally, the pre-trained form extraction model includes a first encoding / decoding module, a second encoding / decoding module, a third encoding / decoding module and an angular attention regression layer; the process of extracting the form using the numerical matrix through the pre-trained form extraction model to obtain the table segmentation model diagram of the report form can be specifically as follows: inputting the numerical matrix into the pre-trained form extraction model; performing entity segmentation extraction of the header area on the numerical matrix through the first encoding / decoding module to obtain the header entity segmentation mask; performing segmentation extraction of each row of the form body on the numerical matrix through the second encoding / decoding module to obtain the row entity segmentation mask. ; The header entity segmentation mask and the row entity segmentation mask are fused, and the corresponding header-form field entity segmentation mask is output through the third encoding / decoding module according to the fusion result, the header entity segmentation mask and the row entity segmentation mask; The header entity segmentation mask and the row entity segmentation mask are processed separately through the angular attention regression layer to obtain the form field entity prediction map and the entity link prediction map, and the form field entity prediction map and the entity link prediction map are associated to obtain the corresponding entity association map; The header-form field entity segmentation mask and the entity association map are combined to obtain the table segmentation model map of the report form.

[0103] In actual application, the encoding / decoding unit composed of U-Net with full convolutional neural network as the underlying architecture will extract the features in the report form. This is a task process of image semantic segmentation. Since the numerical matrix G is a pixel-level representation method, it can be directly used as the input of the form extraction model. Figure 4 , the numerical matrix corresponding to the report form can be input into the form extraction model. The numerical matrix first passes through two U-Net encoding / decoding modules (the first encoding / decoding module and the second encoding / decoding module) in the form extraction model, respectively performing entity segmentation extraction on the header area in the numerical matrix and segmentation extraction on each row of the form body, and outputting the masks of their respective parts, representing the header entity segmentation mask H and the row entity segmentation mask L; the header entity segmentation mask H and the row entity segmentation mask L will enter the third U-Net encoding / decoding module (the third encoding / decoding module) after fusion. Note that the third module also accepts the output information of the first two modules (the header entity segmentation mask H and the row entity segmentation mask L), so as to keep information from being lost in long-distance predictions and prevent the problem of gradient disappearance during training. After fusion, the third module will output the header-form field entity segmentation mask representation M=UNet(H,L). At this time, it is also necessary Figure 3 The right half predicts the link relationship of the entity, which generates a vector field diagram F (entity association diagram) of the entity association relationship. For any cell in the form, the vector field of its area should point to the corresponding header cell in the overall trend. Therefore, the two parts of information (header-form field entity segmentation mask and entity association diagram) are merged to obtain the final form output. That is, the header-form field entity segmentation mask.

[0104] Now explain Figure 3 The workflow on the right. First, the outputs of the first two modules in the left process (header entity segmentation mask H and row entity segmentation mask L) will be sent to the corner attention regression network layer (the structure of the corner regression network layer is as follows) Figure 5 As shown in the figure, the neural network in this layer will look for the features of the table edge corners, that is, the regression features of the four corner points of the table area, and predict the offset between the input text box and the real annotation, which is conducive to correctly locating the table area; the located table area is sent to the form field entity detection and entity connection detection modules (form field entity detection module and entity link detection module) for learning, among which the form field entity detection module is used to predict the table cell domain. For each predicted form field pixel coordinate (i, j), the module predicts the vector v pointing to the true form field coordinate. i,j =(x i,j ,y i,j ), corresponding to the confidence score c i,j and a learning parameter τ i,j , standard deviation parameter σ for the Gaussian convolution kernel i,j , using the confidence score as the weight coefficient, using the Gaussian convolution kernel Γ to smooth the entire output image (the smoothing parameter is the parameter learned by the form field entity detection module), and obtaining the form field entity prediction image Ψ (as shown in formula (3)); for each predicted form field pixel coordinate (i, j), the entity link prediction module predicts the two vectors pointing to the corresponding table header and table sidebar units. And a corresponding confidence score δ i,j ,like Figure 6 As shown, the corresponding entity link prediction graph is obtained; finally, the form field entity prediction graph and the entity link prediction graph are fused to obtain the corresponding entity association graph.

[0105] The embodiment of the present application provides a method for processing a bank report form, which comprises binarizing a scanned copy of the report form uploaded by a user to obtain a series of point sets, and performing downsampling processing on the series of point sets to obtain a target point set; judging whether the report form satisfies a principal component analysis condition according to the target point set; and performing principal component analysis on the report form when the report form satisfies the principal component analysis condition to obtain a rotated document image; locating the coordinates of each character in the rotated document image based on a character space network identification method, and transforming the coordinates of each character in the rotated document image to obtain a corresponding numerical matrix; extracting a form using a numerical matrix through a pre-trained form extraction model to obtain a table segmentation model diagram of the report form; wherein the pre-trained form extraction model is obtained by training a fully convolutional neural network to be trained using a historical numerical matrix, and the historical numerical matrix is ​​obtained by processing historical reports. The technical solution provided by the present application performs principal component analysis on the report form to correct the report form, then locates and rotates each character in the document image through the character space network identification method, and then converts it into a numerical matrix using the character space representation method, and then downsamples it to reduce the resolution. Finally, it is sent to the pre-trained form extraction model, and the form extraction model predicts the entity area of ​​the table header, table sidebar, and the entity area of ​​the form field cell based on the input numerical matrix. After that, the form extraction model will predict and score which table header entity and table sidebar entity each cell belongs to, and select the one with the highest confidence score, thereby completing the entity connection task and obtaining the final table segmentation model diagram, thereby avoiding various errors made by business personnel in the process of manual entry, and solving the problem of rotation, miscutting and deformation of the submitted finished product due to interference from various factors in the user's scanning operation, resulting in difficulty in form recognition.

[0106] Based on the bank statement processing method provided in the above embodiment of the present application, the present application also provides a bank statement processing system accordingly. Figure 7 As shown, the bank statement processing system includes:

[0107] The processing unit 71 is used to perform binarization processing on the scanned copy of the report uploaded by the user to obtain a series of point sets, and perform downsampling processing on the series of point sets to obtain a target point set;

[0108] A judging unit 72, used to judge whether the report sheet meets the principal component analysis condition according to the target point set;

[0109] A principal component analysis unit 73, for performing principal component analysis on the report form to obtain a rotated document image if the report form meets the principal component analysis conditions;

[0110] A conversion unit 74, used for locating the coordinates of each character in the rotated document image based on the character space network identification method, and converting the coordinates of each character in the rotated document image to obtain a corresponding numerical matrix;

[0111] The identification and extraction unit 75 is used to extract the form using the numerical matrix through the pre-trained form extraction model to obtain the table segmentation model diagram of the report form; wherein the pre-trained form extraction model is obtained by the training unit using the historical numerical matrix to train the fully convolutional neural network to be trained, and the historical numerical matrix is ​​obtained by processing the historical report forms.

[0112] The specific principles and execution processes of each unit in the bank statement processing system disclosed in the embodiment of the present application are the same as those of the bank statement processing method disclosed in the above-mentioned embodiment of the present application. Please refer to the corresponding parts of the bank statement processing method disclosed in the above-mentioned embodiment of the present application, and will not be repeated here.

[0113] The embodiment of the present application provides a bank report processing system, which performs binarization processing on a scanned copy of the report uploaded by a user to obtain a series of point sets, and performs downsampling processing on the series of point sets to obtain a target point set; determines whether the report satisfies a principal component analysis condition according to the target point set; and performs principal component analysis on the report to obtain a rotated document image when the report satisfies the principal component analysis condition; locates the coordinates of each character in the rotated document image based on a character space network identification method, and transforms the coordinates of each character in the rotated document image to obtain a corresponding numerical matrix; extracts a form using a numerical matrix through a pre-trained form extraction model to obtain a table segmentation model diagram of the report; wherein the pre-trained form extraction model is obtained by training a fully convolutional neural network to be trained using a historical numerical matrix, and the historical numerical matrix is ​​obtained by processing historical reports. The technical solution provided by the present application performs principal component analysis on the report form to correct the report form, then locates and rotates each character in the document image through the character space network identification method, and then converts it into a numerical matrix using the character space representation method, and then downsamples it to reduce the resolution. Finally, it is sent to the pre-trained form extraction model, and the form extraction model predicts the entity area of ​​the table header, table sidebar, and the entity area of ​​the form field cell based on the input numerical matrix. After that, the form extraction model will predict and score which table header entity and table sidebar entity each cell belongs to, and select the one with the highest confidence score, thereby completing the entity connection task and obtaining the final table segmentation model diagram, thereby avoiding various errors made by business personnel in the process of manual entry, and solving the problem of rotation, miscutting and deformation of the submitted finished product due to interference from various factors in the user's scanning operation, resulting in difficulty in form recognition.

[0114] Optionally, the judging unit comprises:

[0115] A two-tuple construction unit is used to construct a two-tuple of the target point for each target point in the target point set; wherein the two-tuple of the target point includes two components of the target point;

[0116] A mean calculation unit, used to calculate the mean of each component of the target point;

[0117] A covariance matrix construction unit, used to construct a covariance matrix of a target point set according to the mean of each component of each target point;

[0118] A vector extraction unit, used to determine the corresponding first eigenvalue and second eigenvalue according to the covariance matrix of the target point set, and perform vector extraction on the first eigenvalue and the second eigenvalue respectively to obtain a first eigenvector and a second eigenvector;

[0119] A judging subunit is used to determine the argument of the principal component vector according to the principal component direction, and to judge whether the argument of the principal component vector is greater than a threshold angle; wherein the eigenvector of the eigenvalue is the principal component direction;

[0120] A first determination unit is used to determine that the report sheet meets the principal component analysis condition if the vector argument of the principal component direction is greater than a threshold angle;

[0121] The second determination unit is used to determine that the report does not meet the principal component analysis condition if the vector argument of the principal component direction is not greater than a threshold angle.

[0122] Optional, principal component analysis unit, including:

[0123] The principal component analysis subunit is used to rotate the report form using the principal component direction if the target point set meets the principal component analysis conditions, so as to complete the principal component analysis of the report form and obtain the corresponding rotated document image.

[0124] Optionally, the pre-trained form extraction model includes a first encoding / decoding module, a second encoding / decoding module, a third encoding / decoding module and an angular attention regression layer; the recognition extraction unit includes:

[0125] A first input unit, used to input a numerical matrix into a pre-trained form extraction model;

[0126] A first segmentation and extraction unit, configured to perform entity segmentation and extraction of a header region on the numerical matrix through a first encoding / decoding module to obtain a header entity segmentation mask;

[0127] A second segmentation and extraction unit is used to segment and extract each row of the form body of the numerical matrix through a second encoding / decoding module to obtain a row entity segmentation mask;

[0128] A first fusion processing unit is used to fuse the header entity segmentation mask and the row entity segmentation mask, and output a corresponding header-form field entity segmentation mask according to the fusion result, the header entity segmentation mask and the row entity segmentation mask through a third encoding / decoding module;

[0129] An association unit is used to process the header entity segmentation mask and the row entity segmentation mask respectively through an angular attention regression layer to obtain a form field entity prediction map and an entity link prediction map, and associate the form field entity prediction map with the entity link prediction map to obtain a corresponding entity association map;

[0130] The combining unit is used to combine the header-form field entity segmentation mask and the entity association diagram to obtain a table segmentation model diagram of the report form.

[0131] Optionally, the fully convolutional neural network to be trained includes a first encoding / decoding module to be trained, a second encoding / decoding module to be trained, a third encoding / decoding module to be trained, and an angular attention regression layer to be trained; the training unit includes:

[0132] The second input unit is used to input the historical numerical matrix into the fully convolutional neural network to be trained; wherein the historical numerical matrix is ​​obtained by processing the historical report form;

[0133] A third segmentation and extraction unit is used to perform entity segmentation and extraction of the header area on the numerical matrix through the first encoding / decoding module to be trained to obtain a predicted header entity segmentation mask;

[0134] A fourth segmentation and extraction unit is used to segment and extract each row of the form body of the numerical matrix through the second encoding / decoding module to be trained to obtain a predicted row entity segmentation mask;

[0135] A second fusion processing unit is used to fuse the predicted header entity segmentation mask and the predicted row entity segmentation mask, and output a predicted header-form field entity segmentation mask according to the fusion result, the predicted header entity segmentation mask and the predicted row entity segmentation mask through a third encoding / decoding module to be trained;

[0136] A second association unit is used to process the predicted header entity segmentation mask and the predicted row entity segmentation mask respectively through the angular attention regression layer to be trained to obtain a predicted form field entity prediction graph and a predicted entity link prediction graph, and associate the predicted form field entity prediction graph with the predicted entity link prediction graph to obtain a corresponding predicted entity association graph;

[0137] The second combining unit is used to combine the predicted header-form field entity segmentation mask and the predicted entity association graph to obtain a predicted table segmentation model graph, and use the predicted table segmentation model graph that is close to the standard table segmentation model graph of the historical report as a training model to train the fully convolutional neural network to be trained until the fully convolutional neural network to be trained reaches convergence to obtain a pre-trained form extraction model.

[0138] The present application embodiment provides an electronic device, such as Figure 8 As shown, the electronic device includes a processor 801 and a memory 802, the memory 802 is used to store program codes and data for processing bank statements, and the processor 801 is used to call program instructions in the memory to execute the steps shown in the bank statement processing method in the above embodiment.

[0139] An embodiment of the present application provides a storage medium, which includes a storage program, wherein when the program is running, the device where the storage medium is located is controlled to execute the bank statement processing method shown in the above embodiment.

[0140] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, in which the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0141] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0142] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0143] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for processing a bank statement, characterized in that: The method comprises: Binarize the scanned copy of the report uploaded by the user to obtain a series of point sets, and downsample the series of point sets to obtain a target point set; Judging whether the report sheet meets the principal component analysis condition according to the target point set; If the report meets the principal component analysis condition, perform principal component analysis on the report to obtain a rotated document image; Locating the coordinates of each character in the rotated document image based on a character space network identification method, and converting the coordinates of each character in the rotated document image to obtain a corresponding numerical matrix; The form is extracted using the numerical matrix through a pre-trained form extraction model to obtain a table segmentation model diagram of the report; wherein the pre-trained form extraction model is obtained by training a fully convolutional neural network to be trained using a historical numerical matrix, and the historical numerical matrix is ​​obtained by processing historical reports.

2. The method according to claim 1, characterized in that The step of judging whether the report sheet satisfies the principal component analysis condition according to the target point set includes: For each target point in the target point set, construct a binary group of the target point; wherein the binary group of the target point includes two components of the target point; Calculate the mean of each component of the target point; Constructing a covariance matrix of the target point set according to the mean of each component of each target point; Determine the corresponding first eigenvalue and second eigenvalue according to the covariance matrix of the target point set, and perform vector extraction on the first eigenvalue and the second eigenvalue respectively to obtain a first eigenvector and a second eigenvector; Determine the argument of the principal component vector according to the principal component direction, and judge whether the argument of the principal component vector is greater than a threshold angle; wherein the first eigenvector is the principal component direction; If the principal component vector argument is greater than a threshold angle, it is determined that the report meets the principal component analysis condition; If the vector argument of the principal component direction is not greater than a threshold angle, it is determined that the report does not meet the principal component analysis condition.

3. The method according to claim 2, characterized in that If the target point set meets the principal component analysis condition, the report form is subjected to principal component analysis to obtain a rotated document image, including: If the target point set meets the principal component analysis condition, the report form is rotated using the principal component direction to complete the principal component analysis of the report form and obtain a corresponding rotated document image.

4. The method according to claim 1, characterized in that: The pre-trained form extraction model includes a first encoding / decoding module, a second encoding / decoding module, a third encoding / decoding module and an angular attention regression layer; The form extraction is performed using the numerical matrix through the pre-trained form extraction model to obtain the form segmentation model diagram of the report form, including: Inputting the numerical matrix into a pre-trained form extraction model; Perform entity segmentation extraction of the header area on the numerical matrix by the first encoding / decoding module to obtain a header entity segmentation mask; The second encoding / decoding module performs segmentation and extraction of each row of the form body on the numerical matrix to obtain a row entity segmentation mask; The header entity segmentation mask and the row entity segmentation mask are merged, and the corresponding header-form field entity segmentation mask is outputted through the third encoding / decoding module according to the fusion result, the header entity segmentation mask and the row entity segmentation mask; The header entity segmentation mask and the row entity segmentation mask are processed respectively by the angular attention regression layer to obtain a form field entity prediction graph and an entity link prediction graph, and the form field entity prediction graph and the entity link prediction graph are associated to obtain a corresponding entity association graph; The table header-form field entity segmentation mask and the entity association graph are combined to obtain a table segmentation model graph of the report form.

5. The method according to claim 1, characterized in that The fully convolutional neural network to be trained includes a first encoding / decoding module to be trained, a second encoding / decoding module to be trained, a third encoding / decoding module to be trained, and an angular attention regression layer to be trained; The method of using the historical numerical matrix to train the fully convolutional neural network to be trained to obtain the pre-trained form extraction model includes: Inputting the historical numerical matrix into the fully convolutional neural network to be trained; wherein the historical numerical matrix is ​​obtained by processing the historical report form; Perform entity segmentation extraction of the header area on the numerical matrix by using the first encoding / decoding module to be trained to obtain a predicted header entity segmentation mask; Segment and extract each row of the form body of the numerical matrix through the second encoding / decoding module to be trained to obtain a predicted row entity segmentation mask; The predicted header entity segmentation mask and the predicted row entity segmentation mask are fused, and a predicted header-form field entity segmentation mask is output through the third encoding / decoding module to be trained according to the fusion result, the predicted header entity segmentation mask and the predicted row entity segmentation mask; The predicted header entity segmentation mask and the predicted row entity segmentation mask are processed respectively by the angular attention regression layer to be trained to obtain a predicted form field entity prediction graph and a predicted entity link prediction graph, and the predicted form field entity prediction graph and the predicted entity link prediction graph are associated to obtain a corresponding predicted entity association graph; The predicted header-form field entity segmentation mask and the predicted entity association graph are combined to obtain a predicted table segmentation model graph. The predicted table segmentation model graph is close to the standard table segmentation model graph of historical reports as a training model, and the fully convolutional neural network to be trained is trained until the fully convolutional neural network to be trained reaches convergence to obtain a pre-trained form extraction model.

6. A bank statement processing system, characterized in that: The system comprises: A processing unit, used for performing binarization processing on the scanned copy of the report uploaded by the user to obtain a series of point sets, and performing downsampling processing on the series of point sets to obtain a target point set; A judging unit, used for judging whether the report sheet satisfies the principal component analysis condition according to the target point set; A principal component analysis unit, configured to perform principal component analysis on the report form to obtain a rotated document image if the report form meets the principal component analysis conditions; A conversion unit, used for locating the coordinates of each character in the rotated document image based on a character space network identification method, and converting the coordinates of each character in the rotated document image to obtain a corresponding numerical matrix; An identification and extraction unit is used to extract the form using the numerical matrix through a pre-trained form extraction model to obtain a table segmentation model diagram of the report; wherein the pre-trained form extraction model is obtained by the training unit using a historical numerical matrix to train a fully convolutional neural network to be trained, and the historical numerical matrix is ​​obtained by processing historical reports.

7. The system according to claim 6, characterized in that The judging unit comprises: A two-tuple construction unit is used to construct a two-tuple of the target point for each target point in the target point set; wherein the two-tuple of the target point includes two components of the target point; A mean value calculation unit, used for calculating the mean value of each component of the target point; A covariance matrix construction unit, used to construct a covariance matrix of the target point set according to the mean of each component of each target point; A vector extraction unit, used to determine the corresponding first eigenvalue and second eigenvalue according to the covariance matrix of the target point set, and perform vector extraction on the first eigenvalue and the second eigenvalue respectively to obtain a first eigenvector and a second eigenvector; A judging subunit, used to determine the argument of the principal component vector according to the principal component direction, and to judge whether the argument of the principal component vector is greater than a threshold angle; wherein the eigenvector of the eigenvalue is the principal component direction; A first determination unit, configured to determine that the report meets a principal component analysis condition if the vector argument of the principal component direction is greater than a threshold angle; The second determination unit is used to determine that the report does not meet the principal component analysis condition if the vector argument of the principal component direction is not greater than a threshold angle.

8. The system according to claim 7, characterized in that The principal component analysis unit comprises: The principal component analysis subunit is used to rotate the report form using the principal component direction if the target point set meets the principal component analysis conditions, so as to complete the principal component analysis of the report form and obtain the corresponding rotated document image.

9. An electronic device, characterized in that: include: A processor and a memory, wherein the processor and the memory are connected via a communication bus; wherein the processor is used to call and execute a program stored in the memory; The memory is used to store a program, and the program is used to implement the bank statement processing method according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to execute the bank statement processing method according to any one of claims 1 to 5.

Images