Transaction confirmation check processing method and device, equipment and storage medium

Abstract

The invention provides a transaction confirmation book auditing processing method and device, equipment and a storage medium, and the method comprises the steps: matching an element extraction rule and a transaction confirmation book template according to a transaction type in a transaction confirmation book, and carrying out the OCR recognition of the rule and the transaction confirmation book template, and obtaining a first confirmation book text and a first template text; performing layout analysis on the first confirmation book text and the first template text to obtain a second confirmation book text and a second template text, performing element extraction on the second confirmation book text and the second template text according to an element extraction rule to obtain a first extraction result and a second extraction result, and deleting the second extraction result; integrating the remaining parts of the second confirmation text and the second template text after the elements are extracted to obtain a first integrated text and a second integrated text; and comparing the first extraction result with corresponding data in the target system, and comparing the first integrated text with the second integrated text to generate a comparison result with difference points. According to the embodiment of the invention, the efficiency and self-adaptability of transaction confirmation checking processing can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of transaction confirmation letter processing, and in particular to a transaction confirmation letter review and processing method, device, equipment and storage medium. Background Art

[0002] Trade confirmation is an important part of the transaction process, which occurs in the pre-clearing stage of post-transaction processing. The trade confirmation is the written evidence of the trade confirmation. In fact, the trade confirmation is not only a legal contract between the two parties to the transaction, but also contains many key information of the transaction. During the review and processing of the trade confirmation, the operator needs to compare whether the terms of this confirmation are consistent with the standard template of the same transaction type with the same two parties that has passed the legal review in advance, and mark the differences (text addition, reduction, change) for the reviewer to judge whether the trade confirmation is valid. At the same time, it is necessary to compare the transaction elements with the data in the Summit system (fund transaction management system). When comparing, some fields need to be converted to the specified format, such as date format, transaction direction, etc.

[0003] At present, the review of transaction confirmation letters mainly includes two methods: manual review and semi-automatic review (manual + automatic rule matching). Since both types of transaction confirmation letters require human participation, the review efficiency of transaction confirmation letters is relatively low and it is difficult to adapt to the format changes of transaction confirmation letters. Summary of the invention

[0004] The purpose of the embodiments of this specification is to provide a transaction confirmation review processing method, device, equipment and storage medium to improve the efficiency and adaptability of transaction confirmation review processing.

[0005] To achieve the above objectives, on the one hand, the embodiments of this specification provide a transaction confirmation review and processing method, including:

[0006] Determine the transaction type in the target transaction confirmation;

[0007] Matching the element extraction rules and transaction confirmation template corresponding to the transaction type;

[0008] Performing OCR recognition on the target transaction confirmation and the transaction confirmation template to obtain a first confirmation text and a first template text respectively;

[0009] Performing layout analysis on the first confirmation text and the first template text to obtain a second confirmation text and a second template text accordingly;

[0010] Extracting elements from the second confirmation letter text and the second template text according to the element extraction rule, obtaining a first extraction result and a second extraction result accordingly, and deleting the second extraction result;

[0011] Integrate the remaining part of the second confirmation text after extracting the elements to obtain a first integrated text; and integrate the remaining part of the second template text after extracting the elements to obtain a second integrated text;

[0012] The first extraction result is compared with the corresponding data in the target system, and the first integrated text is compared with the second integrated text to generate a comparison result marked with difference points.

[0013] In the transaction confirmation review processing method of the embodiment of this specification, the layout analysis of the first confirmation text and the first template text includes:

[0014] Assemble the first confirmation text and the first template text row by row according to the coordinates of the four corners of the text, and obtain a first assembled text and a second assembled text respectively;

[0015] For each assembled text, the first detection frame of each line of text is compared with the second detection frame of the layout analysis to confirm the category of each line of text and the belonging of the document element area;

[0016] For each assembled text, the text format of each page is organized and non-text information is removed according to the category of each line of text and the affiliation of the document element area, and a second confirmation text and a second template text are obtained accordingly.

[0017] In the transaction confirmation review processing method of the embodiment of this specification, the first detection frame of each line of text is compared with the second detection frame of the layout analysis, including:

[0018] Determine whether the four corner coordinates of the first detection frame of the current line of text are within the range of the second detection frame of the layout analysis; if so, confirm that the document element category of the current line is the same as the second detection frame of the layout analysis.

[0019] In the transaction confirmation review processing method of the embodiment of this specification, the first detection frame of each line of text is compared with the second detection frame of the layout analysis, including:

[0020] Determine whether the IOU of the first detection box of the current line of text and the second detection box of the layout analysis is greater than a first threshold; if so, confirm that the document element category of the current line is the same as the second detection box of the layout analysis.

[0021] In the transaction confirmation review processing method of the embodiment of this specification, the first detection frame of each line of text is compared with the second detection frame of the layout analysis, including:

[0022] Determine whether the quotient of the intersection area of ​​the first detection box of the current line of text and the second detection box of the layout analysis divided by the area of ​​the first detection box of the current line of text is greater than a second threshold; if so, confirm that the document element category of the current line is the same as the second detection box of the layout analysis.

[0023] In the transaction confirmation review processing method of the embodiment of this specification, the element extraction includes table element extraction, and the processing logic of the table element extraction includes:

[0024] Determine whether the currently processed table is the last table on this page; if so, and the first table on the next page has the same number of columns as the current table, merge the last table on this page with the first table on the next page;

[0025] Determine the corresponding extraction rules according to the table title or table header keywords of the merged table;

[0026] According to the field keywords or regular expressions in the extraction rules, the cell texts of the merged table are searched one by one to match the target cells corresponding to each extraction rule, and the extraction rules that do not match the target cells are recorded;

[0027] Query the value corresponding to the target cell according to the set search direction;

[0028] When there is a reference relationship in the value value, the value content of the value value is recursively searched and the final value value is assigned to all key values ​​on the reference chain.

[0029] In the transaction confirmation review processing method of the embodiment of this specification, the element extraction includes text element extraction, and the extraction logic of the text element extraction includes:

[0030] Identify feature areas;

[0031] Determine an extraction rule corresponding to the element area;

[0032] Extract the value in the element area according to the extraction rule;

[0033] When there is a reference relationship in the value value, the value content of the value value is recursively searched and the final value value is assigned to all key values ​​on the reference chain.

[0034] On the other hand, the embodiment of this specification also provides a transaction confirmation review processing device, including:

[0035] A determination module, used to determine the transaction type in the target transaction confirmation;

[0036] A matching module, used for matching the element extraction rules and the transaction confirmation template corresponding to the transaction type;

[0037] An identification module, used for performing OCR identification on the target transaction confirmation and the transaction confirmation template, and correspondingly obtaining a first confirmation text and a first template text;

[0038] An analysis module, used for performing layout analysis on the first confirmation text and the first template text, and correspondingly obtaining a second confirmation text and a second template text;

[0039] An extraction module, used to extract elements from the second confirmation text and the second template text according to the element extraction rule, obtain a first extraction result and a second extraction result accordingly, and delete the second extraction result;

[0040] An integration module, used to integrate the remaining part of the second confirmation text after the elements are extracted to obtain a first integrated text; and integrate the remaining part of the second template text after the elements are extracted to obtain a second integrated text;

[0041] A comparison module is used to compare the first extraction result with the corresponding data in the target system, and to compare the first integrated text with the second integrated text to generate a comparison result marked with differences.

[0042] On the other hand, an embodiment of the present specification further provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein when the computer program is executed by the processor, the instructions of the above method are executed.

[0043] On the other hand, an embodiment of the present specification further provides a computer storage medium on which a computer program is stored. When the computer program is executed by a processor of a computer device, the instructions of the above method are executed.

[0044] On the other hand, an embodiment of the present specification further provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor of a computer device, the computer program executes instructions of the above method.

[0045] It can be seen from the technical solutions provided by the above embodiments of this specification that in the embodiments of this specification, the corresponding element extraction rules and transaction confirmation templates can be directly matched according to the transaction type in the transaction confirmation, and the transaction confirmation review and processing can be carried out accordingly, without considering different transaction confirmation formats and their changes. In this way, the problem of low efficiency in the review and processing of the transaction confirmation due to the large number of transaction confirmation formats and the possibility of changes at any time can be avoided; further, by performing layout analysis on the OCR recognition results of the transaction confirmation and the transaction confirmation template, the content of the transaction confirmation can be accurately identified when a new transaction confirmation style appears or the style changes, thereby improving the adaptability of the review and processing of the transaction confirmation. Not only that, the embodiments of this specification also achieve high-precision comparison of the transaction confirmation by comparing the extraction results of the transaction confirmation with the corresponding data in the target system, and comparing the remaining parts of the transaction confirmation and the transaction confirmation template after extracting the elements, thereby improving the accuracy of the review and processing of the transaction confirmation. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art description. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0047] Figure 1 A schematic diagram of a transaction confirmation review and processing system in some embodiments of this specification is shown;

[0048] Figure 2 A flowchart showing a transaction confirmation review and processing method in some embodiments of this specification;

[0049] Figure 3 Shows Figure 2 A flow chart of performing layout analysis on the first confirmation text and the first template text in the method shown;

[0050] Figure 4 Shows Figure 2 The extraction logic flow chart of text element extraction in the method shown;

[0051] Figure 5 Shows Figure 2 The extraction logic flow chart of the table element extraction in the method shown;

[0052] Figure 6 A schematic diagram showing analysis results of layout analysis and text lines in an exemplary embodiment of this specification;

[0053] Figure 7 It shows a structural block diagram of a transaction confirmation review processing device in some embodiments of this specification;

[0054] Figure 8 The structure block diagram of the computer device in some embodiments of the present specification is shown.

[0055] [Description of Reference Numerals]

[0056] 10. Client;

[0057] 20. Server;

[0058] 71. Determine the module;

[0059] 72. Matching module;

[0060] 73. Identification module;

[0061] 74. Analysis module;

[0062] 75. Extraction module;

[0063] 76. Integration module;

[0064] 77. Comparison module;

[0065] 802. Computer equipment;

[0066] 804, processor;

[0067] 806. Memory;

[0068] 808, driving mechanism;

[0069] 810, input / output interface;

[0070] 812. Input device;

[0071] 814. Output device;

[0072] 816. Presentation equipment;

[0073] 818. Graphical user interface;

[0074] 820, network interface;

[0075] 822, communication link;

[0076] 824. Communication bus. DETAILED DESCRIPTION

[0077] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0078] It should be noted that in the embodiments of this specification, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user and fully authorized by all parties, that is, the acquisition, transmission, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0079] At present, in addition to the purely manual review method, the review of transaction confirmations also includes judging the type of transaction confirmation based on keywords, and then sending it to the semi-automatic (manual + automatic rule matching) processing flow corresponding to the transaction type to conduct clause review or extract elements from the deep learning model corresponding to the transaction type. However, due to the large number of formats of transaction confirmations, hundreds of them, it takes a lot of manpower to design processes, enter terms, and train models one by one, and the implementation cycle is long, making it difficult to meet timeliness requirements. At the same time, the content and format of the transaction confirmation may change at any time. In the scheme for classifying and processing according to different transaction confirmation categories, it is necessary to modify or retrain the process configuration or deep learning model accordingly, especially because model retraining is time-consuming and requires high investment, and it is impossible to meet the business needs of rapid modification and launch, and the adaptability is poor. In view of this, in order to improve the efficiency and adaptability of the review and processing of transaction confirmations, this manual provides an improved transaction confirmation review and processing scheme.

[0080] Figure 1, a schematic diagram of a transaction confirmation review and processing system in some embodiments of the present specification is shown; the transaction confirmation review and processing system includes a client 10 and a server 20. The client 10 can initiate a transaction confirmation review and processing request; in response to the transaction confirmation review and processing request, the server 20 can determine the transaction type in the target transaction confirmation; match the element extraction rule and the transaction confirmation template corresponding to the transaction type; perform OCR recognition on the target transaction confirmation and the transaction confirmation template, and obtain a first confirmation text and a first template text correspondingly; perform layout analysis on the first confirmation text and the first template text, and obtain a second confirmation text and a second template text correspondingly; perform element extraction on the second confirmation text and the second template text according to the element extraction rule, and obtain a first extraction result and a second extraction result correspondingly, and delete the second extraction result; integrate the remaining part of the second confirmation text after the element is extracted to obtain a first integrated text; and integrate the remaining part of the second template text after the element is extracted to obtain a second integrated text; compare the first extraction result with the corresponding data in the target system, and compare the first integrated text with the second integrated text to generate a comparison result marked with difference points.

[0081] In some embodiments of the present specification, the client 10 may be a self-service terminal device, a mobile terminal (i.e., a smart phone), a display, a desktop computer, a tablet computer, a laptop computer, a digital assistant, or a smart wearable device, etc. Among them, the smart wearable device may include a smart bracelet, a smart watch, smart glasses, or a smart helmet, etc. Of course, the client 10 is not limited to the above-mentioned electronic devices with a certain entity, and it may also be software running in the above-mentioned electronic devices.

[0082] In some embodiments of the present specification, the server 20 may be an electronic device with computing and network interaction functions; or may be software running in the electronic device and providing business logic for data processing and network interaction.

[0083] In addition, it should be noted that Figure 1 What is shown is only an application environment provided by this specification. In actual application, there may be multiple clients 10 and multiple servers 20, and this specification does not impose any limitation.

[0084] This specification embodiment provides a transaction confirmation review processing method, which can be applied to the above-mentioned service side, refer to Figure 2 As shown, in some embodiments of this specification, the transaction confirmation review processing method may include the following steps:

[0085] Step 201: Determine the transaction type in the target transaction confirmation.

[0086] Step 202: Match the element extraction rules and transaction confirmation template corresponding to the transaction type.

[0087] Step 203: Perform OCR recognition on the target transaction confirmation and the transaction confirmation template to obtain a first confirmation text and a first template text accordingly.

[0088] Step 204: Perform layout analysis on the first confirmation text and the first template text to obtain a second confirmation text and a second template text accordingly.

[0089] Step 205: extract elements from the second confirmation text and the second template text according to the element extraction rule, obtain a first extraction result and a second extraction result accordingly, and delete the second extraction result.

[0090] Step 206: Integrate the remaining part of the second confirmation text after the elements are extracted to obtain a first integrated text; and integrate the remaining part of the second template text after the elements are extracted to obtain a second integrated text.

[0091] Step 207: compare the first extraction result with the corresponding data in the target system, and compare the first integrated text with the second integrated text to generate a comparison result marked with differences.

[0092] In the embodiments of this specification, the corresponding element extraction rules and transaction confirmation template can be directly matched according to the transaction type in the transaction confirmation, and the transaction confirmation review and processing can be performed accordingly, without considering different transaction confirmation formats and their changes. In this way, the problem of low efficiency in transaction confirmation review and processing caused by the large number of transaction confirmation formats and the possibility of changes at any time can be avoided; further, by performing layout analysis on the OCR recognition results of the transaction confirmation and the transaction confirmation template, the content of the transaction confirmation can be accurately identified when a new transaction confirmation style appears or the style changes, thereby improving the adaptability of the transaction confirmation review and processing. Not only that, the embodiments of this specification also achieve high-precision comparison of the transaction confirmation by comparing the extraction results of the transaction confirmation with the corresponding data in the target system (such as the Summit system, etc.), and comparing the transaction confirmation with the remaining part of the transaction confirmation template after extracting the elements, thereby improving the accuracy of the transaction confirmation review and processing.

[0093] The target transaction confirmation is a transaction confirmation to be processed (pending review). In some embodiments of this specification, the target transaction confirmation can be obtained from a storage path such as a dedicated mailbox based on robotic process automation (RPA). By parsing the target transaction confirmation, the content of the target transaction confirmation can be obtained, so that the transaction type in the target transaction confirmation can be determined based on the name characteristics or specific field characteristics of the target transaction confirmation. Among them, the obtained target transaction confirmation can be in document, picture and / or PDF format; in addition, if the target transaction confirmation is an encrypted document, the encrypted document needs to be decrypted before parsing.

[0094] In some embodiments of this specification, general element extraction rules and transaction confirmation templates can be created in advance according to the transaction type. If the transaction confirmation only changes in layout, the embodiment of this specification does not need to modify the element extraction rules for the new layout, and a new layout template can be provided for comparison; if a new transaction confirmation of a counterparty appears, the same type of element extraction rule configuration can be reused, and it can be checked whether some rules need to be adjusted in a targeted manner; this is conducive to further improving the efficiency and adaptability of the review and processing of the transaction confirmation.

[0095] Element extraction rules refer to the extraction rules (such as extraction method, extraction range, etc.) for transaction elements (such as transaction reference number, transaction party, etc.). For example, taking the transaction reference number as an example, this element often has no key value but only value. You can extract it through the configured regular expression and remove unnecessary text (such as colons, periods, etc.) with the conversion rules.

[0096] The transaction confirmation template is a standard template for transaction confirmations and is the comparison benchmark for pending transaction confirmations. Transaction confirmation templates can be created in advance based on transaction types.

[0097] In some embodiments of the present specification, performing OCR recognition on the target transaction confirmation and the transaction confirmation template may include: performing OCR recognition on each page of the target transaction confirmation and the transaction confirmation template, obtaining OCR recognition results of the text and / or table contained in the image at different rotation angles (0°, 90°, 180°, 270°), and confirming the OCR recognition result in the positive direction accordingly. The OCR recognition result in the positive direction of the target transaction confirmation is the first confirmation text; the OCR recognition result in the positive direction of the transaction confirmation template is the first template text.

[0098] refer to Figure 3 As shown, in some embodiments of this specification, performing layout analysis on the first confirmation text and the first template text may include the following steps:

[0099] Step 301: Assemble the first confirmation text and the first template text row by row according to the four corner coordinates of the text, and obtain a first assembled text and a second assembled text respectively.

[0100] For example, taking the first confirmation text as an example, the OCR recognition results of the corresponding pages in the first confirmation text can be assembled in row order according to the four corner coordinates of each page of the target transaction confirmation, so as to obtain the corresponding assembly results of each page in the first confirmation text, and then form the first assembled text corresponding to the first confirmation text. In the same way, the first template text is assembled to obtain the second assembled text. The assembly of the embodiments of this specification is a structured arrangement, and the OCR recognition results can be standardized and structured through assembly, which is convenient for subsequent processing.

[0101] Step 302: For each assembled text, the first detection frame of each line of text is compared with the second detection frame of the layout analysis to confirm the category of each line of text and the ownership of the document element area.

[0102] The first detection box L is the text coverage of the current line of text, and the second detection box T of the layout analysis is a specific document element area (such as the transaction reference number area, the transaction party area, etc.).

[0103] In some embodiments of the present specification, the first detection box L of each line of text is compared with the second detection box T of the layout analysis, which can be as follows: determine whether the four corner coordinates of the first detection box L of the current line of text are within the range of the second detection box T of the layout analysis; if so, confirm that the document element category of the current line is the same as the second detection box T of the layout analysis; otherwise, confirm that the document element category of the current line is different from the second detection box T of the layout analysis.

[0104] For example, in Figure 6 In the exemplary embodiment shown, the second detection frame includes document element areas such as headers, text paragraphs, tables, transaction element keys, and transaction element values. Figure 6 If the positions of the four black dots in are within the header range of the second detection frame of the layout analysis, then the document element category of the first line of text is the same as the document element category of the second detection frame (that is, the first line of text is the header); and so on.

[0105] In other embodiments of the present specification, the first detection box L of each line of text is compared with the second detection box T of the layout analysis, which can be:

[0106] Determine whether the Intersection over Union (IOU) between the first detection box L of the current line of text and the second detection box T of the layout analysis is greater than a first threshold; if so, confirm that the document element category of the current line is the same as the second detection box T of the layout analysis; otherwise, confirm that the document element category of the current line is different from the second detection box T of the layout analysis. The overlap refers to the ratio of the overlapping area of ​​two detection boxes to their union area.

[0107] In other embodiments of the present specification, the first detection frame L of each line of text is compared with the second detection frame T of the layout analysis to determine whether the quotient obtained by dividing the intersection area of ​​the first detection frame of the current line of text and the second detection frame of the layout analysis by the area of ​​the first detection frame of the current line of text is greater than the second threshold, that is, to determine whether the intersection area of ​​the first detection frame of the current line of text and the second detection frame of the layout analysis is greater than the second threshold. Is it greater than the second threshold, where s L is the area of ​​the first detection box of the current line of text, s T is the area of ​​the second detection box T of the layout analysis; if so, it is confirmed that the document element category of the current row is the same as the second detection box of the layout analysis; otherwise, it is confirmed that the document element category of the current row is different from the second detection box T of the layout analysis.

[0108] When the document element category of the current row is different from the second detection box T of the layout analysis, there may be the following two situations:

[0109] (1) The current text line does not match only one document element region of the layout analysis, but may still match other document element regions that have not been compared.

[0110] (2) If the current text line does not match any element area of ​​the layout analysis, no category information will be assigned to the text line. This may be caused by multiple inspections by the text detection model or missed inspections by the layout analysis file.

[0111] Step 303: For each assembled text, the text format of each page content is sorted and non-text information (such as headers, footers, etc.) is removed according to the category of each line of text and the affiliation of the document element area, and the second confirmation text and the second template text are obtained accordingly.

[0112] By organizing the text format of each page, the text format can be further standardized and structured; by eliminating non-text information, the efficiency of subsequent feature extraction and comparison can be improved, avoiding the situation where it is difficult to accurately extract feature information due to the diversity and complexity of typesetting (columns, tables, multiple lines, cross-page spreads, etc.).

[0113] Since the layout analysis in the embodiments of this specification is essentially a target detection / segmentation + classification task, in other embodiments of this specification, a pre-trained deep learning model can also be used to implement layout analysis. For example, a pre-trained Mask DINO model based on the improved DETR model can be used to implement layout analysis. The Mask DINO model is a unified target detection and segmentation framework, which realizes the mutual improvement of detection and segmentation tasks in the Transformer-based model and has good results. In some embodiments of this specification, the model is trained and optimized for the transaction confirmation scenario.

[0114] For example, in terms of data annotation, the way data is annotated greatly affects the effect of the model. Inappropriate annotation methods will make it difficult for the model to learn the features that should be paid attention to. In the embodiment of this specification, eight categories of document elements are determined based on the layout and element features of the transaction confirmation: text paragraphs, headers, footers, headings, transaction element keys, transaction element values, tables, and signatures, while ensuring the distinction of each category and the similarity of the same category. In addition, unified specifications can be formulated for data annotation (such as the size of the annotation box, element segmentation rules, category distinction standards, etc.) to maintain data consistency.

[0115] For example, in terms of data enhancement, the following optimizations were made based on the characteristics of the transaction confirmation:

[0116] Since document-type data does not have left-right flipping, this enhancement is removed; since the ratio of some document elements to the complete image is small, a smaller image scaling ratio is not conducive to the model learning smaller document element features, so the dynamic scaling ratio is adjusted; for scanned copies of transaction confirmations, small-angle rotations are inevitable during scanning. For such scenarios, random small-angle (-1.5 to 1.5) rotations can be performed during data enhancement to improve model robustness.

[0117] In some embodiments of the present specification, element extraction may include text element extraction and / or table element extraction. In the embodiments of the present specification, extracting elements is a write operation, and after the elements are extracted, there are no corresponding elements in the second confirmation text and the second template text. In addition, in the embodiments of the present specification, the purpose of extracting the second extraction result from the second template text is to obtain a second template text that does not contain the second extraction result. Therefore, the second extraction result itself is useless, so the second extraction result can be deleted. In other words, extracting the second extraction result from the second template text is equivalent to filtering or removing the data corresponding to the second extraction result from the second template text (generally the corresponding historical data in the template).

[0118] refer to Figure 4As shown, in some embodiments of this specification, the extraction logic of text element extraction may include the following steps:

[0119] Step 401: Identify feature areas.

[0120] The extraction rules of text elements are divided into areas (i.e. element areas), such as the full text range, or the range between specified keywords. In addition, the table area can be configured to be processed according to the logic of text element extraction to adapt to some special transaction confirmation formats. The extraction rule configuration within the area is basically the same as that of the table elements, but a maximum number of rows corresponding to each field is added.

[0121] Therefore, the element area (starting line, ending line) where the transaction element is located can be determined based on the transaction element area positioning keywords and the results of layout analysis. For example, first determine whether there is a positioning keyword that defines the element area. If not, it means that the current configuration is full-text search; if there is a positioning keyword that defines the element area, such as the starting keyword is: TRADEDETAILS, and the ending keyword is: Table of Terms for Each Transaction. Then, combined with the transaction elements and paragraph information given by the layout analysis, find the starting text line and the ending text line of the element area.

[0122] Step 402: Determine the extraction rule corresponding to the element area.

[0123] Different element regions may predefine corresponding extraction rules, and corresponding extraction rules may be matched from an extraction rule set through the element regions.

[0124] Step 403: extract the value in the element area according to the extraction rule.

[0125] For example, according to the configured element keywords, search and match are performed one by one in the element area. For example, the keyword of "buyer" is Buyer or Premium Amounts Buyer, and the relative position of value to the keyword is right. After finding the keyword, look for the value on the right side of the keyword, and perform multi-line search based on the configured maximum number of value rows and the paragraph information of the layout model. The index of the finally extracted content in the text row is recorded for subsequent comparison and elimination. Since some value values ​​are partially extracted from the text, the entire line of text cannot be recorded as extracted, and the unextracted part still needs to be compared and reviewed.

[0126] Step 404: When a reference relationship exists in the value value, a recursive search is performed on the value content of the value value, and the final value value is assigned to all key values ​​on the reference chain.

[0127] Specifically, if there is a reference relationship in the value value, a recursive search is performed on the value value content, and the key value of the reference target is found according to the value value content. The reference relationship is judged for the value value corresponding to the key value until there is no reference relationship in the value value, and the final value value is assigned to all key values ​​on the reference chain.

[0128] refer to Figure 5 As shown, in some embodiments of the present specification, the extraction logic of table element extraction may include the following steps:

[0129] Step 501, determine whether the currently processed form is the last form on this page. If the currently processed form is the last form on this page, execute step 502, otherwise execute step 504.

[0130] Step 502: Determine whether the last table on this page has the same number of columns as the first table on the next page. If the last table on this page has the same number of columns as the first table on the next page, execute step 503; otherwise, execute step 504.

[0131] Step 503: merge the last table on this page with the first table on the next page.

[0132] Step 504: Determine corresponding extraction rules according to the table title or table header keywords of the table.

[0133] For example, if you need to extract a detailed table of each transaction, the table usually appears in the attachment of the transaction confirmation. Usually there is no text to clearly indicate the name of the table, so you can judge it by the table header. In this scenario, the rules for table extraction mainly include:

[0134] (1) Header keywords to be extracted: for example, transaction number (TransactionNumber), valuation date (ValuationDate), settlement date (Settlement Date), etc.;

[0135] (2) Specific extraction configuration for each header: the position of the value relative to the header (top, bottom, left, right), header search keywords (may include regular expressions), regular extraction rules for the value, conversion rules for the value (partial content replacement), reference relationship search logic, and element properties (whether to return, whether separate regularization and conversion processing is required if there is a reference).

[0136] Step 505: Search the cell texts of the merged table one by one according to the field keywords or regular expressions in the extraction rules to match the target cells corresponding to each extraction rule, and record the extraction rules that do not match the target cells.

[0137] Step 506: Query the value corresponding to the target cell according to the set search direction.

[0138] For the field rules that match the cell, search for the value corresponding to the field according to the set search direction (up, down, left, right), which may be a single value or a list value.

[0139] If the matched cell in the first row and first column is TransactionNumber, the value of the element is found according to the positional relationship between the configured value and the header. If the configured search method is down (i.e. downward), then search downward and record until other header keywords are encountered or the table ends. If the field has a configured reference relationship (i.e. the value corresponding to other headers), the found value is replaced with the reference value, that is, in the extraction logic of the table element extraction, when there is a reference relationship in the value value, the value content of the value value is recursively searched, and the final value value is assigned to all key values ​​on the reference chain. Finally, the text is processed according to the configured regular rules and conversion rules, such as date extraction, amount formatting, etc. In some embodiments of the present specification, when the first extraction result is compared with the corresponding data in the target system, and when the first integrated text and the second integrated text are compared, a difference or similarity comparison method such as edit distance can be performed, and the difference points can be marked.

[0140] Taking the comparison between the first integrated text and the second integrated text as an example, for the comparison of ordinary text, the text content can be compared in paragraphs, so that the comparison differences caused by different row texts due to factors such as fonts, text sizes, and page margins can be eliminated; for the comparison of tables, the extracted elements are removed according to the cell matching information recorded when the elements are extracted, and only the fixed content is compared; for the comparison of transaction element areas, the element key value is compared with the corresponding template value. At the same time, according to the extracted element text index recorded when the elements are extracted, the extracted part of the value is removed from the comparison range.

[0141] In addition, in some embodiments of the present specification, the obtained comparison result and the first extraction result can be stored in a specified file format. For example, the first extraction result can be written into the ledger for storage after JSON parsing, so that the subsequent ledger data can be checked with the corresponding data in the SUMMIT system.

[0142] Although the process flows described above include multiple operations that occur in a particular order, it should be clearly understood that the processes may include more or fewer operations, which may be performed sequentially or in parallel (eg, using parallel processors or a multi-threaded environment).

[0143] Corresponding to the transaction confirmation review and processing method described above, the embodiment of this specification also provides a transaction confirmation review and processing device, which can be configured on the above-mentioned server, referring to Figure 7 As shown, in some embodiments of this specification, the transaction confirmation review processing device may include:

[0144] A determination module 71, used to determine the transaction type in the target transaction confirmation;

[0145] A matching module 72, used for matching the element extraction rules and the transaction confirmation template corresponding to the transaction type;

[0146] The recognition module 73 is used to perform OCR recognition on the target transaction confirmation and the transaction confirmation template, and obtain a first confirmation text and a first template text accordingly;

[0147] An analysis module 74 is used to perform layout analysis on the first confirmation text and the first template text, and obtain a second confirmation text and a second template text accordingly;

[0148] An extraction module 75, configured to extract elements from the second confirmation text and the second template text according to the element extraction rule, obtain a first extraction result and a second extraction result accordingly, and delete the second extraction result;

[0149] An integration module 76 is used to integrate the remaining part of the second confirmation text after the elements are extracted to obtain a first integrated text; and integrate the remaining part of the second template text after the elements are extracted to obtain a second integrated text;

[0150] The comparison module 77 is used to compare the first extraction result with the corresponding data in the target system, and to compare the first integrated text with the second integrated text to generate a comparison result marked with differences.

[0151] For the convenience of description, the above device is described by dividing it into various units according to its functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0152] The embodiments of this specification also provide a computer device to execute the above method, such as Figure 8As shown, in some embodiments of the present specification, the computer device 802 may include one or more processors 804, such as one or more central processing units (CPUs) or graphics processing units (GPUs), and each processing unit may implement one or more hardware threads. The computer device 802 may also include any memory 806, which is used to store any kind of information such as code, settings, data, etc. In a specific embodiment, the computer program on the memory 806 and can be run on the processor 804, when the computer program is run by the processor 804, can execute the instructions of the transaction confirmation audit processing method described in any of the above embodiments. Non-limiting, for example, the memory 806 may include any one or more combinations of the following: any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Further, any memory can provide volatile or non-volatile retention of information. Further, any memory can represent a fixed or removable component of the computer device 802. In one case, when the processor 804 executes the associated instructions stored in any memory or combination of memories, the computer device 802 can perform any operation of the associated instructions. The computer device 802 also includes one or more drive mechanisms 808 for interacting with any storage, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.

[0153] The computer device 802 may also include an input / output interface 810 (I / O) for receiving various inputs (via input devices 812) and for providing various outputs (via output devices 814). A specific output mechanism may include a presentation device 816 and an associated graphical user interface 818 (GUI). In other embodiments, the input / output interface 810 (I / O), input device 812, and output device 814 may not be included, and the computer device 802 may be used as a computer device in a network. The computer device 802 may also include one or more network interfaces 820 for exchanging data with other devices via one or more communication links 822. One or more communication buses 824 couple the components described above together.

[0154] The communication link 822 may be implemented in any manner, for example, through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 822 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc. governed by any protocol or combination of protocols.

[0155] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), computer-readable storage media, and computer program products of some embodiments of the present specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processor to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processor generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0156] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processor to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0157] These computer program instructions can also be loaded onto a computer or other programmable data processor so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0158] In a typical configuration, a computer device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0159] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0160] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computer device. As defined in this specification, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0161] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may take the form of complete hardware embodiments, complete software embodiments or embodiments combining software and hardware. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0162] The present specification embodiments may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present specification embodiments may also be practiced in distributed computing environments where tasks are performed by remote processors connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0163] It should also be understood that in the embodiments of this specification, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0164] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0165] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of this specification. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0166] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A transaction confirmation review and processing method, characterized in that: include: Determine the transaction type in the target transaction confirmation; Matching the element extraction rules and transaction confirmation template corresponding to the transaction type; Performing OCR recognition on the target transaction confirmation and the transaction confirmation template to obtain a first confirmation text and a first template text respectively; Performing layout analysis on the first confirmation text and the first template text to obtain a second confirmation text and a second template text accordingly; Extracting elements from the second confirmation letter text and the second template text according to the element extraction rule, obtaining a first extraction result and a second extraction result accordingly, and deleting the second extraction result; Integrate the remaining part of the second confirmation text after extracting the elements to obtain a first integrated text; and integrating the remaining part of the second template text after the extracted elements to obtain a second integrated text; The first extraction result is compared with the corresponding data in the target system, and the first integrated text is compared with the second integrated text to generate a comparison result marked with difference points.

2. The transaction confirmation review and processing method according to claim 1, characterized in that: Performing layout analysis on the first confirmation text and the first template text, including: Assemble the first confirmation text and the first template text row by row according to the coordinates of the four corners of the text, and obtain a first assembled text and a second assembled text respectively; For each assembled text, the first detection frame of each line of text is compared with the second detection frame of the layout analysis to confirm the category of each line of text and the belonging of the document element area; For each assembled text, the text format of each page is organized and non-text information is removed according to the category of each line of text and the affiliation of the document element area, and a second confirmation text and a second template text are obtained accordingly.

3. The transaction confirmation review and processing method according to claim 2, characterized in that: The step of comparing the first detection frame of each line of text with the second detection frame of the layout analysis includes: Determine whether the four corner coordinates of the first detection frame of the current line of text are within the range of the second detection frame of the layout analysis; if so, confirm that the document element category of the current line is the same as the second detection frame of the layout analysis.

4. The transaction confirmation review and processing method according to claim 2, characterized in that: The step of comparing the first detection frame of each line of text with the second detection frame of the layout analysis includes: Determine whether the IOU of the first detection box of the current line of text and the second detection box of the layout analysis is greater than a first threshold; if so, confirm that the document element category of the current line is the same as the second detection box of the layout analysis.

5. The transaction confirmation review and processing method according to claim 2, characterized in that: The step of comparing the first detection frame of each line of text with the second detection frame of the layout analysis includes: Determine whether the quotient of the intersection area of ​​the first detection box of the current line of text and the second detection box of the layout analysis divided by the area of ​​the first detection box of the current line of text is greater than a second threshold; if so, confirm that the document element category of the current line is the same as the second detection box of the layout analysis.

6. The transaction confirmation review and processing method according to claim 1, characterized in that: The element extraction includes table element extraction, and the processing logic of the table element extraction includes: Determine whether the currently processed table is the last table on this page; if so, and the first table on the next page has the same number of columns as the current table, merge the last table on this page with the first table on the next page; Determine the corresponding extraction rules according to the table title or table header keywords of the merged table; According to the field keywords or regular expressions in the extraction rules, the cell texts of the merged table are searched one by one to match the target cells corresponding to each extraction rule, and the extraction rules that do not match the target cells are recorded; Query the value corresponding to the target cell according to the set search direction; When there is a reference relationship in the value value, the value content of the value value is recursively searched and the final value value is assigned to all key values ​​on the reference chain.

7. The transaction confirmation review and processing method according to claim 1, characterized in that: The element extraction includes text element extraction, and the extraction logic of the text element extraction includes: Identify feature areas; Determine an extraction rule corresponding to the element area; Extract the value in the element area according to the extraction rule; When there is a reference relationship in the value value, the value content of the value value is recursively searched and the final value value is assigned to all key values ​​on the reference chain.

8. A transaction confirmation review and processing device, characterized in that: include: A determination module, used to determine the transaction type in the target transaction confirmation; A matching module, used for matching the element extraction rules and the transaction confirmation template corresponding to the transaction type; An identification module, used for performing OCR identification on the target transaction confirmation and the transaction confirmation template, and correspondingly obtaining a first confirmation text and a first template text; An analysis module, used for performing layout analysis on the first confirmation text and the first template text, and correspondingly obtaining a second confirmation text and a second template text; An extraction module, used to extract elements from the second confirmation text and the second template text according to the element extraction rule, obtain a first extraction result and a second extraction result accordingly, and delete the second extraction result; An integration module, used for integrating the remaining part of the second confirmation text after the extracted elements, to obtain a first integrated text; and integrating the remaining part of the second template text after the extracted elements to obtain a second integrated text; A comparison module is used to compare the first extraction result with the corresponding data in the target system, and to compare the first integrated text with the second integrated text to generate a comparison result marked with difference points.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: When the computer program is executed by the processor, the computer program executes the instructions of the method according to any one of claims 1 to 7.

10. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor of a computer device, the computer program executes instructions of the method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor of a computer device, the computer program executes instructions of the method according to any one of claims 1 to 7.

Images