A method, apparatus, electronic device, and storage medium for reviewing financial contracts.

Abstract

This application provides a method, apparatus, electronic device, and storage medium for reviewing financial contracts. The method includes: determining a pre-trained target segmentation model based on the template type corresponding to the financial contract to be reviewed; inputting the financial contract to be signed and the signed financial contract into the target segmentation model as images to output text segmentation information corresponding to each contract; inputting the text segmentation screenshots corresponding to the segmentation screenshots of the financial contract to be signed and the signed financial contract into a pre-trained optical character recognition model to output text recognition results corresponding to each text segmentation screenshot; and performing text matching on the text recognition results corresponding to the text segmentation screenshots of the financial contract to be signed and the signed financial contract that have the same text category to determine whether the financial contract to be signed and the signed financial contract meet the contract signing standards.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a method, apparatus, electronic device, and storage medium for reviewing financial contracts. Background Technology

[0002] Contracts serve as proof of transactions across various industries. Due to their wide range of applications, a vast amount of contract text data is generated in the physical world. This large volume of contract text exacerbates contract review time and lengthens customer waiting periods. A common industry solution involves directly performing text recognition on the contract text, then matching text lines, and comparing the text to confirm whether the contract has been modified.

[0003] However, when using the above methods to compare contract texts, issues such as missing or misaligned lines often occur, ultimately leading to discrepancies in the comparison results, affecting the review progress, and increasing the workload of manual secondary review. Therefore, a more accurate and efficient contract review method is needed. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method, apparatus, electronic device and storage medium for reviewing financial contracts, so as to improve the efficiency and accuracy of contract review.

[0005] Firstly, this application provides a method for reviewing financial contracts. The method includes determining a pre-trained target segmentation model based on the template type corresponding to the financial contract to be reviewed. The financial contract to be reviewed includes a financial contract to be signed drafted by one of the contracting parties and a financial contract already signed as reported by the other party. The financial contract to be signed and the financial contract already signed are input into the target segmentation model in image format to output text segmentation information corresponding to each contract. The text segmentation information includes at least a text segmentation screenshot and the text category corresponding to each text segmentation screenshot. The text segmentation screenshots of the financial contract to be signed and the financial contract already signed are input into a pre-trained optical character recognition model to output the text recognition result corresponding to each text segmentation screenshot. Text matching is performed on the text recognition results corresponding to the text segmentation screenshots of the financial contract to be signed and the financial contract already signed that have the same text category to determine whether the financial contract to be signed and the financial contract already signed meet the contract signing criteria.

[0006] Preferably, the text segmentation information also includes the vertex coordinates of each text segmentation screenshot. Before performing text matching on the text recognition results corresponding to the text segmentation screenshots with the same text category between the financial contract to be signed and the signed financial contract to determine whether the financial contract to be signed and the signed financial contract meet the contract signing standards, the method further includes: for any contract between the financial contract to be signed and the signed financial contract, sorting all text segmentation screenshots based on the vertex coordinates of all text segmentation screenshots of that contract, and combining a text segmentation screenshot of a financial contract to be signed with the same sequence number and a text segmentation screenshot of a signed financial contract into a screenshot data group; for each screenshot data group, performing text category matching on the two text segmentation screenshots within that screenshot data group; if the text categories of each screenshot data group are successfully matched, then performing the text matching step; if there are groups with mismatched text categories, then marking the two text segmentation screenshots within that group.

[0007] Preferably, the text recognition result includes multiple fields and the coordinates of the recognition box for each field. The step of matching the text recognition results corresponding to text segmentation screenshots with the same text category between the financial contract to be signed and the already signed financial contract to determine whether the financial contract to be signed and the already signed financial contract meet the contract signing criteria specifically includes: for any text segmentation screenshot, arranging and connecting the fields in the text recognition result corresponding to that text segmentation screenshot according to the coordinates of the recognition box for each field to obtain the text within that text segmentation screenshot; performing character-by-character matching on the text of two text segmentation screenshots with the same text category; if the characters between the texts in each group of text segmentation screenshots with the same text category are matched, then it is determined that the financial contract to be signed and the already signed financial contract meet the contract signing criteria; otherwise, they do not meet the contract signing criteria.

[0008] Preferably, if the characters in the text segmentation screenshots of the same text category in each group are not completely matched, then these two text segmentation screenshots are marked; and based on the marking of all text segmentation screenshots, a financial contract audit report is generated and output.

[0009] Preferably, the target segmentation model includes a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit, a region generation network unit, a segmentation unit, a classification unit, and a detection unit. The input of the first feature extraction unit serves as the input of the target segmentation model. The output of the first feature extraction unit is connected to the input of the second feature extraction unit. The output of the second feature extraction unit is connected to the input of the third feature extraction unit. The output of the third feature extraction unit is connected to the input of the region generation network unit. The output of the region generation network unit is connected to the input of the fourth feature extraction unit. The output of the region generation network unit is also connected to the input of the segmentation unit. The output of the segmentation unit serves as the first output of the target segmentation model, used to output a text segmentation screenshot. The output of the fourth feature extraction unit is connected to the input of the classification unit. The output of the classification unit serves as the second output of the target segmentation model, used to output the text category corresponding to the text segmentation screenshot. The output of the fourth feature extraction unit is also connected to the input of the detection unit. The output of the detection unit serves as the third output of the target segmentation model, used to output the vertex coordinates of the text segmentation screenshot.

[0010] Preferably, each of the first feature extraction unit, the second feature extraction unit, the third feature extraction unit, and the fourth feature extraction unit includes multiple attention sub-units connected in sequence. Each attention sub-unit includes multiple single-head attention blocks. Each single-head attention block includes multiple attention layers, pooling layers, direct-connected layers, and fully connected layers. The number of attention layers in different feature extraction units is different.

[0011] Preferably, the target segmentation model is trained and generated in the following manner: For each training sample, the labeled training sample is input into the initial segmentation model to be trained, so as to determine the first loss value corresponding to the output of the region generation network unit, the second loss value corresponding to the output of the segmentation unit, the third loss value corresponding to the output of the classification unit, and the fourth loss value corresponding to the output of the detection unit respectively; the sum of the first loss value, the second loss value, the third loss value, and the fourth loss value is taken as the total loss value;

[0012] The parameters of the initial segmentation model are adjusted based on the total loss value to generate the target segmentation model.

[0013] Preferably, for each screenshot data group, text category matching is performed in the following way: determine whether the text categories of two text segmentation screenshots within the screenshot data group are the same; if they are the same, the screenshot data group is determined to be a successful match; if they are not the same, the screenshot data group is determined to be a mismatch.

[0014] Preferably, the annotation scenarios include text mismatch and text category mismatch. The financial contract review report is generated in the following way: when the text category does not match, both the financial contract to be signed and the signed financial contract are added completely to the financial contract review report page; when the text does not match, screenshots of the text segmentation that do not match between the financial contract to be signed and the signed financial contract are added to the financial contract review report page respectively.

[0015] Preferably, when both text category mismatch and text mismatch exist, the financial contract to be signed and the signed financial contract are added completely to the financial contract review report page. Based on the vertex coordinates of the text segmentation screenshots marked as text mismatch, a first screenshot frame is generated and added to the upper layer of the financial contract to be signed and the signed financial contract to generate the financial contract review report.

[0016] Preferably, for each group of text segmentation screenshots with the same text category, when there are mismatched characters between the two, the vertex coordinates of the text recognition box are determined and recorded.

[0017] Preferably, when the text does not match, a second screenshot frame is generated based on the vertex coordinates of the text recognition frame and added to the upper layer of the corresponding text segmentation image to generate a financial contract audit report.

[0018] Secondly, this application provides a device for reviewing financial contracts, the device comprising:

[0019] The pre-selection module is used to determine the pre-trained target segmentation model based on the template type corresponding to the financial contracts to be reviewed. The financial contracts to be reviewed include financial contracts to be signed drafted by one of the contracting parties and financial contracts already signed reported by the other party.

[0020] The segmentation module is used to input the financial contracts to be signed and the financial contracts already signed into the target segmentation model in image format, and output the text segmentation information corresponding to the financial contracts to be signed and the financial contracts already signed. The text segmentation information includes at least the text segmentation screenshot and the text category corresponding to each text segmentation screenshot.

[0021] The recognition module is used to input the text segmentation screenshots corresponding to the segmentation screenshots of the financial contracts to be signed and the financial contracts that have been signed into a pre-trained optical character recognition model, so as to output the text recognition result corresponding to each text segmentation screenshot;

[0022] The verification module is used to perform text matching on the text recognition results corresponding to the text segmentation screenshots of the financial contracts to be signed and the financial contracts that have been signed, in order to determine whether the financial contracts to be signed and the financial contracts that have been signed meet the contract signing standards.

[0023] Thirdly, this application also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions that can be executed by the processor. When the electronic device is running, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the financial contract review method described above are performed.

[0024] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the financial contract review method described above.

[0025] This application provides a method, apparatus, electronic device, and storage medium for reviewing financial contracts. The method includes: determining a pre-trained target segmentation model based on the template type corresponding to the financial contract to be reviewed; the financial contract to be reviewed includes a draft financial contract to be signed by one party and a signed financial contract reported by the other party; inputting the financial contract to be signed and the signed financial contract into the target segmentation model as images to output text segmentation information corresponding to each contract, the text segmentation information including at least text segmentation screenshots and the text category corresponding to each screenshot; inputting the text segmentation screenshots corresponding to the screenshots of the financial contract to be signed and the signed financial contract into a pre-trained optical character recognition model to output the text recognition result corresponding to each screenshot; and performing text matching on the text recognition results corresponding to the text segmentation screenshots of the financial contract to be signed and the signed financial contract that have the same text category to determine whether the financial contract to be signed and the signed financial contract meet the contract signing criteria. By first segmenting the contract into regions, then extracting text from each segmented region, and finally matching the text at the region level, the probability of cross-referencing during text recognition is reduced, thus improving the efficiency and accuracy of contract review.

[0026] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating a method for reviewing financial contracts provided in this application embodiment;

[0029] Figure 2 A flowchart illustrating the comparison steps for text segmentation screenshots provided in the embodiments of this application;

[0030] Figure 3 A flowchart illustrating a text recognition result comparison step provided in an embodiment of this application;

[0031] Figure 4 A schematic diagram of the structure of a financial contract review device provided in an embodiment of this application;

[0032] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0034] First, the applicable scenarios for this application will be introduced. This application can be applied to the review of contracts before and after signing during the transaction process.

[0035] Contracts serve as proof of transactions across various industries. Due to their diverse applications, a vast amount of contract text data is generated in the physical world. This large volume of contract text exacerbates contract review time and lengthens customer waiting periods. A common industry solution involves directly performing text recognition on the contract text, matching lines, and comparing the text to confirm whether the contract has been modified. However, this method often results in line breaks and misalignments, leading to inaccurate comparison results, impacting review progress, and increasing the workload of secondary manual review. Therefore, a more accurate and efficient contract review method is needed.

[0036] Based on this, embodiments of this application provide a method, apparatus, electronic device, and storage medium for reviewing financial contracts, so as to improve the efficiency and accuracy of contract review.

[0037] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for reviewing financial contracts provided in an embodiment of this application. Figure 1 As shown in the embodiment of this application, a method for reviewing financial contracts includes:

[0038] S101. Based on the template type corresponding to the financial contracts to be reviewed, determine the pre-trained target segmentation model. The financial contracts to be reviewed include financial contracts to be signed drafted by one of the contracting parties and financial contracts already signed reported by the other party.

[0039] The financial contracts mentioned here can be payment agreements, transaction contracts, insurance policies, etc., arising from transactions and payments in the financial fields such as wealth management, securities, and banking. They can also be transaction contracts signed in other fields. The financial contracts to be signed here are those drafted by one party to the transaction and then given to the other party for signature confirmation. The signed financial contracts here are those received by one party to the transaction, signed, and returned to that party afterward. The content of the financial contracts to be signed and the signed financial contracts should be completely identical; however, to prevent modification, both contracts need to be proofread. The financial contracts can be in electronic or paper form, but they need to be converted or scanned into image format before processing.

[0040] Understandably, for different types of transactions, the financial contracts here usually have fixed templates, and the contract formats are mostly similar. Therefore, for financial contracts of the same format, a corresponding segmentation model can be trained. Thus, in step S101, before reviewing the financial contract, the corresponding model can be selected based on the financial contract template to ensure the accuracy of segmentation.

[0041] S102. Input the financial contracts to be signed and the financial contracts already signed into the target segmentation model in image format, so as to output the text segmentation information corresponding to the financial contracts to be signed and the financial contracts already signed. The text segmentation information includes at least the text segmentation screenshot and the text category corresponding to each text segmentation screenshot.

[0042] The segmentation model here can be based on the sections and text content of the financial contract. For example, text categories can include titles, information about Party A and Party B, content columns, lists, clauses, and seals, among others. These text categories can be determined based on the content of the financial contract.

[0043] Each text segmentation screenshot is a region extracted from the financial contract that corresponds to the text category.

[0044] S103. Input the text segmentation screenshots of the financial contracts to be signed and the financial contracts that have been signed into the pre-trained optical character recognition model to output the text recognition result corresponding to each text segmentation screenshot.

[0045] In step S103, an optical character recognition (OCR) model can be used to extract text from each text segmentation screenshot to obtain all the text in the screenshot.

[0046] S104. Perform text matching on the text segmentation screenshots of the financial contracts to be signed and the financial contracts already signed, which are of the same text category, to determine whether the financial contracts to be signed and the financial contracts already signed meet the contract signing criteria.

[0047] Specifically, for each screenshot data group, text category matching is performed as follows: It is determined whether the text categories of two text-segmented screenshots within the same screenshot data group are the same. If they are the same, the screenshot data group is considered a successful match; otherwise, it is considered a mismatch.

[0048] For example, the first text recognition result corresponding to the title extracted from the financial contract to be signed is matched word by word with the second text recognition result corresponding to the title extracted from the signed financial contract to determine whether the titles of the two contracts are completely identical. This process is repeated sequentially, comparing the text in areas such as the information of Party A and Party B, content columns, lists, clauses, and seals, according to the layout order of the financial contracts, to complete the review of the financial contracts and determine whether there are any changes to the content between the financial contract to be signed and the signed financial contract. For mismatched parts, the corresponding text segments can be taken and compiled into a unified review report for secondary review, or direct manual proofreading can be performed.

[0049] The financial contract review method provided in this application first divides the contract into regions, then extracts text from each region, and then performs text matching at the region level. This reduces the probability of serial errors during text recognition and improves the efficiency and accuracy of contract review.

[0050] Please see Figure 2 , Figure 2 This is a flowchart illustrating the comparison steps for text segmentation screenshots according to an embodiment of this application. Figure 2As shown, the text segmentation information also includes the vertex coordinates of each text segmentation screenshot. Before performing text matching on the text recognition results corresponding to the text segmentation screenshots with the same text category between the financial contract to be signed and the signed financial contract to determine whether the financial contract to be signed and the signed financial contract meet the contract signing criteria, the text segmentation screenshots can also be compared in the following ways:

[0051] S201. For any contract among the financial contracts to be signed and the financial contracts that have been signed, sort all the text segmentation screenshots based on the vertex coordinates of all the text segmentation screenshots of the contract, and combine the text segmentation screenshots of the financial contracts to be signed and the text segmentation screenshots of the financial contracts that have been signed with the same sequence number into a screenshot data group.

[0052] Understandably, text segmentation screenshots are ordered according to their layout within the financial contract. For example, they can be ordered in the following order: title, Party A / Party B information, content sections, list, clauses, and seal. For a single financial contract, each text category must have at least one text segmentation screenshot. For instance, Party A / Party B information can be grouped into one text segmentation screenshot, or Party A information can be grouped into one text segmentation screenshot, and Party B information into another. However, there is a fixed layout between these two text segmentation screenshots, and therefore, their order is also fixed. For example, Party A / Party B information can be arranged in the order of Party A information followed by Party B information.

[0053] S202. For each screenshot data group, perform text category matching on the two text segmentation screenshots within that screenshot data group.

[0054] S203. If the text categories of each screenshot data group match successfully, then perform the text matching step.

[0055] For example, for the sorted text segmentation screenshot sequences of the financial contracts to be signed and the financial contracts already signed, the first text segmentation screenshot of the financial contract to be signed is matched with the first text segmentation screenshot of the financial contract already signed by matching the text categories. If the text categories corresponding to the two text segmentation screenshots are the same, such as both being titles, then this set of matches is determined to be successful. If all screenshot data sets are successfully matched, then step S104 can be performed.

[0056] S204. If there are groups where the text categories do not match, then mark the two text segmentation screenshots within that group.

[0057] For two text segmentation screenshots in a screenshot data group where the text categories do not match, these two text segmentation screenshots need to be marked as text category mismatch screenshot data group to facilitate subsequent manual review.

[0058] For example, the first text recognition result corresponding to the title extracted from the financial contract to be signed is matched word by word with the second text recognition result corresponding to the title extracted from the signed financial contract to determine whether the titles of the two contracts are completely identical. Similarly, the third text recognition result corresponding to the information of Party A and Party B extracted from the financial contract to be signed is matched word by word with the fourth text recognition result corresponding to the information of Party A and Party B extracted from the signed financial contract, and so on.

[0059] Figure 3 This is a flowchart illustrating a text recognition result comparison step provided in an embodiment of this application. In one embodiment, the text recognition result includes multiple fields and the coordinates of the recognition box for each field. The step involves matching the text recognition results corresponding to text segmentation screenshots of the same text category between the financial contract to be signed and the already signed financial contract to determine whether the financial contract to be signed and the already signed financial contract meet the contract signing criteria. Specifically, this includes:

[0060] S301. For any text segmentation screenshot, arrange and connect the fields in the text recognition result corresponding to the text segmentation screenshot according to the coordinates of the recognition box of the field to obtain the text within the text segmentation screenshot.

[0061] For any text segmentation screenshot, multiple fields identified need to be concatenated according to their order of arrangement in the screenshot to obtain a complete text paragraph.

[0062] S302. Perform character-by-character matching on the text of two text segmentation screenshots with the same text category.

[0063] S303. If the words in the text segmentation screenshots of each group of texts of the same category are matched, then the financial contract to be signed and the financial contract already signed meet the contract signing criteria; otherwise, the contract signing criteria are not met.

[0064] Here, it is necessary to compare whether the text in each position is exactly the same. If they are exactly the same, it can be determined that the contract signing standard is met, indicating that the content of the financial contract to be signed and the financial contract already signed have not been modified.

[0065] S304. If the characters in the text segmentation screenshots of the same text category in each group are not completely matched, then mark the two text segmentation screenshots.

[0066] Furthermore, it can also generate and output a financial contract audit report based on the annotation of all text segmentation screenshots.

[0067] All the marked text segments can be taken and compiled into a financial contract audit report. Staff can clearly determine whether there are any modifications between two financial contracts, conduct a second audit, discover problems in time, and avoid losses.

[0068] In one embodiment of this application, a financial contract review report can be generated through the following methods:

[0069] First, the annotation situations include text mismatch and text category mismatch. The financial contract review report is generated in the following way: when the text category does not match, the financial contract to be signed and the signed financial contract are added completely to the financial contract review report page; when the text does not match, screenshots of the text segmentation between the financial contract to be signed and the signed financial contract are added to the financial contract review report page respectively.

[0070] When both text category mismatch and text mismatch exist, the financial contract to be signed and the signed financial contract are added completely to the financial contract review report page. Based on the vertex coordinates of the text segmentation screenshots marked as text mismatch, a first screenshot frame is generated and added on top of the financial contract to be signed and the signed financial contract to generate the financial contract review report.

[0071] In addition to financial contracts or screenshots, the report should at least indicate the specific errors, such as which text category failed to match or which text failed to match.

[0072] Furthermore, for each group of text segmentation screenshots with the same text category, when there are mismatched characters between the two, the vertex coordinates of the text recognition box are determined and recorded.

[0073] When the text does not match, a second screenshot frame is generated based on the vertex coordinates of the text recognition frame and added to the upper layer of the corresponding text segmentation image to generate a financial contract audit report.

[0074] In this way, when auditors receive the financial contract audit report and perform a second verification, they can immediately identify the points where the matching fails, thereby improving the overall efficiency of the audit.

[0075] In one embodiment of this application, a segmentation model corresponding to the template type is constructed in the following manner:

[0076] The first step is to select the initial segmentation model. Here, the Swin backbone network is chosen as the prototype for constructing the three branches of segmentation, detection, and classification.

[0077] The target segmentation model includes a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit, a region generation network unit, a segmentation unit, a classification unit, and a detection unit. The input of the first feature extraction unit serves as the input of the target segmentation model. The output of the first feature extraction unit is connected to the input of the second feature extraction unit. The output of the second feature extraction unit is connected to the input of the third feature extraction unit. The output of the third feature extraction unit is connected to the input of the region generation network unit. The output of the region generation network unit is connected to the input of the fourth feature extraction unit. The output of the region generation network unit is also connected to the input of the segmentation unit. The output of the segmentation unit serves as the first output of the target segmentation model, used to output a text segmentation screenshot. The output of the fourth feature extraction unit is connected to the input of the classification unit. The output of the classification unit serves as the second output of the target segmentation model, used to output the text category corresponding to the text segmentation screenshot. The output of the fourth feature extraction unit is also connected to the input of the detection unit. The output of the detection unit serves as the third output of the target segmentation model, used to output the vertex coordinates of the text segmentation screenshot.

[0078] Each of the first, second, third, and fourth feature extraction units includes multiple sequentially connected attention sub-units. Each attention sub-unit includes multiple single-head attention blocks. Each single-head attention block includes multiple attention layers, pooling layers, short-cut layers, and fully connected layers. The number of attention layers varies in different feature extraction units.

[0079] The attention layer here consists of key, query, and value matrix operations. Multiple single-head attention blocks form an attention subunit with a multi-head attention mechanism. Specifically, in the first, second, third, and fourth feature extraction units, the number of attention layers in each single-head attention block is set in a ratio of 3:6:6:9.

[0080] Based on Mask R-CNN, a Region Proposal Network (RPN) unit is added between the third and fourth feature extraction units. This RPN unit consists of convolutional layers, activation layers, and pooling layers.

[0081] Based on the feature extraction backbone network, three parallel pipelines are set up to recognize and detect contract content. The feature vector output by the third feature extraction unit passes through a region generation network unit (outputting the coordinates of the detection box), and finally, the segmentation unit crops the corresponding text cutout. The segmentation unit here consists of a ResNet-based block structure (including convolutional layers, pooling layers, activation layers, and short-cut layers) and a deconvolutional layer. Finally, a 1x1 convolution maps the features into a k-dimensional cutout of image size.

[0082] The feature vector output by the fourth feature extraction unit is then processed by the classification unit to determine the text category of the corresponding text segmentation screenshot. The classification unit here uses an MLP (Multi-Layer Perceptron).

[0083] The detection unit has a similar structure to the classification unit. The detection unit is used to output the vertex coordinates of the text segmentation screenshot. The vertex coordinates are used to indicate the relative position of the text segmentation screenshot in the corresponding financial contract image.

[0084] In one embodiment of this application, the target segmentation model is trained and generated in the following manner:

[0085] Step 1: For each training sample, input the labeled training sample into the initial segmentation model to be trained, so as to determine the first loss value corresponding to the output of the region generation network unit, the second loss value corresponding to the output of the segmentation unit, the third loss value corresponding to the output of the classification unit, and the fourth loss value corresponding to the output of the detection unit.

[0086] Step 2: The sum of the first loss value, the second loss value, the third loss value, and the fourth loss value is taken as the total loss value;

[0087] Step 3: Adjust the parameters of the initial segmentation model based on the total loss value to generate the target segmentation model.

[0088] Next, based on the two-stage segmentation model training method using the transformer (attention mechanism) backbone network, task definition, loss function design, and iterative training are performed:

[0089] A suitable training sample was identified, which consisted of a standardized template financial contract. Unlike other contracts, financial contracts possess stable structural characteristics. After analyzing the collected data, they can be segmented into sections such as title, information about the parties involved, content sections, lists, clauses, and seals.

[0090] The collected training samples were labeled on the cloud annotation platform according to the detection boxes, text segmentation regions and corresponding text categories, and divided into training set, validation set and test set to form a complete dataset.

[0091] The loss function design consists of four parts: the MSE Loss (first loss value) calculated between the RPN layer output and the ground truth bounding box; the MSE Loss (second loss value) calculated between the detection layer output and the ground truth bounding box; the CE Loss (third loss value) calculated between the classification layer and the ground truth class; and the Focal Loss (fourth loss value) calculated between the segmentation layer output and the ground truth mask. Finally, these four losses are combined to form the total loss.

[0092] The learning rate is set to 1e-3, the batch size is set to 64, and the resolution is set to 1024 (short sides are padded). The Adam optimizer is used, and the model is trained iteratively according to the gradient descent algorithm until the loss variation range of the model is very small, thus obtaining the trained segmentation model.

[0093] Based on the same inventive concept, this application also provides a financial contract review device corresponding to the financial contract review method. Since the principle of the device in this application is similar to the financial contract review method described above in this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0094] Please see Figure 4 , Figure 4 This is a schematic diagram of a financial contract review device provided in an embodiment of this application. Figure 4 As shown, the auditing device 400 includes:

[0095] The pre-selection module 410 is used to determine the pre-trained target segmentation model based on the template type corresponding to the financial contract to be reviewed. The financial contract to be reviewed includes a financial contract to be signed drafted by one of the contracting parties and a financial contract already signed reported by the other party.

[0096] The segmentation module 420 is used to input the financial contract to be signed and the financial contract already signed into the target segmentation model in the form of images, so as to output the text segmentation information corresponding to the financial contract to be signed and the financial contract already signed. The text segmentation information includes at least a text segmentation screenshot and the text category corresponding to each text segmentation screenshot.

[0097] The recognition module 430 is used to input the text segmentation screenshots corresponding to the segmentation screenshots of the financial contracts to be signed and the financial contracts that have been signed into a pre-trained optical character recognition model, so as to output the text recognition result corresponding to each text segmentation screenshot;

[0098] The verification module 440 is used to perform text matching on the text recognition results corresponding to the text segmentation screenshots of the financial contracts to be signed and the financial contracts that have been signed, so as to determine whether the financial contracts to be signed and the financial contracts that have been signed meet the contract signing standards.

[0099] In a preferred embodiment, the text segmentation information further includes the vertex coordinates of each text segmentation screenshot, and a matching module (not shown in the figure) for sorting all text segmentation screenshots based on the vertex coordinates of all text segmentation screenshots for any contract between the financial contract to be signed and the signed financial contract. The module combines text segmentation screenshots of a financial contract to be signed and a text segmentation screenshot of a signed financial contract with the same sequence number into a screenshot data group. For each screenshot data group, the module performs text category matching on the two text segmentation screenshots within that screenshot data group. If the text categories of each screenshot data group match successfully, the text matching step is performed. If there are groups where the text categories do not match, the two text segmentation screenshots within that group are marked.

[0100] In a preferred embodiment, the text recognition result includes multiple fields and the coordinates of the recognition box for each field. The verification module 440 is specifically used to, for any text segmentation screenshot, arrange and connect the fields in the text recognition result corresponding to the text segmentation screenshot according to the coordinates of the recognition box of the field to obtain the text within the text segmentation screenshot; perform character-by-character matching on the text of two text segmentation screenshots with the same text category; if the characters between the texts of each group of text segmentation screenshots with the same text category are matched, then it is determined that the financial contract to be signed and the financial contract already signed meet the contract signing standard; otherwise, the contract signing standard is not met.

[0101] In a preferred embodiment, the verification module 440 is further configured to: mark the two text segmentation screenshots if the characters between the texts in each group of text segmentation screenshots of the same text category are not completely matched; and generate and output a financial contract audit report based on the marking of all text segmentation screenshots.

[0102] In a preferred embodiment, the target segmentation model includes a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit, a region generation network unit, a segmentation unit, a classification unit, and a detection unit. The input of the first feature extraction unit serves as the input of the target segmentation model. The output of the first feature extraction unit is connected to the input of the second feature extraction unit. The output of the second feature extraction unit is connected to the input of the third feature extraction unit. The output of the third feature extraction unit is connected to the input of the region generation network unit. The output of the region generation network unit is connected to the input of the fourth feature extraction unit. The output of the region generation network unit is also connected to the input of the segmentation unit. The output of the segmentation unit serves as the first output of the target segmentation model, used to output a text segmentation screenshot. The output of the fourth feature extraction unit is connected to the input of the classification unit. The output of the classification unit serves as the second output of the target segmentation model, used to output the text category corresponding to the text segmentation screenshot. The output of the fourth feature extraction unit is also connected to the input of the detection unit. The output of the detection unit serves as the third output of the target segmentation model, used to output the vertex coordinates of the text segmentation screenshot.

[0103] In a preferred embodiment, each of the first feature extraction unit, the second feature extraction unit, the third feature extraction unit, and the fourth feature extraction unit includes multiple attention sub-units connected in sequence. Each attention sub-unit includes multiple single-head attention blocks. Each single-head attention block includes multiple attention layers, pooling layers, direct-connected layers, and fully connected layers. The number of attention layers in different feature extraction units is different.

[0104] In a preferred embodiment, a training module (not shown in the figure) is further included, which is used to train and generate a target segmentation model in the following manner: for each training sample, the labeled training sample is input into the initial segmentation model to be trained to determine the first loss value corresponding to the output of the region generation network unit, the second loss value corresponding to the output of the segmentation unit, the third loss value corresponding to the output of the classification unit, and the fourth loss value corresponding to the output of the detection unit; the sum of the first loss value, the second loss value, the third loss value, and the fourth loss value is used as the total loss value; the parameters of the initial segmentation model are adjusted based on the total loss value to generate the target segmentation model.

[0105] In a preferred embodiment, for each screenshot data group, the matching module performs text category matching in the following manner: determining whether the text categories of two text segmentation screenshots within the screenshot data group are the same; if they are the same, the screenshot data group is determined to be successfully matched; if they are not the same, the screenshot data group is determined to be mismatched.

[0106] In a preferred embodiment, the annotation situations include text mismatch and text category mismatch. The financial contract review report is generated in the following way: when the text category does not match, the financial contract to be signed and the signed financial contract are added completely to the financial contract review report page; when the text does not match, screenshots of the text segmentation that do not match between the financial contract to be signed and the signed financial contract are added to the financial contract review report page respectively.

[0107] In a preferred embodiment, when both text category mismatch and text mismatch exist, the financial contract to be signed and the signed financial contract are completely added to the financial contract review report page. Based on the vertex coordinates of the text segmentation screenshots marked as text mismatch, a first screenshot frame is generated and added to the upper layer of the financial contract to be signed and the signed financial contract to generate the financial contract review report.

[0108] In a preferred embodiment, for each group of text segmentation screenshots with the same text category, when there are mismatched characters between the two, the vertex coordinates of the text recognition box are determined and recorded.

[0109] In a preferred embodiment, when the text does not match, a second screenshot frame is generated based on the vertex coordinates of the text recognition frame and added to the upper layer of the corresponding text segmentation image to generate a financial contract review report.

[0110] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 500 includes a processor 510, a memory 520, and a bus 530.

[0111] The memory 520 stores machine-readable instructions executable by the processor 510. When the electronic device 500 is running, the processor 510 and the memory 520 communicate via the bus 530. When the machine-readable instructions are executed by the processor 510, they can perform the operations described above. Figure 1 The steps of the financial contract review method in the illustrated method embodiment can be found in the method embodiment for specific implementation, and will not be repeated here.

[0112] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the financial contract review method in the illustrated method embodiment can be found in the method embodiment for specific implementation, and will not be repeated here.

[0113] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0114] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0115] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0116] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0117] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0118] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of auditing a financial contract, characterized by, The method comprises: According to the template type corresponding to the financial contract to be audited, a pre-trained target segmentation model is determined, the financial contract to be audited comprises a to-be-signed financial contract prepared by one party of the contract and a signed financial contract fed back by the other party of the contract, and the segmentation model is segmented according to the division and text content of the financial contract; The to-be-signed financial contract and the signed financial contract are input into the target segmentation model in the format of pictures respectively to output the text segmentation information corresponding to the to-be-signed financial contract and the signed financial contract respectively, and the text segmentation information at least comprises text segmentation screenshots and text categories corresponding to each text segmentation screenshot; The text segmentation screenshots of the to-be-signed financial contract and the signed financial contract are input into a pre-trained optical character recognition model to output the text recognition result corresponding to each text segmentation screenshot; The text recognition results of the text segmentation screenshots with the same text category between the to-be-signed financial contract and the signed financial contract are matched to determine whether the to-be-signed financial contract and the signed financial contract meet the contract signing standard; The text segmentation information further comprises the vertex coordinates of each text segmentation screenshot, and before the step of performing text matching on the text recognition results of the text segmentation screenshots with the same text category between the to-be-signed financial contract and the signed financial contract to determine whether the to-be-signed financial contract and the signed financial contract meet the contract signing standard, the step further comprises: For any one of the to-be-signed financial contract and the signed financial contract, based on the vertex coordinates of all text segmentation screenshots of the contract, all text segmentation screenshots are sorted according to the layout position of the text segmentation screenshots in the financial contract, and a text segmentation screenshot of the to-be-signed financial contract and a text segmentation screenshot of the signed financial contract with the same serial number are combined into a screenshot data group; For each screenshot data group, the two text segmentation screenshots in the screenshot data group are matched in terms of text category; If the text categories of each screenshot data group are successfully matched, the step of text matching is performed; If there is a group with unmatched text categories, the two text segmentation screenshots in the group are marked for manual auditing.

2. The method of claim 1, wherein, The text recognition result comprises a plurality of fields and recognition box coordinates of each field, and the step of performing text matching on the text recognition results of the text segmentation screenshots with the same text category between the to-be-signed financial contract and the signed financial contract to determine whether the to-be-signed financial contract and the signed financial contract meet the contract signing standard specifically comprises: For any one text segmentation screenshot, the fields in the text recognition result corresponding to the text segmentation screenshot are arranged and connected according to the recognition box coordinates of the fields to obtain the text in the text segmentation screenshot; The texts of the two text segmentation screenshots with the same text category are matched word by word. If the words between the texts of the text segmentation screenshots of the same text category in each group are matched, it is determined that the to-be-signed financial contract and the signed financial contract meet the contract signing standard, otherwise, the contract signing standard is not met.

3. The method of claim 2, wherein, If the words between the texts of the text segmentation screenshots of the same text category in each group are not completely matched, the two text segmentation screenshots are marked; and according to the marking of all text segmentation screenshots, a financial contract auditing report is generated and output.

4. The method of claim 1, wherein, The target segmentation model comprises a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit, a region generation network unit, a segmentation unit, a classification unit and a detection unit, wherein the input of the first feature extraction unit is taken as the input of the target segmentation model, the output of the first feature extraction unit is connected with the input of the second feature extraction unit, the output of the second feature extraction unit is connected with the input of the third feature extraction unit, the output of the third feature extraction unit is connected with the input of the region generation network unit, the output of the region generation network unit is connected with the input of the fourth feature extraction unit, the output of the region generation network unit is also connected with the input of the segmentation unit, the output of the segmentation unit is taken as the first output of the target model and used for outputting the text segmentation screenshot, the output of the fourth feature extraction unit is connected with the input of the classification unit, the output of the classification unit is taken as the second output of the target model and used for outputting the text category corresponding to the text segmentation screenshot, and the output of the fourth feature extraction unit is also connected with the input of the detection unit, the output of the detection unit is taken as the third output of the target model and used for outputting the vertex coordinates of the text segmentation screenshot.

5. The method of claim 4, wherein, Each of the first feature extraction unit, the second feature extraction unit, the third feature extraction unit and the fourth feature extraction unit comprises a plurality of attention subunits connected in sequence, each attention subunit comprises a plurality of single-head attention blocks, each single-head attention block comprises a plurality of attention layers, a pooling layer, a direct connection layer and a full connection layer, and the number of attention layers in different feature extraction units is different.

6. The method of claim 4, wherein, The target segmentation model is generated by the following method: For each training sample, the labeled training sample is input into the initial segmentation model to be trained to determine a first loss value corresponding to the output of the region generation network unit, a second loss value corresponding to the output of the segmentation unit, a third loss value corresponding to the output of the classification unit and a fourth loss value corresponding to the output of the detection unit, respectively; The sum of the first loss value, the second loss value, the third loss value and the fourth loss value is taken as a total loss value; The initial segmentation model is adjusted in parameters based on the total loss value to generate the target segmentation model.

7. The method of claim 1, wherein, For each screenshot data group, the matching of the text category is performed by the following method: determining whether the text categories of two text segmentation screenshots in the screenshot data group are the same; If yes, it is determined that the screenshot data group is matched successfully; If no, it is determined that the screenshot data group is not matched.

8. The method of claim 3, wherein, The marking includes text part matching and text category mismatching, and the financial contract auditing report is generated by the following method: When the text categories do not match, the complete signed and unsigned financial contracts are added to the page of the financial contract audit report; When the text does not match, the text segmentation screenshots of the signed and unsigned financial contracts that do not match are added to the page of the financial contract audit report.

9. The method of claim 8, wherein, When the text categories do not match and the text does not match at the same time, the complete signed and unsigned financial contracts are added to the page of the financial contract audit report, and a first screenshot frame is generated according to the vertex coordinates of the text segmentation screenshot marked as text not matching, and is added to the upper layer of the signed and unsigned financial contracts to generate the financial contract audit report.

10. The method of claim 9, wherein, For each set of text segmentation screenshots with the same text category, when there is a text that does not match between the two, the vertex coordinates of the text recognition box are determined and recorded.

11. The method of claim 10, wherein, When the text does not match, a second screenshot frame is generated according to the vertex coordinates of the text recognition box and added to the upper layer of the corresponding text segmentation screenshot to generate the financial contract audit report.

12. An apparatus for auditing a financial contract, the apparatus comprising: The device comprises: a pre-selection module configured to determine a pre-trained target segmentation model according to a template type corresponding to a financial contract to be audited, the financial contract to be audited including an unsigned financial contract prepared by one party of the contract and a signed financial contract fed back by the other party of the contract; a segmentation module configured to input the unsigned financial contract and the signed financial contract into the target segmentation model in the form of pictures respectively, to output text segmentation information corresponding to the unsigned financial contract and the signed financial contract respectively, the text segmentation information including at least text segmentation screenshots and text categories corresponding to each text segmentation screenshot, and the text segmentation information further including vertex coordinates of each text segmentation screenshot; an identification module configured to input the text segmentation screenshots corresponding to the unsigned financial contract and the signed financial contract respectively into a pre-trained optical character recognition model, to output text recognition results corresponding to each text segmentation screenshot; a comparison module configured to perform text matching on the text recognition results corresponding to the text segmentation screenshots with the same text category between the unsigned financial contract and the signed financial contract, to determine whether the unsigned financial contract and the signed financial contract meet the contract signing standard; a matching module configured to, for any one of the unsigned financial contract and the signed financial contract, sort all text segmentation screenshots based on the vertex coordinates of the text segmentation screenshots of the contract, combine a text segmentation screenshot of the unsigned financial contract and a text segmentation screenshot of the signed financial contract with the same serial number into a screenshot data group, perform text category matching on the two text segmentation screenshots in each screenshot data group, perform the text matching step if the text categories of each screenshot data group match successfully, and mark the two text segmentation screenshots in the group if there is a text category mismatch.

13. An electronic device, comprising: The device comprises: A processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor communicates with the memory through the bus, the processor executes the machine readable instructions to execute the steps of the auditing method of the financial contract as claimed in any one of claims 1 to 11.

14. A computer readable storage medium characterized by: The computer readable storage medium stores a computer program, when the processor runs the computer program, the steps of the auditing method of the financial contract as claimed in any one of claims 1 to 11 are executed.

Images