Financial material auditing method, electronic equipment, storage medium and program product

Abstract

The invention provides a financial material auditing method, electronic equipment, a storage medium and a program product. The method comprises the following steps: acquiring a financial material to be audited; the financial material comprises at least one piece of financial image-text data; determining financial categories corresponding to the financial image-text data, and generating image-text features corresponding to the financial image-text data; matching each image-text feature with a preset risk feature library to obtain a feature matching result of the financial material; the risk feature library comprises a plurality of risk feature sub-libraries corresponding to the financial category, and image-text features corresponding to tampering financial materials are stored in the risk feature sub-libraries; and determining an audit result of the financial material according to the feature matching result. The method is used for achieving the technical effects of improving the material auditing accuracy and being high in reliability.

Description

Technical Field

[0001] This application relates to image processing technology, and more particularly to a financial document review method, electronic device, storage medium, and program product. Background Technology

[0002] With increasingly stringent financial regulations, banks and other financial institutions require customers to submit various financial documentation, including ID cards, bank statements, purchase and sales contracts, and property ownership certificates, for loan applications. To improve review efficiency and reduce operational risks, financial institutions are actively promoting the automation and intelligent transformation of their document review processes to cope with the increasing volume of business and the complexity of the documents.

[0003] Currently, the relevant technical solutions mainly rely on traditional image processing technology and optical character recognition (OCR) methods, which combine a preset rule engine to extract information from the materials and verify their format compliance.

[0004] However, the above methods rely on surface image features and text content, making it difficult to identify visual fraud committed through complex means such as Photoshop retouching, photocopy splicing, and partial tampering. This results in insufficient accuracy in determining the authenticity of materials and fails to meet the needs of financial risk control audits. Summary of the Invention

[0005] This application provides a financial material review method, electronic device, storage medium, and program product to achieve high accuracy and reliability in material review.

[0006] In a first aspect, embodiments of this application provide a method for reviewing financial materials, including: obtaining financial materials to be reviewed; the financial materials include at least one set of financial graphic data;

[0007] Determine the financial category corresponding to each of the aforementioned financial graphic data, and generate graphic features corresponding to the financial graphic data;

[0008] The graphic and textual features are matched with a preset risk feature library to obtain the feature matching result of the financial material; the risk feature library contains multiple risk feature sub-libraries corresponding to the financial category, and each risk feature sub-library stores graphic and textual features corresponding to the tampering of financial materials;

[0009] Based on the feature matching results, the review result of the financial materials is determined.

[0010] In one possible implementation, generating graphic features corresponding to the financial graphic data includes:

[0011] For any financial graphic data, at least one target region is determined based on its corresponding financial category; wherein the target region contains semantic information that matches the financial category;

[0012] Obtain the target region features corresponding to the target region, and the non-target region features corresponding to the non-target regions other than the target region in the financial graphic data;

[0013] Based on the calculation of the target region features and their corresponding first weight coefficients, and the calculation of the non-target regions and their corresponding second weight coefficients, the graphic features corresponding to the financial graphic data are generated.

[0014] In one possible implementation, determining at least one target area contained in the financial graphic data according to its corresponding financial category includes:

[0015] Obtain a pre-trained region recognition model; the region recognition model includes multiple preset region recognition sub-models corresponding to different financial categories.

[0016] The financial graphic data and its financial category are input into the region recognition model. The region recognition model calls the corresponding region recognition sub-model according to the financial type corresponding to the financial graphic data and outputs at least one identified target region.

[0017] In one possible implementation, obtaining the target region features corresponding to the target region includes:

[0018] For each target region, a preset image coding model is used to extract the image features of the target region, resulting in multiple initial image features at different scales;

[0019] The initial image features are fused and scaled to obtain image features at a preset scale.

[0020] The text features of the target region are extracted using a preset text encoding model to obtain initial text features at a single scale;

[0021] The initial text features are scaled to obtain text features at a preset scale;

[0022] The image features and text features at the preset scale are fused to obtain the target region features of the target region.

[0023] In one possible implementation, the financial type and each of the graphic features are matched with a preset risk feature database to obtain the feature matching result of the financial material, including:

[0024] For any image and text feature corresponding to any financial image data, a risk feature sub-library corresponding to the image and text feature is determined from the risk feature library based on the financial type corresponding to the financial image data.

[0025] Calculate the feature similarity between the image and text features and the pre-stored risk features in the risk feature sub-library, and obtain the feature matching result corresponding to the image and text features based on the feature similarity and the preset similarity threshold;

[0026] Based on the feature matching results corresponding to each of the aforementioned graphic and textual features, the feature matching results of the financial materials are obtained.

[0027] In one possible implementation, determining the review result of the financial materials based on the feature matching result includes:

[0028] If any of the feature matching results indicates the existence of a successfully matched risk feature, then an audit result indicating that the audit has failed is generated.

[0029] If all the feature matching results indicate that there are no successfully matched risk features, then the target areas with correlation in each of the financial graphic data are obtained;

[0030] Based on preset association logic rules, the logical conflicts of each target region are reviewed to obtain the logical review results;

[0031] If the logical audit result representation has a logical conflict, an audit result requiring manual review will be generated;

[0032] If the logical review result indicates that there is no logical conflict, then a review result indicating that the review has passed is generated.

[0033] Secondly, embodiments of this application provide a financial document verification device, comprising:

[0034] The data acquisition module is used to acquire financial materials to be reviewed; the financial materials include at least one set of financial graphic data.

[0035] The image and text feature generation module is used to determine the financial category corresponding to each of the financial image and text data, and to generate image and text features corresponding to the financial image and text data.

[0036] The feature matching module is used to match each of the image and text features with a preset risk feature library to obtain the feature matching result of the financial material; the feature library contains multiple risk feature sub-libraries corresponding to the financial category, and each risk feature sub-library stores features of tampered financial images and / or features of financial images with mismatched image and text content;

[0037] The materials review module is used to determine the review result of the financial materials based on the feature matching results.

[0038] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0039] The memory stores computer-executed instructions;

[0040] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0041] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0042] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0043] The financial material review method, electronic device, storage medium, and program product provided in this application acquire the materials to be reviewed, which contain financial graphic data, automatically identify their financial categories and generate corresponding graphic features, and then match them with a preset risk feature library classified by financial category, thereby determining the review conclusion based on the feature matching results. This method realizes the automation and accuracy of financial material review, can effectively identify risk features such as tampering, improve review efficiency and reduce the risk of human error, and enhances the pertinence and reliability of risk identification through the construction of a classification feature library. Attached Figure Description

[0044] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0045] Figure 1 A diagram illustrating an application scenario of the financial material review method provided for this application;

[0046] Figure 2 A flowchart illustrating the financial material review method provided for this application;

[0047] Figure 3 A schematic diagram of the financial document review device provided for this application;

[0048] Figure 4 A schematic diagram of the structure of the electronic device provided in this application.

[0049] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0050] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0051] The collection, storage, use, processing, transmission, provision, and disclosure of financial data or user data involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0052] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of the relevant data shall comply with relevant laws, regulations and standards, and corresponding operation entry points shall be provided for users to choose to authorize or refuse.

[0053] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0054] With increasingly stringent financial regulations, banks and other financial institutions require customers to submit various financial documentation, including ID cards, bank statements, purchase and sales contracts, and property ownership certificates, for loan applications. To improve review efficiency and reduce operational risks, financial institutions are actively promoting the automation and intelligent transformation of their document review processes to cope with the increasing volume of business and the complexity of the documents.

[0055] Currently, the relevant technical solutions mainly rely on traditional image processing technology and optical character recognition (OCR) methods, which combine a preset rule engine to extract information from the materials and verify their format compliance.

[0056] However, the above methods rely on surface image features and text content, making it difficult to identify visual fraud committed through complex means such as Photoshop retouching, photocopy splicing, and partial tampering. This results in insufficient accuracy in determining the authenticity of materials and fails to meet the needs of financial risk control audits.

[0057] The financial material review method provided in this application aims to solve the aforementioned technical problems of existing technologies. Specifically, by acquiring the materials to be reviewed, which contain financial graphic data, the method automatically identifies their financial category and generates corresponding graphic features, which are then matched with a pre-defined risk feature library categorized by financial category. Based on the feature matching results, the review conclusion is determined. This method automates and improves the accuracy of financial material review, effectively identifies risk features such as tampering, improves review efficiency, and reduces the risk of human error. Furthermore, the construction of the classification feature library enhances the targeting and reliability of risk identification.

[0058] The financial material review method provided in this application is applicable to various business scenarios that require verification of the authenticity and compliance of financial materials.

[0059] For example, in financial scenarios, this method can be applied to verify the qualifications of bank loan applicants and the documents and materials for insurance claims.

[0060] In addition, the above methods can also be applied to scenarios such as identity and asset verification for securities account opening, verification of the authenticity of materials related to risky transactions, and verification of financial reports during corporate financing.

[0061] In summary, any business process that requires efficient and accurate automatic identification of financial material tampering falls under the application scenarios of the technical solution in this application.

[0062] For ease of understanding, the following is based on Figure 1 The application scenarios applicable to the embodiments of this application are described below. Figure 1 A diagram illustrating an application scenario of the financial document review method provided in this application. See also... Figure 1 Taking a bank loan review scenario as an example, this scenario mainly involves a document uploading terminal and a document review terminal. It is understood that the document uploading terminal and the document review terminal can be physically the same entity, or they can be two different entity terminals that can communicate with each other via wired or wireless networks; this application does not specifically limit this.

[0063] Based on this, the financial document review method in this scenario includes the following steps:

[0064] 1. Users transmit the financial materials to be reviewed to the material review terminal for review and processing via the material upload terminal;

[0065] 2. The document review terminal identifies the financial category corresponding to each financial graphic data in the financial documents and generates the corresponding graphic features;

[0066] 3. The material review terminal is also equipped with a risk feature database classified by financial category. Based on this, the graphic features are matched with the pre-stored risks in the risk feature database, and the review conclusion is determined based on the feature matching results.

[0067] The above implementation process can automate and improve the accuracy of financial material review, effectively identify risk characteristics such as tampering, improve review efficiency and reduce the risk of human error, and enhance the pertinence and reliability of risk identification through the construction of a classification feature library.

[0068] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0069] Figure 2 This is a flowchart illustrating the financial document review method provided in this application. The financial document review device can be a server or an electronic device; the following description uses a server as an example. The method in this embodiment can be implemented through software, hardware, or a combination of both. Figure 2 As shown, the method includes:

[0070] S201. Obtain the financial materials to be reviewed; the financial materials include at least one set of financial graphic data.

[0071] In this embodiment, financial materials can refer to the collection of supporting documents submitted by the user to complete financial transactions such as loan applications and insurance claims.

[0072] Specifically, the material can be received by the back-end servers of financial institutions such as banks and insurance companies through network communication interfaces, and its source includes terminal devices such as smartphones, tablets, and personal computers used by users.

[0073] It should be understood that the submitted financial materials should include at least one piece of financial graphic data. Here, financial graphic data refers to electronic files containing image and / or text information. Specific forms include, but are not limited to: digital images of documents such as ID cards and driver's licenses taken by the user using a mobile phone camera; high-resolution scanned documents of paper pay slips, bank statements, and other materials obtained through a scanner; and screenshots of electronic account statements, asset certificates, etc., taken from online banking or wealth management applications.

[0074] Upon receiving financial materials, data augmentation processing can be performed on the various financial graphic and textual data contained therein to improve the efficiency and accuracy of subsequent feature extraction and material review.

[0075] Specifically, for financial materials in multi-page file format, pagination can be performed first. Based on the layout analysis results, the material can be split into independent single-page images, i.e., financial graphic data, which facilitates subsequent page-by-page processing.

[0076] Optionally, for single-page images with view distortion, the document edge contours can be detected by Hough transform, and the homography matrix H can be solved based on the extracted feature points. Then, the document region can be corrected to a front view by perspective transformation.

[0077] To enhance image quality, a deep learning-based deblurring algorithm can be used to process the image. This neural network model can effectively recover motion blur and scattering blur caused by shooting shake or inaccurate focus, thereby improving image sharpness. Simultaneously, multi-scale artifact removal techniques, combined with wavelet transform and thresholding methods, can be used to specifically remove image artifacts at different scales, preserving the original image details to the maximum extent while eliminating artifacts.

[0078] S202. Determine the financial category corresponding to each financial graphic data, and generate the graphic features of the corresponding financial graphic data.

[0079] In this embodiment, financial categories include, but are not limited to: identity verification documents, income verification documents, asset verification documents, credit reports, and insurance policies.

[0080] Specifically, multimodal feature extraction can be performed on financial graphic data. This involves using an OCR engine to identify and extract text features from the document, while simultaneously using a convolutional neural network to extract image features such as layout, seal position, and table structure. The extracted text and image features are then integrated to generate graphic features that characterize the financial graphic data.

[0081] Then, the image and text features are input into a classification model pre-trained based on a deep neural network. The model classifies and identifies the image and text features to determine the financial category to which the financial image and text data belongs.

[0082] Of course, in some scenarios, financial graphic data can also be directly input into a classification model pre-trained based on a deep neural network. The model can then classify and identify the financial graphic data to determine its financial category.

[0083] Furthermore, after identifying the financial category to which the financial graphic data belongs, textual and image features are extracted in a targeted manner based on the specific semantic requirements corresponding to that financial category, thereby generating graphic features that are highly adapted to that category.

[0084] Based on this, an optional implementation of generating graphic features of financial graphic data based on financial categories provided in this embodiment may include: for any financial graphic data, determining at least one target region contained in the financial graphic data according to its corresponding financial category; wherein, the target region contains semantic information matching the financial category; obtaining the target region features corresponding to the target region, and the non-target region features corresponding to the non-target regions in the financial graphic data other than the target region; generating graphic features corresponding to the financial graphic data based on the calculation of the target region features and their corresponding first weight coefficients, and the calculation of the non-target regions and their corresponding second weight coefficients.

[0085] Considering the differentiated graphic and textual styles of data with different financial attributes, this embodiment, after determining the financial attribute of the data, will identify the target area based on the typical graphic and textual style corresponding to that attribute. It should be understood that graphic and textual style can specifically manifest as fixed patterns in layout, field positions, and visual elements, thereby locating and extracting at least one key target area. It should also be understood that in this embodiment, the target area can refer to an area highly related to the graphic and textual style and core semantic information of the current financial category. For example, in the identity document category, the target area may include the name field, the ID number field, and the portrait photo area; in the asset document category, the target area may include key areas such as account balance and financial institution signatures.

[0086] After identifying the target region, the target region features and the non-target region features of the remaining parts of the financial graphic data (i.e., non-target regions) can be extracted separately. Then, a first weight coefficient is assigned to the target region features, and a second weight coefficient is assigned to the non-target region features. The two types of features are then fused through weighted calculation, and the final graphic features of the financial graphic data are generated based on the weighted calculation results.

[0087] It should be noted that in this embodiment, the first weight coefficient is usually set to be greater than the second weight coefficient in order to highlight the contribution of category-related key information in feature representation.

[0088] Through the above method, this embodiment achieves adaptive weighting of feature extraction categories, which can effectively enhance the feature representation of key areas and improve the accuracy and reliability of subsequent review and recognition processes.

[0089] In the above implementation, an optional implementation of determining the target area based on the financial category may include: obtaining a pre-trained area recognition model; the area recognition model includes multiple preset area recognition sub-models corresponding to different financial categories; inputting financial graphic data and its amount category into the area recognition model, the area recognition model calling the corresponding area recognition sub-model according to the financial type corresponding to the financial graphic data, and outputting at least one identified target area.

[0090] Specifically, a pre-trained region recognition model is obtained. This model contains multiple region recognition sub-models, each specifically trained for a pre-defined financial category, capable of recognizing key regions unique to that category. For example, the identity verification sub-model is specifically used to recognize the name field, ID number field, and facial image area; the asset verification sub-model is specifically used to recognize areas such as the amount field and institutional seals.

[0091] In the specific identification process, financial graphic data and its financial category are input into the region identification model. This model determines the financial category to which the input monetary graphic data belongs from the input data, and then automatically calls the region identification sub-model corresponding to that category. The called sub-model processes the input financial graphic data and outputs the location information of at least one identified target region.

[0092] In the above process, by calling sub-models based on financial categories, accurate region recognition can be provided for different categories of financial documents, which not only ensures recognition accuracy but also improves processing efficiency, providing a reliable foundation for subsequent feature extraction and weighted calculation.

[0093] Based on the extracted target regions, feature extraction is performed on each target region to obtain corresponding target region features. One optional implementation method for feature extraction may include: for each target region, using a preset image coding model to extract image features of the target region, obtaining multiple initial image features at different scales; performing feature fusion and scale adjustment on the initial image features to obtain image features at a preset scale; using a preset text coding model to extract text features of the target region, obtaining initial text features at a single scale; performing scale adjustment on the initial text features to obtain text features at a preset scale; and fusing the image features at the preset scale and the text features at the preset scale to obtain the target region features of the target region.

[0094] For any target region, the extraction processes of image features and text features can be performed in parallel to improve feature extraction efficiency, thereby improving the overall efficiency of material review.

[0095] During image feature extraction, a pre-defined image coding model, such as a feature pyramid model based on a convolutional neural network, can be used to process the image in the region, generating a series of initial image features with different receptive fields, thereby forming a multi-scale visual feature representation. In this embodiment, extracting features at multiple scales can simultaneously capture both local detail information and global structural information of the image, providing richer feature basis for subsequent tampering detection.

[0096] In other words, fine-grained features obtained through a smaller receptive field can effectively capture local details in an image, helping to identify subtle tampering and anomalies. Conversely, macroscopic structural features obtained through a larger receptive field can grasp the overall layout and structural relationships of an image, which can be used to analyze the logical rationality and structural integrity of the layout. Based on this, combining visual features at different scales yields feature representations of both local details and global structure, improving the accuracy and reliability of subsequent material review.

[0097] Subsequently, these multi-scale features are fused using a structural feature fusion module that employs top-down and horizontal connections, and the scale of the fused features is adjusted to output an image feature vector with a uniform preset scale.

[0098] In the text feature extraction process, the text content within the target area can first be identified using an OCR engine. Then, a pre-set text encoding model, such as a pre-trained language model based on Transformer, is used to encode the identified text, generating initial text features at a single scale. Subsequently, the scale of these text features is adjusted using structures such as fully connected layers, transforming them into a pre-set scale text feature vector that matches the dimensions of the image features.

[0099] It should be noted that, considering that the financial material review scenario mainly focuses on the macro semantic features and key data of the document, and has a low dependence on local image details, the dimensionality adjustment of text features and image features in this embodiment can be understood as dimensionality reduction of the extracted features to effectively reduce computational overhead, improve system processing efficiency, and at the same time retain the key feature information required to complete the review task.

[0100] S203. Match each graphic feature with the preset risk feature database to obtain the feature matching results of the financial materials.

[0101] In this embodiment, the risk feature library includes multiple risk feature sub-libraries corresponding to financial categories, and each risk feature sub-library stores graphic and textual features corresponding to the tampering of financial materials.

[0102] In other words, the pre-built risk feature library in this embodiment adopts a hierarchical storage structure, containing multiple risk feature sub-libraries corresponding to financial categories. Each risk feature sub-library specifically stores the graphic and textual features corresponding to known tampering samples in a specific category of financial materials. For example, the identity verification risk feature sub-library stores features of tampered materials such as forged ID cards and altered passports; the asset verification risk feature sub-library stores features of tampered bank statements and forged deposit certificates.

[0103] Risk features are identified for the graphic and textual features of various financial graphic and textual data in financial materials. One optional matching method may include: for any graphic and textual feature corresponding to any financial image data, determining the risk feature sub-library corresponding to the graphic and textual feature from the risk feature library according to the financial type corresponding to the financial image data; calculating the feature similarity between the graphic and textual feature and the pre-stored risk features in the risk feature sub-library; obtaining the feature matching result corresponding to the graphic and textual feature based on the feature similarity and a preset similarity threshold; and obtaining the feature matching result of the financial materials based on the feature matching results corresponding to each graphic and textual feature.

[0104] For any given financial graphic data point and its corresponding graphic features, a corresponding risk feature sub-library is determined from the risk feature library based on the financial category associated with those features. For example, when the financial category is identity verification, the identity verification risk feature sub-library will be selected as the matching benchmark; when the financial category is asset verification, the asset verification risk feature sub-library will be selected.

[0105] After determining the target risk feature sub-library, the feature similarity between the current image / text feature and each pre-stored risk feature in the sub-library is calculated. Optionally, cosine similarity, Euclidean distance, or other applicable similarity measurement methods can be used, and feature matching results corresponding to the image / text feature are generated based on the calculated feature similarity value and a preset similarity threshold.

[0106] Specifically, if the feature similarity value exceeds a preset threshold, it indicates that there is a risk feature in the risk feature library that matches the current image and text feature; otherwise, it indicates that there is no risk feature in the risk feature library that matches the current image and text feature.

[0107] Through the above-mentioned hierarchical matching, the potential risk characteristics in financial materials can be accurately identified, providing a reliable basis for subsequent risk assessment. Furthermore, while effectively ensuring the accuracy of identification, it also improves the overall processing efficiency of material review.

[0108] S204. Determine the review result of the financial materials based on the feature matching results.

[0109] In this embodiment, one optional implementation of determining the review result based on the feature matching result may include: if any feature matching result indicates the existence of a successfully matched risk feature, then a review result of "review failed" is generated; if all feature matching results indicate the absence of a successfully matched risk feature, then target areas with correlation relationships in each financial graphic data are obtained; based on preset correlation logic rules, logical conflicts in each target area are reviewed to obtain a logical review result; if the logical review result indicates the existence of logical conflicts, then a review result requiring manual review is generated; if the logical review result indicates the existence of no logical conflicts, then a review result of "review passed" is generated.

[0110] Specifically, after the financial graphic data completes risk feature matching, the review result can be determined progressively according to the following logic: First, all risk feature matching results are traversed. If any feature matching result indicates a match with a preset risk feature, a review failure result is directly generated, and the type and specific location of the successfully matched risk feature are marked in the result. If all risk feature matching results show no successfully matched risk features, it means that the financial graphic data has not triggered the preset risk rules. At this time, the system further extracts target areas with correlations in each financial graphic data.

[0111] Subsequently, pre-defined association logic is invoked, such as identity information consistency verification logic, income data reasonableness verification logic, and transaction entity association verification logic, to cross-validate the content of each associated target area and determine whether there is a logical conflict. For example, if the scanned ID card shows the applicant's name as "Zhang San" while the income certificate shows the applicant's name as "Li Si", then the identity information association area is determined to have a logical conflict. If the income certificate indicates a monthly income of 50,000 yuan, while the bank statement shows an average monthly deposit of only 10,000 yuan over the past 6 months, and there is no reasonable supplementary explanation of income, then the income data association area is determined to have a logical conflict.

[0112] Furthermore, if the verified representation shows the aforementioned logical conflicts, a review result requiring manual verification will be generated, along with specific information about the conflicting areas and explanations of the related logic, for reviewers to further examine. Conversely, if the verification result shows no logical conflicts between the related target areas and the financial graphic data complies with business review standards, a final review result that passes the review will be generated.

[0113] The above-mentioned review logic can achieve rapid and accurate risk interception, and also provide a backup for logical conflict scenarios through manual review, thus balancing review efficiency and accuracy and ensuring the reliability of financial business review.

[0114] Figure 3 A schematic diagram of the financial document verification device provided in this application. Figure 3 As shown, the financial material review device 30 provided in this embodiment includes:

[0115] Data acquisition module 301 is used to acquire financial materials to be reviewed; the financial materials include at least one set of financial graphic data;

[0116] The graphic feature generation module 302 is used to determine the financial category corresponding to each financial graphic data and generate the graphic features of the corresponding financial graphic data.

[0117] The feature matching module 303 is used to match each image and text feature with a preset risk feature library to obtain the feature matching result of the financial material; the feature library contains multiple risk feature sub-libraries corresponding to financial categories, and each risk feature sub-library stores the features of tampered financial images and / or the features of financial images whose image and text content do not match.

[0118] The materials review module 304 is used to determine the review result of financial materials based on the feature matching results.

[0119] In one optional implementation, the image and text feature generation module includes:

[0120] The target region determination submodule is used to determine, for any given financial graphic data, at least one target region contained in the financial graphic data according to its corresponding financial category; wherein, the target region contains semantic information that matches the financial category;

[0121] The feature extraction submodule is used to obtain the target region features corresponding to the target region, as well as the non-target region features corresponding to the non-target regions in the financial graphic data other than the target region;

[0122] The image and text feature generation submodule is used to generate image and text features corresponding to financial image and text data based on the calculation of the target region features and their corresponding first weight coefficients, and the calculation of the non-target regions and their corresponding second weight coefficients.

[0123] In one alternative implementation, the target region determination submodule includes:

[0124] The target region identification model acquisition unit is used to acquire a pre-trained region identification model; the region identification model includes multiple pre-set region identification sub-models corresponding to different financial categories;

[0125] The target region determination unit is used to input financial graphic data and its financial category into the region recognition model. The region recognition model calls the corresponding region recognition sub-model according to the financial type corresponding to the financial graphic data and outputs at least one identified target region.

[0126] In one alternative implementation, the feature extraction submodule includes:

[0127] The initial image feature acquisition unit is used to extract image features of each target region using a preset image coding model to obtain multiple initial image features at different scales.

[0128] The image feature acquisition unit is used to perform feature fusion and scale adjustment on the initial image features to obtain image features at a preset scale.

[0129] The initial text feature acquisition unit is used to extract text features of the target region using a preset text encoding model to obtain initial text features at a single scale.

[0130] The text feature determination unit is used to scale the initial text features to obtain text features at a preset scale.

[0131] The target region feature generation unit is used to fuse image features and text features of a preset scale to obtain the target region features.

[0132] In one alternative implementation, the feature matching module includes:

[0133] The risk feature sub-library determination module is used to determine the risk feature sub-library corresponding to the image and text features of any financial image data, based on the financial type corresponding to the financial image data;

[0134] The feature matching result determination submodule is used to calculate the feature similarity between the image and text features and the pre-stored risk features in the risk feature sub-library. Based on the feature similarity and the preset similarity threshold, the feature matching result corresponding to the image and text features is obtained.

[0135] The feature matching result integration submodule is used to obtain the feature matching results of financial materials based on the feature matching results corresponding to each image and text feature.

[0136] In one alternative implementation, the materials review module includes:

[0137] The first review result generation submodule is used to generate a review result of failure if any feature matching result indicates that there is a risk feature that has been successfully matched.

[0138] The target region determination submodule is used to obtain the target regions with correlation in each financial graphic data if all feature matching results indicate that there are no successfully matched risk features.

[0139] The logical conflict determination submodule is used to review the logical conflicts of each target area based on preset association logic rules and obtain the logical review results.

[0140] The second audit result generation submodule is used to generate an audit result that requires manual review if there is a logical conflict in the logical audit result representation.

[0141] The third audit result generation submodule is used to generate an audit result that has passed if there is no logical conflict in the logical audit result representation.

[0142] The financial material review device provided in the example can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0143] Figure 4 A schematic diagram of the structure of the electronic device provided in this application. See also... Figure 4 The device 400 may include one or more of the following components: processing component 402, memory 404, power supply component 404, multimedia component 408, audio component 410, input / output interface 412, sensor component 414, and communication component 416.

[0144] Processing component 402 typically controls the overall operation of device 400, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 402 may include one or more processors 420 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 402 may include one or more modules to facilitate interaction between processing component 402 and other components. For example, processing component 402 may include a multimedia module to facilitate interaction between multimedia component 408 and processing component 402.

[0145] Memory 404 is configured to store various types of data to support the operation of device 400. Examples of this data include instructions for any application or method operating on device 400, contact data, phonebook data, messages, pictures, videos, etc. Memory 404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0146] Power supply component 404 provides power to various components of device 400. Power supply component 404 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 400.

[0147] Multimedia component 408 includes a screen that provides an output interface between device 400 and the user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen may be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 408 includes a front-facing camera and / or a rear-facing camera. When device 400 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0148] Audio component 410 is configured to output and / or input audio signals. For example, audio component 410 includes a microphone (MIC) configured to receive external audio signals when device 400 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 404 or transmitted via communication component 416. In some embodiments, audio component 410 also includes a speaker for outputting audio signals.

[0149] Input / output interface 412 provides an interface between processing component 402 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, start buttons, and lock buttons.

[0150] Sensor assembly 414 includes one or more sensors for providing state assessments of various aspects of device 400. For example, sensor assembly 414 may detect the on / off state of device 400, the relative positioning of components such as the display and keypad of device 400, changes in the position of device 400 or a component of device 400, the presence or absence of user contact with device 400, the orientation or acceleration / deceleration of device 400, and temperature changes of device 400. Sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 414 may also include an optical sensor, such as a complementary metal-oxide-semiconductor (CMOS) sensor or a charge-coupled device (CCD) sensor, for use in imaging applications. In some embodiments, sensor assembly 414 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0151] Communication component 416 is configured to facilitate wired or wireless communication between device 400 and other devices. Device 400 can access wireless networks based on communication standards, such as WiFi, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 416 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 416 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wide Band (UWB), Bluetooth (BT), and other technologies.

[0152] In an exemplary embodiment, device 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processors (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0153] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, which can be executed by a processor 420 of the device 400 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0154] A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a server's processor, enable the server to perform the methods described above.

[0155] This application also provides a chip for executing instructions, which is used to execute the technical solutions in the above embodiments.

[0156] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed on a computer, cause the computer to perform the technical solutions described above.

[0157] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, it can implement the technical solutions in the above embodiments.

[0158] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0159] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

[0160] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0161] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method of auditing financial material, characterized by, The method comprises the following steps: obtaining financial materials to be audited; the financial materials include at least one piece of financial image-text data; determining the corresponding financial category of each financial image-text data, and generating image-text features corresponding to the financial image-text data; matching each image-text feature with a preset risk feature library to obtain a feature matching result of the financial materials; the risk feature library contains a plurality of risk feature sub-libraries corresponding to the financial categories, and each risk feature sub-library stores the image-text features corresponding to tampered financial materials; determine the audit result of the financial materials according to the feature matching result.

2. The method of claim 1, wherein, generate image-text features corresponding to the financial image-text data, including: for any financial image-text data, determine at least one target area contained in the financial image-text data according to the corresponding financial category; wherein the target area contains semantic information matched with the financial category; obtain the target area feature corresponding to the target area, and the non-target area feature corresponding to the non-target area in the financial image-text data except the target area; generate the image-text feature corresponding to the financial image-text data according to the calculation of the target area feature and the first weight coefficient corresponding thereto, and the calculation of the non-target area and the second weight coefficient corresponding thereto.

3. The method of claim 2, wherein, determine at least one target area contained in the financial image-text data according to the corresponding financial category, including: obtain a pre-trained region recognition model; the region recognition model includes a plurality of region recognition sub-models corresponding to a plurality of preset financial categories; input the financial image-text data and its financial category into the region recognition model, and the region recognition model calls the corresponding region recognition sub-model according to the financial type corresponding to the financial image-text data, and outputs at least one identified target area.

4. The method of claim 2, wherein, obtain the target area feature corresponding to the target area, including: for each target area, extract the image features of the target area using a preset image encoding model to obtain a plurality of initial image features of different scales; perform feature fusion and scale adjustment on the initial image features to obtain image features of a preset scale; extract the text features of the target area using a preset text encoding model to obtain initial text features of a single scale; perform scale adjustment on the initial text features to obtain text features of a preset scale; fuse the image features of the preset scale and the text features of the preset scale to obtain the target area feature of the target area.

5. The method of claim 1, wherein, match the financial type and each image-text feature with a preset risk feature library to obtain a feature matching result of the financial materials, including: for the image-text feature corresponding to any financial image data, determine the risk feature sub-library corresponding to the image-text feature from the risk feature library according to the financial type corresponding to the financial image data; calculate the feature similarity between the image-text feature and the pre-stored risk features in the risk feature sub-library, and obtain the feature matching result corresponding to the image-text feature according to the feature similarity and a preset similarity threshold; obtain the feature matching result of the financial materials according to the feature matching result corresponding to each image-text feature.

6. The method of claim 5, wherein, According to the feature matching result, a review result of the financial material is determined, including: If any of the feature matching results indicates that there is a matching successful risk feature, a review result of not passing is generated; If all of the feature matching results indicate that there is no matching successful risk feature, target regions with an association relationship in each of the financial graphic and text data are obtained; Based on a preset association logic rule, a logic conflict of each of the target regions is reviewed to obtain a logic review result; If the logic review result represents that there is a logic conflict, a review result that needs manual review is generated; If the logic review result represents that there is no logic conflict, a review result of passing is generated.

7. A financial material auditing apparatus characterized by comprising: Comprising: A data acquisition module is configured to acquire financial materials to be reviewed; the financial materials include at least one financial graphic and text data; A graphic and text feature generation module is configured to determine a financial category corresponding to each of the financial graphic and text data, and generate graphic and text features corresponding to the financial graphic and text data; A feature matching module is configured to match each of the graphic and text features with a preset risk feature library to obtain a feature matching result of the financial material; the feature library includes a plurality of risk feature sub-libraries corresponding to the financial category, and each of the risk feature sub-libraries stores features of tampered financial images and / or features of financial images with unmatched graphic and text content; A material review module is configured to determine a review result of the financial material according to the feature matching result.

8. An electronic device, comprising: Comprising: A memory and a processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method of any one of claims 1-6.

10. A computer program product, characterised in that, A computer program is included, and the computer program is executed by the processor to implement the method of any one of claims 1-6.

Images