Multi-modal financial data self-reconciliation method, system and electronic device

Abstract

The application relates to the technical field of artificial intelligence, and provides a multi-modal financial data self-checking method, system and electronic equipment, the method comprising the following steps: acquiring multiple heterogeneous financial data from different data sources; respectively inputting the heterogeneous financial data into a pre-trained semantic alignment model to obtain feature vectors matched with the heterogeneous financial data output by the semantic alignment model, wherein the feature vectors are obtained by mapping the heterogeneous financial data to a unified semantic embedding space; and performing matching operation and processing on the heterogeneous financial data based on the feature vectors by multiple checking intelligent agents in a trusted privacy computing space to obtain checking matching results between different heterogeneous financial data. The application realizes efficient and accurate checking of multi-modal financial data, improves the processing capacity of heterogeneous data, and enhances the flexibility and accuracy of intelligent matching.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a multimodal financial data autonomous reconciliation method, system, and electronic device. Background Technology

[0002] Financial reconciliation is a core business process for ensuring the safety of funds and the accuracy of accounts for enterprises and financial institutions. As businesses expand and transaction scenarios become more complex, the sources of data required for reconciliation are becoming increasingly diverse, significantly increasing data complexity. Current mainstream reconciliation systems primarily rely on structured data, such as database tables and standard messages, for processing. Their core methods are typically based on pre-defined deterministic rules, such as comparing the "exactly the same amount and the same date" between the two parties to determine if a transaction matches.

[0003] However, this rigid matching logic faces numerous technical limitations in practical applications. First, traditional systems lack effective processing capabilities for unstructured data, such as PDF invoices and scanned paper receipts, and semi-structured data, such as detailed SWIFT MT messages and JSON-formatted payment gateway responses. In actual business operations, a large number of transaction vouchers exist in unstructured or semi-structured forms, which existing systems cannot directly parse and utilize. They still rely on manual optical character recognition (OCR) to extract key information and perform semantic verification, resulting in inefficient reconciliation processes and unavoidable human error. Second, rigid rule-based matching methods are ill-suited to the complexities of real-world transactions. For example, an online payment may correspond to the combined payment of multiple orders, or an order may be split due to partial refunds. Exchange rate fluctuations in cross-border transactions, differences in fee deduction methods among different banks, and abbreviations or variations of merchant names on the payment channel side can all cause strict rule matching to fail, generating numerous "false positive" or "false negative" errors requiring manual review.

[0004] Therefore, improving the reconciliation system's ability to process heterogeneous data and enhancing the flexibility and accuracy of intelligent matching are pressing technical challenges that need to be addressed in the fintech field. Summary of the Invention

[0005] This invention provides a method, system, and electronic device for autonomous reconciliation of multimodal financial data, which enables efficient and accurate reconciliation of multimodal financial data, improves the processing capability of heterogeneous data, and enhances the flexibility and accuracy of intelligent matching.

[0006] This invention provides a multimodal financial data autonomous reconciliation method, the method comprising: acquiring multiple heterogeneous financial data from different data sources, wherein the heterogeneous financial data includes structured transaction records, semi-structured messages, and unstructured bills; inputting the heterogeneous financial data into a pre-trained semantic alignment model to obtain feature vectors output by the semantic alignment model that match the heterogeneous financial data, wherein the feature vectors are obtained by mapping the heterogeneous financial data to a unified semantic embedding space; and, within a trusted privacy computing space, performing matching operations on the heterogeneous financial data based on the feature vectors by multiple reconciliation agents to obtain reconciliation matching results between different heterogeneous financial data.

[0007] According to a multimodal financial data autonomous reconciliation method provided by the present invention, the step of performing matching operations on heterogeneous financial data based on the feature vector within a trusted privacy computing space by multiple reconciliation agents to obtain reconciliation matching results between different heterogeneous financial data includes: performing matching operations on heterogeneous financial data based on a secure federated consensus protocol and the feature vector within a trusted privacy computing space to obtain reconciliation matching results between different heterogeneous financial data, wherein the reconciliation matching results are generated without disclosing the heterogeneous financial data.

[0008] According to a multimodal financial data autonomous reconciliation method provided by the present invention, the step of performing matching operations on heterogeneous financial data based on the feature vector within a trusted privacy computing space by multiple reconciliation agents to obtain reconciliation matching results between different heterogeneous financial data includes: within the trusted privacy computing space, performing matching operations on the heterogeneous financial data based on the feature vector by each of the reconciliation agents to obtain candidate reconciliation matching results corresponding to each of the reconciliation agents; determining the weight of each of the reconciliation agents and determining the agent confidence level of each of the reconciliation agents, wherein the agent confidence level is used to characterize the confidence level of the reconciliation matching results self-assessed by the reconciliation agents; and determining the reconciliation matching results between different heterogeneous financial data based on the candidate reconciliation matching results corresponding to each of the reconciliation agents, the weight of each of the reconciliation agents, and the agent confidence level of each of the reconciliation agents.

[0009] According to a multimodal financial data autonomous reconciliation method provided by the present invention, after obtaining the reconciliation matching results between different heterogeneous financial data, the method further includes: determining unmatched discrepancies based on the reconciliation matching results, wherein the unmatched discrepancies are heterogeneous financial data that cannot be reconciled and matched with other heterogeneous financial data; obtaining a transaction causal graph for the unmatched discrepancies, wherein the transaction causal graph uses the influencing factors affecting the transaction posting results of the unmatched discrepancies as nodes in the graph, and the causal relationships between the nodes as directed edges in the graph; performing counterfactual reasoning based on the transaction causal graph to quantify the contribution of each influencing factor to the reconciliation amount discrepancy; and determining a reconciliation discrepancy root cause analysis report based on the contribution of each influencing factor to the reconciliation amount discrepancy.

[0010] According to a multimodal financial data autonomous reconciliation method provided by the present invention, the semantic alignment model is trained in the following manner: obtaining training samples, wherein the training samples include seed samples, positive samples from the same transaction as the seed samples, and negative samples from different transactions as the seed samples; constructing a loss function based on the cosine similarity between the seed samples and the positive samples, and the cosine similarity between the seed samples and the negative samples; and training the semantic alignment model based on the loss function and the training samples to obtain a trained semantic alignment model.

[0011] According to a multimodal financial data autonomous reconciliation method provided by the present invention, before constructing a loss function based on the cosine similarity between the seed sample and the positive sample, and the cosine similarity between the seed sample and the negative sample, the method further includes: setting a weighting coefficient for each negative sample, wherein the weighting coefficient is used to characterize the difficulty of distinguishing the negative sample from the seed sample; the construction of the loss function based on the cosine similarity between the seed sample and the positive sample, and the cosine similarity between the seed sample and the negative sample includes: constructing a loss function based on the cosine similarity between the seed sample and the positive sample, the cosine similarity between the seed sample and the negative sample, and the weighting coefficient of the negative sample.

[0012] According to a multimodal financial data self-reconciliation method provided by the present invention, the weighting coefficient is determined in the following manner: obtaining the first transaction amount and first transaction time of the seed sample, and obtaining the second transaction amount and second transaction time of the negative sample; determining the transaction amount sensitivity bandwidth and the transaction time sensitivity bandwidth respectively; and determining the weighting coefficient based on the first transaction amount, the first transaction time, the second transaction amount, the second transaction time, the transaction amount sensitivity bandwidth, and the transaction time sensitivity bandwidth.

[0013] This invention also provides a multimodal financial data autonomous reconciliation system, the system comprising: an acquisition module for acquiring multiple heterogeneous financial data from different data sources, wherein the heterogeneous financial data includes structured transaction records, semi-structured messages, and unstructured bill files; a processing module for inputting the heterogeneous financial data into a pre-trained semantic alignment model to obtain feature vectors output by the semantic alignment model that match the heterogeneous financial data, wherein the feature vectors are obtained by mapping the heterogeneous financial data to a unified semantic embedding space; and a reconciliation module for performing matching operations on the heterogeneous financial data based on the feature vectors within a trusted privacy computing space through multiple reconciliation agents to obtain reconciliation matching results between different heterogeneous financial data.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multimodal financial data autonomous reconciliation method as described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multimodal financial data autonomous reconciliation method as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the multimodal financial data autonomous reconciliation method as described above.

[0017] This invention provides a method, system, and electronic device for autonomous reconciliation of multimodal financial data. The method includes: acquiring multiple heterogeneous financial data from different data sources; inputting the heterogeneous financial data into a pre-trained semantic alignment model to obtain feature vectors output by the semantic alignment model that match the heterogeneous financial data, wherein the feature vectors are obtained by mapping the heterogeneous financial data to a unified semantic embedding space; and, within a trusted privacy computing space, performing matching operations on the heterogeneous financial data based on the feature vectors through multiple reconciliation agents to obtain reconciliation matching results between different heterogeneous financial data. This achieves efficient and accurate reconciliation of multimodal financial data, improves the processing capability of heterogeneous data, and enhances the flexibility and accuracy of intelligent matching. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts illustrating the multimodal financial data self-reconciliation method provided by this invention.

[0020] Figure 2 This invention provides a schematic diagram of the process by which multiple reconciliation agents perform matching operations on heterogeneous financial data based on the feature vector within a trusted privacy computing space to obtain reconciliation matching results between different heterogeneous financial data.

[0021] Figure 3 This is the second flowchart of the multimodal financial data self-reconciliation method provided by the present invention.

[0022] Figure 4 This is a schematic diagram of the structure of the multimodal financial data self-reconciliation system provided by the present invention.

[0023] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] This invention provides a multimodal financial data autonomous reconciliation method, which can be applied to scenarios requiring reconciliation, such as financial institutions, corporate financial systems, or third-party payment platforms. It aims to automate the processing of heterogeneous financial data from different data sources and achieve efficient and accurate cross-institutional reconciliation while ensuring data privacy.

[0026] Figure 1 This is one of the flowcharts illustrating the multimodal financial data self-reconciliation method provided by this invention.

[0027] The following will combine Figure 1 The process of the multimodal financial data autonomous reconciliation method provided by this invention is described.

[0028] In an exemplary embodiment of the present invention, combined with Figure 1 As can be seen, the multimodal financial data self-reconciliation method may include steps 110 to 130, and each step will be described below.

[0029] In step 110, multiple heterogeneous financial data from different data sources are acquired, including structured transaction records, semi-structured messages, and unstructured bills.

[0030] In one embodiment, various interface adapters can be used to connect to different data sources to obtain the raw financial data to be reconciled, i.e., heterogeneous financial data. These data sources include, but are not limited to, internal enterprise systems, banks and payment institutions, and unstructured documents. For internal enterprise systems, such as ERP systems (SAP, Oracle, etc.), financial management systems, and inventory management systems, structured transaction records are obtained, such as accounting entries, bank journals, and accounts receivable and payable details. For banks and payment institutions, semi-structured message data is obtained through direct bank-enterprise connections, third-party payment gateway APIs, etc., such as SWIFT MT940 / 950 account statements, UnionPay transaction details, and JSON response data from Alipay / WeChat Pay. For unstructured documents, unstructured files received through scanners, faxes, email attachments, etc., such as PDF invoices, JPG / PNG check receipts, and scanned copies of contracts, are obtained.

[0031] Heterogeneous financial data can include structured transaction records (such as transaction logs in database tables and CSV files), semi-structured messages (such as detailed data in XML, JSON, and SWIFT formats), and unstructured document files (such as PDF invoices and scanned receipts). The system performs preliminary cleaning on the acquired data to remove obvious noise and de-identifies or encrypts sensitive information (such as account numbers and account names) in preparation for subsequent processing.

[0032] In step 120, heterogeneous financial data are input into a pre-trained semantic alignment model to obtain feature vectors that match the heterogeneous financial data output by the semantic alignment model. The feature vectors are obtained by mapping the heterogeneous financial data to a unified semantic embedding space.

[0033] In one embodiment, a unified semantic alignment model can be used to map financial data from different modalities into the same semantic space, allowing invoices, transaction records, and messages from the same real transaction to be close to each other within that space. Specifically, the pre-trained semantic alignment model can employ a deep neural network architecture based on contrastive learning.

[0034] During the training phase, the semantic alignment model can be optimized using a precision-weighted contrastive loss function specific to financial scenarios. This loss function, based on the traditional InfoNCE loss, introduces hard negative sample penalty weights that are sensitive to amount and time, enabling the model to more precisely distinguish between samples with similar amounts but different transactions and samples with similar times but different business operations. After training, the model can convert any input heterogeneous financial data into fixed-length feature vectors, and these vectors are aligned in a unified semantic embedding space.

[0035] In application, heterogeneous financial data, such as invoice images, bank transaction records, and payment messages, can be input into the model, which then outputs corresponding feature vectors. These vectors have eliminated the differences between the original modalities, unifying them within the same semantic space, and can be directly used for similarity calculation.

[0036] In step 130, within the trusted privacy computing space, multiple reconciliation agents perform matching operations on heterogeneous financial data based on feature vectors to obtain reconciliation matching results between different heterogeneous financial data.

[0037] In one embodiment, within a trusted privacy computing space, the final reconciliation matching result between different heterogeneous financial data can be determined by combining the matching operation results of multiple reconciliation agents based on feature vectors on heterogeneous financial data.

[0038] This invention provides a multimodal financial data autonomous reconciliation method. The method includes: acquiring multiple heterogeneous financial data from different data sources; inputting the heterogeneous financial data into a pre-trained semantic alignment model to obtain feature vectors output by the semantic alignment model that match the heterogeneous financial data, wherein the feature vectors are obtained by mapping the heterogeneous financial data to a unified semantic embedding space; and, within a trusted privacy computing space, performing matching operations on the heterogeneous financial data based on the feature vectors through multiple reconciliation agents to obtain reconciliation matching results between different heterogeneous financial data. This method achieves efficient and accurate reconciliation of multimodal financial data, improves the processing capability of heterogeneous data, and enhances the flexibility and accuracy of intelligent matching.

[0039] In yet another exemplary embodiment of the present invention, within a trusted privacy computing space, multiple reconciliation agents perform matching operations on the heterogeneous financial data based on the feature vector to obtain reconciliation matching results between different heterogeneous financial data. This can be achieved in the following manner: Within a trusted privacy computing space, multiple reconciliation agents perform matching operations on heterogeneous financial data based on a secure federated consensus protocol and feature vectors to obtain reconciliation matching results between different heterogeneous financial data. These reconciliation matching results are generated without disclosing the heterogeneous financial data.

[0040] In one embodiment, to protect the data privacy of all parties involved (especially when cross-institutional reconciliation is involved), matching operations can be performed within a trusted privacy computing space. This space can be a protected, isolated execution space based on a hardware trusted execution environment (TEE, such as Intel SGX, AMD SEV), such as an enclave, or a cryptographic computing domain based on a multi-party secure computation (MPC) protocol. Each party encrypts the generated feature vector (rather than the original data) and inputs it into this space.

[0041] In another embodiment, multiple reconciliation agents with different responsibilities can be deployed within a trusted space. These agents operate in software form and collaborate to complete matching tasks through a Secure Federation Consensus Protocol (SFAC). The reconciliation agents can include matching agents, auditing agents, and reconnaissance agents. The matching agent is responsible for calculating the similarity (e.g., cosine similarity) between feature vectors from different sources. For internal bank statements and invoices, the matching agent calculates directly; for cross-institutional reconciliations, the matching agent calculates the inner product of encrypted vectors in the ciphertext domain and obtains the similarity after decryption using a trusted oracle. When the similarity exceeds a preset threshold, a candidate matching pair is generated. The auditing agent simulates human review rules to perform secondary verification on the candidate matching pairs. For example, it verifies whether the transaction time is on a weekday and whether the amount conforms to business routines. If the auditing agent detects anomalies (e.g., large transfers on weekends), it can lower the confidence level of the match or send it back for re-examination. The reconnaissance agent focuses on unmatched transactions, performing multi-hop tracing, such as checking for related transactions involving merged or split payments, and preparing data for subsequent attribution analysis.

[0042] In one embodiment, multiple agents can vote on the matching results through a consensus mechanism to ultimately reach a consistent reconciliation result. For example, for a bank statement and an invoice matching, the matching agent gives a "match" (similarity 0.95), the auditing agent considers it "questionable" (the transaction occurred on a non-working day), and the investigation agent discovers that the invoice corresponds to the combined payment of multiple orders. The system then considers the opinions of all three and may mark the transaction as "requiring manual review" or make a final judgment based on a preset weighting strategy.

[0043] All raw data never leaves the domain; only matching results and necessary metadata are output, thus achieving privacy-preserving reconciliation where "data is available but not visible."

[0044] Figure 2 This invention provides a schematic diagram of the process by which multiple reconciliation agents perform matching operations on heterogeneous financial data based on the feature vector within a trusted privacy computing space to obtain reconciliation matching results between different heterogeneous financial data.

[0045] The following will combine Figure 2 The present invention describes the process by which multiple reconciliation agents perform matching operations on heterogeneous financial data based on the feature vector within a trusted privacy computing space to obtain reconciliation matching results between different heterogeneous financial data.

[0046] In an exemplary embodiment of the present invention, combined with Figure 2 As can be seen, within the trusted privacy computing space, the process of obtaining reconciliation matching results between different heterogeneous financial data by multiple reconciliation agents performing matching operations on the heterogeneous financial data based on the feature vector can include steps 210 to 230, which will be described in detail below.

[0047] In step 210, within the trusted privacy computing space, each reconciliation agent performs matching operations on heterogeneous financial data based on feature vectors to obtain candidate reconciliation matching results corresponding to each reconciliation agent.

[0048] In one embodiment, multiple reconciliation agents with different functions and perspectives are pre-deployed within a trusted privacy computing space. These agents share feature vectors from different data sources. Each agent independently processes these feature vectors according to its built-in algorithms or rules and outputs its own candidate reconciliation matching results.

[0049] The matching agent is primarily responsible for quantitative calculations. It calculates the cosine similarity between feature vectors of different heterogeneous financial data. Based on the similarity score, the matching agent outputs a preliminary judgment result S1.

[0050] The audit agent is primarily responsible for verifying business compliance. It does not directly use feature vectors, but instead accesses metadata associated with them (such as transaction timestamps, the counterparty's location, and transaction channels; this metadata is already attached when entering the trusted space). The audit agent applies pre-defined audit rules, such as "transactions should not occur in the early morning of non-working days" and "the amount of a single transaction should not exceed the company's daily limit." If the original data corresponding to a feature vector violates a rule, the audit agent's output judgment S2 may be "mismatch" or "questionable," accompanied by a low confidence level.

[0051] The reconnaissance agent is primarily responsible for pattern matching based on historical data. It maintains a database of historical transaction features stored in an encrypted manner within a trusted space. When processing a current transaction, it retrieves similar feature vector combinations from the historical database and calculates the historical reconciliation success rate of these combinations. If similar feature vector combinations have frequently failed to reconcile historically, the historical experience agent's output judgment S3 may be "mismatch," otherwise it will be "match."

[0052] In this way, each reconciliation agent independently generates a candidate reconciliation matching result based on its own professional perspective.

[0053] In step 220, the weight of each reconciliation agent is determined, and the agent confidence level of each reconciliation agent is determined, wherein the agent confidence level is used to characterize the confidence level of the reconciliation matching results self-assessed by the reconciliation agent.

[0054] In step 230, the reconciliation matching results between different heterogeneous financial data are determined based on the candidate reconciliation matching results corresponding to each reconciliation agent, the weight of each reconciliation agent, and the agent confidence of each reconciliation agent.

[0055] In one embodiment, the weights and agent confidence levels of each reconciliation agent can be determined. Based on the candidate reconciliation matching results corresponding to each reconciliation agent, the weights of each reconciliation agent, and the agent confidence levels of each reconciliation agent, the reconciliation matching results between different heterogeneous financial data can be determined. The reconciliation matching results between different heterogeneous financial data can be achieved using formula (1): (1) The formula includes the judgment results of each agent m. (1 for a match, 0 for no match), meaning the candidate reconciliation matching results are weighted by function. And agent confidence The weighted average is then subtracted from the consistency penalty term for different modal representations. This represents the variance of the set of embedding vectors for the transaction across different modalities, used to measure data consistency.

[0056] in, This represents the total number of agents involved in the decision-making process. This represents the role weight of the m-th agent, also known as the functional weight. This represents the binary judgment of the m-th agent (1 indicates a match, 0 indicates a mismatch), which is the judgment result mentioned above; Also known as agent confidence, it represents the matching confidence of the m-th agent (such as the probability from the LLM output). This represents the variance penalty coefficient, used to control the intensity of the penalty when modes are inconsistent; Represents the embedding vector of the transaction in different modalities. The variance, which is the variance of the set of embedding vectors of the transaction in each modality mentioned above, is used to measure the consistency of multimodal information.

[0057] It comprehensively reflects the degree of consensus and modal consistency among multiple agents and can serve as the final reconciliation and matching result; the numerator aggregates the weighted confidence of each agent, and the denominator is the total weight; the larger the variance term, the more serious the contradiction of multimodal information, and the lower the confidence of automatic matching needs to be.

[0058] By introducing a variance penalty mechanism, when there are significant differences between different modalities, the system automatically reduces the probability of automatic approval and triggers manual review, thereby avoiding misjudgment by a single model. This achieves an internal check and balance similar to "bookkeeping-review," significantly reducing the false alarm rate and improving the reliability of reconciliation results through multi-agent collaboration.

[0059] Figure 3 This is the second flowchart of the multimodal financial data self-reconciliation method provided by the present invention.

[0060] The following will combine Figure 3 The process of another multimodal financial data autonomous reconciliation method provided by the present invention is described.

[0061] In an exemplary embodiment of the present invention, combined with Figure 3 As can be seen, after obtaining the reconciliation and matching results between different heterogeneous financial data, the multimodal financial data autonomous reconciliation method can also include steps 310 to 340, which will be introduced below.

[0062] In step 310, based on the reconciliation matching results, unmatched discrepancies are identified, wherein unmatched discrepancies are heterogeneous financial data that cannot be reconciled and matched with other heterogeneous financial data.

[0063] In one embodiment, after obtaining the reconciliation matching results, a complete set of reconciliation matching results can be obtained. This set contains two types of data: successfully matched reconciliation items and unmatched reconciliation items. During application, by traversing the reconciliation matching results, all heterogeneous financial data that failed to match can be filtered out and marked as "unmatched discrepancies". These unmatched discrepancies will be sent to the subsequent attribution analysis process to explore their root causes.

[0064] In step 320, a transaction causal graph for unmatched discrepancies is obtained. The transaction causal graph uses the factors affecting the transaction posting results of unmatched discrepancies as nodes in the graph, and the causal relationships between nodes as directed edges in the graph.

[0065] In one embodiment, once a mismatch is identified, the system triggers an attribution analysis process. A transaction causal graph can be constructed regarding this mismatch. A transaction causal graph is a directed acyclic graph where nodes represent various factors that may affect the final settlement result of a transaction. These factors may include the original transaction amount, actual settlement amount, bank fees, withheld taxes, exchange rate conversion, transaction timestamp, payment channel type, merchant name, transaction serial number, bank fee policies, holiday postponement rules, cross-border settlement time differences, etc. Directed edges in the graph represent causal relationships between nodes.

[0066] In step 330, counterfactual reasoning is performed based on the transaction causal graph to quantify the contribution of each influencing factor to the difference in reconciliation amount.

[0067] In step 340, a root cause analysis report on reconciliation discrepancies is determined based on the contribution of each influencing factor to the reconciliation amount discrepancy.

[0068] In one embodiment, after obtaining the transaction causal graph, counterfactual reasoning techniques from causal inference can be used to quantify the contribution of each factor to the final amount difference. The core idea of ​​counterfactual reasoning is to pose a hypothetical question: "What would happen if a certain factor did not exist (or was set to a specific value)?" By traversing all possible factors in the causal graph (transaction fees, taxes, exchange rates, time differences, etc.), the system obtains a set of quantified attribution contribution scores.

[0069] Furthermore, the calculated quantitative attribution results are summarized and sorted to generate an interpretable root cause analysis report of reconciliation discrepancies.

[0070] In another embodiment, the reconnaissance agent can construct a transaction causal graph based on historical data, using factors that may affect the final amount received, such as the original order amount, exchange rate, and bank fee strategy, as nodes to establish potential causal relationships. Then, for each assumed factor X, its contribution to the difference Y is calculated. Attribution calculation based on counterfactual reasoning is employed.

[0071] Among them, the single-factor attribution measure, that is, the contribution of the influencing factors to the difference in reconciliation amount, can be achieved by formula (2): (2) This formula measures the reconciliation difference Y (e.g., when factor X is removed (set to 0 or assumed to have no effect)). The change in (X). The absolute value of the difference represents the degree of influence of factor X on the difference.

[0072] Where Y represents the observed difference in amount; The difference between the two is the predicted difference under the counterfactual scenario (assuming X does not exist); the difference quantifies the contribution of factor X.

[0073] By calculating the attribution values ​​of each potential factor, the system can automatically identify the most likely root cause of the discrepancy. It provides interpretable root cause analysis results, helping decision-makers quickly pinpoint the source of the problem and significantly reducing the time and cost of manual investigation.

[0074] The estimation of counterfactual account amounts can be achieved using formula (3): (3) Where X represents the assumed influencing factors (such as handling fees, taxes, exchange rates, etc.); Y represents the actual reconciliation difference (also known as the observed amount difference), that is, the difference between the amount credited to the bank and the amount recorded in the ERP. This represents the amount of difference predicted under a counterfactual scenario (factor X is invalid) (also known as the difference value predicted under a counterfactual scenario (assuming X does not exist)). This represents a structured causal model function used to compute the numerical part that depends on the causal parent node; λ represents the corrected value output by the large language model after performing semantic reasoning on factor X based on the context; λ represents the weighting coefficient, used to fuse the results of structured prediction and semantic reasoning.

[0075] This formula is used to calculate the difference value under counterfactual circumstances. .in This indicates numerical extrapolation based on a structured causal model (SCM) (e.g., substituting factors such as fee rates, tax rates, and transaction fees), while This represents the correction amount inferred from the unstructured context (such as bank statement remarks) by the large language model, which is the correction value output by the large language model after performing semantic reasoning on factor X based on the context. λ is the fusion coefficient of the two, also known as the weighting coefficient.

[0076] Responsible for quantitatively calculating the basic prediction when considering parent node factors. Then, textual clues are used to make semantic corrections to the results; The formula balances the weights of both factors. By combining numerical models and semantic reasoning, it can more accurately simulate incoming amounts in counterfactual scenarios, and is particularly suitable for complex situations such as fee-based models and shared fees.

[0077] In yet another exemplary embodiment of the present invention, the semantic alignment model can be trained in the following manner: Obtain training samples, which include seed samples, positive samples from the same transaction as the seed samples, and negative samples from different transactions than the seed samples; A loss function is constructed based on the cosine similarity between the seed sample and the positive sample, and the cosine similarity between the seed sample and the negative sample. Based on the loss function and the training samples, the semantic alignment model is trained to obtain the trained semantic alignment model.

[0078] In an exemplary embodiment of the present invention, to train the semantic alignment model, a high-quality training dataset in the financial domain is first required. The construction of this dataset follows a contrastive learning paradigm, where each training sample unit consists of three parts: a seed sample, a positive sample, and a negative sample. A training batch contains N sets of triplet sample pairs (seed sample, positive sample, negative sample). The seed sample, serving as the anchor point for contrastive learning, can be any modality of financial data unit. For example, it could be a PDF invoice image, a JSON payment gateway callback record, or a CSV bank statement.

[0079] Positive samples can be other modal data from the same real transaction as the seed sample. Positive samples are the targets that the semantic alignment model hopes to approximate. For example, if the seed sample is an invoice image, its positive sample could be the accounts payable transaction record for that transaction in the ERP system. If the seed sample is a bank receipt, its positive sample could be the JSON message of the original order that triggered the receipt. The construction of positive samples relies on the relationships within the business systems, such as cross-system associations through key fields like transaction number, order number, and invoice number.

[0080] Negative samples, along with seed samples, come from other modalities of data from different transactions. Negative samples are the targets that the semantic alignment model aims to push further. For example, for the seed sample of the invoice image mentioned above, its negative sample could be a bank transaction record from a completely different transaction. The construction of negative samples employs a random sampling strategy, randomly selecting other data units from the entire dataset that do not constitute the same transaction as the seed sample.

[0081] In another embodiment, a loss function can be constructed to drive model learning. The design goal of this loss function is to maximize the similarity between the seed sample and its positive samples in the embedding space, while minimizing the similarity between the seed sample and all its negative samples. Furthermore, the semantic alignment model can be trained based on the loss function and the training samples to obtain a trained semantic alignment model.

[0082] In yet another exemplary embodiment of the present invention, before constructing the loss function based on the cosine similarity between the seed sample and the positive sample, and the cosine similarity between the seed sample and the negative sample, the method further includes: A weighting coefficient is assigned to each negative sample, where the weighting coefficient is used to characterize the difficulty of distinguishing the negative sample from the seed sample; The loss function, based on the cosine similarity between the seed sample and the positive sample, and the cosine similarity between the seed sample and the negative sample, can be constructed in the following way: A loss function is constructed based on the cosine similarity between the seed sample and the positive sample, the cosine similarity between the seed sample and the negative sample, and the weighting coefficient of the negative sample.

[0083] In one embodiment, the loss function can be implemented using formula (4): (4) This loss function uses cosine similarity sim(·) as the matching metric for positive and negative samples, and adjusts the distribution smoothness using a temperature coefficient τ. Unlike the traditional InfoNCE loss, this loss introduces a weighting coefficient for each negative sample. This is used to adjust the impact. Through this formula, the model is driven during training to maximize the vector similarity of different modalities of the same transaction (such as invoice images and transaction records), while giving a stronger rejection force to transactions that are close in value and time but do not match.

[0084] in, The overall comparison loss value is represented by the numerator, which evaluates the similarity of correctly matched pairs; the denominator adds the weighted similarity sum of all other negative sample pairs. This loss aligns the multimodal features of financial transactions in the semantic space, ensuring that different representations of the same transaction have high similarity, while different transactions are distinguished. It significantly reduces false alarms from fuzzy matching during reconciliation, improves the system's ability to capture numerical precision, and is particularly effective in distinguishing between "two transactions of the same amount but different transactions on the same day."

[0085] In yet another exemplary embodiment of the present invention, the weighting coefficients can be determined in the following manner: Obtain the first transaction amount and first transaction time of the seed sample, and obtain the second transaction amount and second transaction time of the negative sample; Determine the transaction amount sensitivity bandwidth and the transaction time sensitivity bandwidth respectively; The weighting coefficients are determined based on the first transaction amount, the first transaction time, the second transaction amount, the second transaction time, the transaction amount sensitivity bandwidth, and the transaction time sensitivity bandwidth.

[0086] In one embodiment, the weighting coefficients can be implemented using formula (5): (5) This formula defines the additional penalty weight for negative samples. Also known as a weighting coefficient, it is used to more effectively exclude negative samples that are difficult to distinguish during training. The baseline weight is 1, when a negative sample... In terms of amount or time, compared with seed samples When they are very close, the exponent term makes The significant increase forces the model to learn more refined distinctions.

[0087] Reflecting negative samples With seed sample The degree of difficulty in distinguishing between them; α and β are hyperparameters used to adjust the repulsion strength when the amount and time are similar; These are the amounts of two transactions, corresponding to the first transaction amount and the second transaction amount, respectively. These are the transaction timestamps, corresponding to the first and second transaction times, respectively.

[0088] This weighting term imposes stronger constraints on negative samples that are "similar in amount and time but do not actually match" during model training, forcing the network to notice subtle numerical and contextual differences. This significantly improves the accuracy and robustness of reconciliation and reduces the number of false matches generated when relying on fuzzy amount matching.

[0089] in, This represents the total number of sample pairs in the training batch; This represents the temperature coefficient, used to scale the cosine similarity. Represents the vector cosine similarity function; , Indicates sample and Embedded representation; Indicates negative samples Embedded representation; This represents the weighting coefficient of the k-th negative sample relative to the seed sample i; This represents a coefficient used to control the intensity of monetary and time-based penalties. These represent the bandwidth for monetary sensitivity and the bandwidth for time sensitivity, respectively. These represent the amounts of transactions i and k, respectively; that is, the first transaction amount and the second transaction amount. This represents the timestamps of transactions i and k; that is, the first transaction time and the second transaction time.

[0090] As described above, this invention provides a multimodal financial data autonomous reconciliation method that realizes a complete process from data preprocessing, multimodal alignment, multi-party secure reconciliation to discrepancy attribution analysis. During implementation, each step works in concert: multimodal representation provides input for alignment, precision-weighted alignment ensures matching accuracy, the multi-agent matching and decision-making mechanism in the trusted space guarantees privacy, security, and high reliability, and causal attribution provides interpretable analytical basis for unmatched results. This tightly coordinated process has significant advantages in financial reconciliation scenarios, overcoming the shortcomings of existing technologies.

[0091] Figure 4 This is a schematic diagram of the structure of the multimodal financial data self-reconciliation system provided by the present invention.

[0092] The following will combine Figure 4 The structure of the multimodal financial data self-reconciliation system provided by this invention is described.

[0093] The multimodal financial data self-reconciliation system provided by this invention is described below. The multimodal financial data self-reconciliation system described below can be referred to in correspondence with the multimodal financial data self-reconciliation method described above.

[0094] In an exemplary embodiment of the present invention, combined with Figure 4 As can be seen, the multimodal financial data self-reconciliation system may include an acquisition module 410, a processing module 420, and a reconciliation module 430. Each module will be introduced below.

[0095] The acquisition module 410 can be configured to acquire multiple heterogeneous financial data from different data sources, wherein the heterogeneous financial data includes structured transaction records, semi-structured messages, and unstructured bill files; The processing module 420 can be configured to input the heterogeneous financial data into a pre-trained semantic alignment model to obtain a feature vector output by the semantic alignment model that matches the heterogeneous financial data, wherein the feature vector is obtained by mapping the heterogeneous financial data to a unified semantic embedding space. The reconciliation module 430 can be configured to perform matching operations on the heterogeneous financial data based on the feature vector within a trusted privacy computing space through multiple reconciliation agents, thereby obtaining reconciliation matching results between different heterogeneous financial data.

[0096] In an exemplary embodiment of the present invention, the reconciliation module 430 can achieve the following: within a trusted privacy computing space, multiple reconciliation agents perform matching operations on the heterogeneous financial data based on the feature vector to obtain reconciliation matching results between different heterogeneous financial data: Within a trusted privacy computing space, multiple reconciliation agents perform matching operations on the heterogeneous financial data based on a secure federated consensus protocol and the feature vectors to obtain reconciliation matching results between different heterogeneous financial data. These reconciliation matching results are generated without disclosing the heterogeneous financial data.

[0097] In an exemplary embodiment of the present invention, the reconciliation module 430 can achieve the following: within a trusted privacy computing space, multiple reconciliation agents perform matching operations on the heterogeneous financial data based on the feature vector to obtain reconciliation matching results between different heterogeneous financial data: Within the trusted privacy computing space, each of the aforementioned reconciliation agents performs matching operations on the heterogeneous financial data based on the feature vectors to obtain candidate reconciliation matching results corresponding to each of the aforementioned reconciliation agents. The weights of each reconciliation agent are determined, and the agent confidence of each reconciliation agent is determined, wherein the agent confidence is used to characterize the confidence of the reconciliation matching results self-assessed by the reconciliation agent. Based on the candidate reconciliation matching results corresponding to each of the reconciliation agents, the weight of each of the reconciliation agents, and the agent confidence of each of the reconciliation agents, the reconciliation matching results between different heterogeneous financial data are determined.

[0098] In an exemplary embodiment of the present invention, the reconciliation module 430 may further be configured to: Based on the reconciliation results, unmatched discrepancies are identified, wherein the unmatched discrepancies are heterogeneous financial data that cannot be reconciled and matched with other heterogeneous financial data. Obtain a transaction causal graph for the unmatched discrepancies, wherein the transaction causal graph uses the factors affecting the transaction posting results of the unmatched discrepancies as nodes in the graph, and the causal relationships between the nodes as directed edges in the graph; Counterfactual reasoning is performed based on the aforementioned transaction causal graph to quantify the contribution of each influencing factor to the difference in reconciliation amount; Based on the contribution of each influencing factor to the discrepancy in reconciliation amounts, a root cause analysis report on reconciliation discrepancies is determined.

[0099] In an exemplary embodiment of the present invention, the processing module 420 may train the semantic alignment model in the following manner: Obtain training samples, wherein the training samples include seed samples, positive samples from the same transaction as the seed samples, and negative samples from different transactions than the seed samples; A loss function is constructed based on the cosine similarity between the seed sample and the positive sample, and the cosine similarity between the seed sample and the negative sample; Based on the loss function and the training samples, the semantic alignment model is trained to obtain a trained semantic alignment model.

[0100] In an exemplary embodiment of the present invention, the processing module 420 may further be configured to: A weighting coefficient is assigned to each negative sample, wherein the weighting coefficient is used to characterize the difficulty of distinguishing the negative sample from the seed sample; The processing module 420 can construct a loss function based on the cosine similarity between the seed sample and the positive sample, and the cosine similarity between the seed sample and the negative sample, in the following manner: A loss function is constructed based on the cosine similarity between the seed sample and the positive sample, the cosine similarity between the seed sample and the negative sample, and the weighting coefficient of the negative sample.

[0101] In an exemplary embodiment of the present invention, the processing module 420 may determine the weighting coefficients in the following manner: Obtain the first transaction amount and first transaction time of the seed sample, and obtain the second transaction amount and second transaction time of the negative sample; Determine the transaction amount sensitivity bandwidth and the transaction time sensitivity bandwidth respectively; The weighting coefficient is determined based on the first transaction amount, the first transaction time, the second transaction amount, the second transaction time, the transaction amount sensitivity bandwidth, and the transaction time sensitivity bandwidth.

[0102] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a multimodal financial data autonomous reconciliation method. This method includes: acquiring multiple heterogeneous financial data from different data sources, wherein the heterogeneous financial data includes structured transaction records, semi-structured messages, and unstructured bills; inputting the heterogeneous financial data into a pre-trained semantic alignment model to obtain feature vectors output by the semantic alignment model that match the heterogeneous financial data, wherein the feature vectors are obtained by mapping the heterogeneous financial data to a unified semantic embedding space; and within a trusted privacy computing space, performing matching operations on the heterogeneous financial data based on the feature vectors through multiple reconciliation agents to obtain reconciliation matching results between different heterogeneous financial data.

[0103] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0104] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multimodal financial data autonomous reconciliation method provided by the above methods. The method includes: acquiring multiple heterogeneous financial data from different data sources, wherein the heterogeneous financial data includes structured transaction records, semi-structured messages, and unstructured bills; inputting the heterogeneous financial data into a pre-trained semantic alignment model to obtain feature vectors output by the semantic alignment model that match the heterogeneous financial data, wherein the feature vectors are obtained by mapping the heterogeneous financial data to a unified semantic embedding space; and, within a trusted privacy computing space, performing matching operations on the heterogeneous financial data based on the feature vectors by multiple reconciliation agents to obtain reconciliation matching results between different heterogeneous financial data.

[0105] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the multimodal financial data autonomous reconciliation method provided by the above methods. The method includes: acquiring multiple heterogeneous financial data from different data sources, wherein the heterogeneous financial data includes structured transaction records, semi-structured messages, and unstructured bills; inputting the heterogeneous financial data into a pre-trained semantic alignment model to obtain feature vectors output by the semantic alignment model that match the heterogeneous financial data, wherein the feature vectors are obtained by mapping the heterogeneous financial data to a unified semantic embedding space; and, within a trusted privacy computing space, performing matching operations on the heterogeneous financial data based on the feature vectors by multiple reconciliation agents to obtain reconciliation matching results between different heterogeneous financial data.

[0106] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0107] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multimodal financial data self-reconciliation method, characterized in that, The method includes: Acquire multiple heterogeneous financial data from different data sources, wherein the heterogeneous financial data includes structured transaction records, semi-structured messages, and unstructured bill files; The heterogeneous financial data are input into a pre-trained semantic alignment model to obtain a feature vector output by the semantic alignment model that matches the heterogeneous financial data. The feature vector is obtained by mapping the heterogeneous financial data to a unified semantic embedding space. Within a trusted privacy computing space, multiple reconciliation agents perform matching operations on the heterogeneous financial data based on the feature vectors to obtain reconciliation matching results between different heterogeneous financial data.

2. The multimodal financial data self-reconciliation method according to claim 1, characterized in that, Within a trusted privacy computing space, multiple reconciliation agents perform matching operations on the heterogeneous financial data based on the feature vectors to obtain reconciliation matching results between different heterogeneous financial data, including: Within a trusted privacy computing space, multiple reconciliation agents perform matching operations on the heterogeneous financial data based on a secure federated consensus protocol and the feature vectors to obtain reconciliation matching results between different heterogeneous financial data. These reconciliation matching results are generated without disclosing the heterogeneous financial data.

3. The multimodal financial data self-reconciliation method according to claim 1 or 2, characterized in that, Within a trusted privacy computing space, multiple reconciliation agents perform matching operations on the heterogeneous financial data based on the feature vectors to obtain reconciliation matching results between different heterogeneous financial data, including: Within the trusted privacy computing space, each of the aforementioned reconciliation agents performs matching operations on the heterogeneous financial data based on the feature vectors to obtain candidate reconciliation matching results corresponding to each of the aforementioned reconciliation agents. The weights of each reconciliation agent are determined, and the agent confidence of each reconciliation agent is determined, wherein the agent confidence is used to characterize the confidence of the reconciliation matching results self-assessed by the reconciliation agent. Based on the candidate reconciliation matching results corresponding to each of the reconciliation agents, the weight of each of the reconciliation agents, and the agent confidence of each of the reconciliation agents, the reconciliation matching results between different heterogeneous financial data are determined.

4. The multimodal financial data self-reconciliation method according to claim 1, characterized in that, After obtaining the reconciliation and matching results between the different heterogeneous financial data, the method further includes: Based on the reconciliation results, unmatched discrepancies are identified, wherein the unmatched discrepancies are heterogeneous financial data that cannot be reconciled and matched with other heterogeneous financial data. Obtain a transaction causal graph for the unmatched discrepancies, wherein the transaction causal graph uses the factors affecting the transaction posting results of the unmatched discrepancies as nodes in the graph, and the causal relationships between the nodes as directed edges in the graph; Counterfactual reasoning is performed based on the aforementioned transaction causal graph to quantify the contribution of each influencing factor to the difference in reconciliation amount; Based on the contribution of each influencing factor to the discrepancy in reconciliation amounts, a root cause analysis report on reconciliation discrepancies is determined.

5. The multimodal financial data self-reconciliation method according to claim 1, characterized in that, The semantic alignment model was trained in the following manner: Obtain training samples, wherein the training samples include seed samples, positive samples from the same transaction as the seed samples, and negative samples from different transactions than the seed samples; A loss function is constructed based on the cosine similarity between the seed sample and the positive sample, and the cosine similarity between the seed sample and the negative sample; Based on the loss function and the training samples, the semantic alignment model is trained to obtain a trained semantic alignment model.

6. The multimodal financial data self-reconciliation method according to claim 5, characterized in that, Before constructing the loss function based on the cosine similarity between the seed sample and the positive sample, and the cosine similarity between the seed sample and the negative sample, the method further includes: A weighting coefficient is assigned to each negative sample, wherein the weighting coefficient is used to characterize the difficulty of distinguishing the negative sample from the seed sample; The loss function is constructed based on the cosine similarity between the seed sample and the positive sample, and the cosine similarity between the seed sample and the negative sample, including: A loss function is constructed based on the cosine similarity between the seed sample and the positive sample, the cosine similarity between the seed sample and the negative sample, and the weighting coefficient of the negative sample.

7. The multimodal financial data self-reconciliation method according to claim 6, characterized in that, The weighting coefficients are determined in the following manner: Obtain the first transaction amount and first transaction time of the seed sample, and obtain the second transaction amount and second transaction time of the negative sample; Determine the transaction amount sensitivity bandwidth and the transaction time sensitivity bandwidth respectively; The weighting coefficient is determined based on the first transaction amount, the first transaction time, the second transaction amount, the second transaction time, the transaction amount sensitivity bandwidth, and the transaction time sensitivity bandwidth.

8. A multimodal financial data self-reconciliation system, characterized in that, The system includes: The acquisition module is used to acquire multiple heterogeneous financial data from different data sources, wherein the heterogeneous financial data includes structured transaction records, semi-structured messages, and unstructured bill files; The processing module is used to input the heterogeneous financial data into a pre-trained semantic alignment model to obtain a feature vector output by the semantic alignment model that matches the heterogeneous financial data. The feature vector is obtained by mapping the heterogeneous financial data to a unified semantic embedding space. The reconciliation module is used to perform matching operations on the heterogeneous financial data based on the feature vector within a trusted privacy computing space through multiple reconciliation agents to obtain reconciliation matching results between different heterogeneous financial data.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the multimodal financial data autonomous reconciliation method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multimodal financial data autonomous reconciliation method as described in any one of claims 1 to 7.

Images