Intelligent expense reimbursement auditing system based on multi-modal identification

Abstract

The invention discloses an intelligent expense reimbursement auditing system based on multi-modal identification, and relates to the technical field of artificial intelligence and financial informatization. According to the system, invoice images, structured transaction data and text description information are acquired through a multi-modal data acquisition module, multi-modal feature weighted fusion is realized by using an attention mechanism of a feature fusion module and a tensor fusion technology, and a multi-modal feature fusion model is established by combining Bayesian reasoning of an auditing decision module, Mahalanobis distance of an anomaly detection module and an auto-encoder algorithm. And a rule engine module, an interpretability analysis module and a federal learning architecture, so that the problems of low efficiency, high misjudgment rate, poor rule adaptability, insufficient risk detection capability and the like of a traditional system are solved. The system can shorten the single checking time from 5-10 minutes to a second level, the recognition accuracy rate exceeds 99%, the misjudgment rate is lower than 0.5%, 10 + type high-risk behaviors can be automatically recognized, dynamic policy updating and interpretability analysis are supported, data privacy is guaranteed through federal learning, and the system is suitable for enterprise financial reimbursement checking.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and financial information technology, and in particular to an intelligent expense reimbursement review system based on multimodal recognition. Background Art

[0002] Expense reimbursement review is a high-frequency and critical component of a company's financial management process. Traditional reimbursement review relies primarily on manual processes, requiring financial personnel to individually verify the authenticity, accuracy, and compliance with company reimbursement policies of various documents (such as invoices, travel receipts, and expense lists). This manual review process is not only inefficient, with an average processing time of 5-10 minutes per document, but is also prone to misreviews and omissions due to fatigue or subjective judgment. Statistics show that the error rate of manual review can reach 3%-5%. This problem is particularly prominent in scenarios with large document volumes or complex rules (such as cross-border travel and project sharing).

[0003] The mainstream reimbursement review system currently on the market usually consists of an OCR recognition module, a rule engine, and a database. Its typical workflow is as follows:

[0004] 1. OCR recognition: Extract key information on the invoice (such as amount, tax number, date) through optical character recognition technology;

[0005] 2. Rule verification: Perform logical judgment based on preset rules (such as "travel expenses ≤ budget limit");

[0006] 3. Manual review: Manual intervention for abnormal cases that the system cannot handle.

[0007] However, the existing system has obvious limitations:

[0008] 1. Single-modal dependency: It can only process structured data or image text and cannot integrate multi-source information (such as invoice image anti-counterfeiting features + ERP system transaction records);

[0009] 2. Static rule defects: Unable to dynamically adapt to policy changes or complex scenarios (such as temporary adjustments to reimbursement standards);

[0010] 3. Hidden risk blind spots: Lack of intelligent detection capabilities for risks such as bill authenticity and data tampering.

[0011] Therefore, in view of this, the existing technology was studied and improved, and an intelligent expense reimbursement review system based on multimodal recognition was proposed. Summary of the Invention

[0012] The technical problem to be solved by the present invention is that the traditional expense reimbursement review system has low efficiency, high misjudgment rate, poor rule adaptability and insufficient risk detection capability due to its reliance on manual or single modality recognition technology (such as OCR).

[0013] The technical solution adopted by the present invention is: an intelligent expense reimbursement review system based on multimodal recognition, comprising:

[0014] Multimodal data collection module, used to collect invoice images, structured transaction data and text description information;

[0015] Feature fusion module, which uses attention mechanism to achieve weighted fusion of multimodal features;

[0016] Audit decision module, used to output audit results based on fusion features;

[0017] The fusion weight of the feature fusion module is calculated as follows:

[0018]

[0019] where h i is the eigenvector of the i-th mode, W i is the trainable weight matrix.

[0020] As a further solution of the present invention: the multimodal data acquisition module includes:

[0021] Image feature extractor based on convolutional neural network, whose loss function adopts improved TripletLoss:

[0022]

[0023] where x a is the anchor point sample, x p is a positive sample, x n is a negative sample, and α is the boundary threshold.

[0024] As a further solution of the present invention: the feature fusion module adopts tensor fusion technology to construct high-order feature interaction:

[0025] where ||u i ||2=||v i ||2=||w i ||2=1

[0026] in represents the tensor product, and k is the decomposition rank.

[0027] As a further solution of the present invention, it also includes an anomaly detection module, which uses outlier detection based on Mahalanobis distance:

[0028]

[0029] Where μ is the sample mean, S is the covariance matrix, when It is judged as abnormal.

[0030] As a further solution of the present invention: the anomaly detection module integrates an autoencoder network, and its reconstruction error is calculated as:

[0031]

[0032] where f φ is the encoder, g θ is the decoder, and Σ is the covariance matrix of the feature space.

[0033] As a further solution of the present invention: a rule engine module is included, and its logical expression is:

[0034]

[0035] where q ij is a regular vector, b ij is the threshold, and m is the total number of rules.

[0036] As a further solution of the present invention: a multi-task learning framework is adopted, and its joint loss function is:

[0037]

[0038] in is the loss of the t-th task, Ω is the shared parameter matrix, and γ is the regularization coefficient.

[0039] As a further solution of the present invention: the audit decision module adopts Bayesian reasoning:

[0040]

[0041] The prior probability P(y) is estimated through historical data, and the likelihood function P(x|y) is modeled using a Gaussian mixture model.

[0042] As a further solution of the present invention: an interpretability analysis module is included, and its SHAP value is calculated as:

[0043]

[0044] Where M is the total number of features, S is the feature subset, and f is the model prediction function.

[0045] As a further solution of the present invention: the system is deployed as a federated learning architecture, and its global parameter update is implemented by the following differential equation:

[0046]

[0047] Where η is the learning rate, K is the number of clients, n k is the data volume of the kth client, and ξ(t) is the Brownian motion noise that satisfies differential privacy.

[0048] Beneficial effects of the present invention:

[0049] 1. Audit efficiency has been greatly improved

[0050] Through the automated fusion analysis of multimodal data (images, structured data, text), the review time for a single reimbursement is shortened from 5-10 minutes manually to seconds, increasing processing efficiency by more than 10 times.

[0051] Supports batch processing of high-concurrency reimbursement documents to meet the large-scale financial operation needs of enterprises.

[0052] 2. Recognition accuracy is significantly improved

[0053] Combining deep learning with attention mechanisms, multimodal fusion technology can cross-verify the authenticity of bills (such as invoice stamps and tax number consistency), increase the accuracy of key information recognition to 99%+, and reduce the misjudgment rate to below 0.5%.

[0054] The dynamic weight adjustment mechanism (such as image modality has a higher weight in anti-counterfeiting detection) further optimizes the decision reliability in complex scenarios.

[0055] 3. Enhanced risk prevention and control capabilities

[0056] Through anomaly detection algorithms (Mahalanobis distance + autoencoder), it automatically identifies 10+ types of high-risk behaviors such as tampering with bills, duplicate reimbursements, and excessive applications, covering hidden vulnerabilities that traditional rule engines cannot handle.

[0057] Real-time linkage with enterprise ERP and budget system data ensures that each reimbursement complies with dynamic policy requirements (such as adjustments to sudden expense limits).

[0058] 4. Compliance and explainability optimization

[0059] Built-in explainable AI technology (SHAP value analysis) generates visual audit reports and clearly marks the basis for passing / rejecting decisions (such as "Reason for rejection: invoice amount exceeds travel standards") to meet audit traceability needs.

[0060] The rule engine supports natural language configuration, and financial personnel can quickly update policy terms (such as "the upper limit of travel and meal subsidies in 2024 will be adjusted to 150 yuan / day") without code modification.

[0061] 5. Scalability and privacy protection

[0062] It adopts a federated learning architecture to support cross-branch data collaborative training models, while ensuring that sensitive financial data does not leave the local area, complying with data compliance requirements such as GDPR.

[0063] The modular design allows for flexible integration into the company's existing financial systems (such as SAP and UFIDA), reducing deployment costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is an architectural diagram of an intelligent expense reimbursement review system based on multimodal recognition according to the present invention.

[0065] Figure 2 This is a schematic diagram of the audit decision process of an intelligent expense reimbursement audit system based on multimodal recognition in the present invention. DETAILED DESCRIPTION

[0066] The present invention will be further described below.

[0067] See also Figure 1-2

[0068] Example 1: Travel expense reimbursement review

[0069] Scenario description: An employee submits a travel reimbursement application, which includes an electronic invoice, itinerary, and expense description.

[0070] System workflow:

[0071] 1. Multimodal data acquisition:

[0072] Image modality: CNN is used to extract invoice anti-counterfeiting features (such as the location of the supervision stamp and the QR code area), and an improved TripletLoss (α = 0.3) is used to ensure that the distance between similar invoice features is smaller than that between cross-class samples.

[0073] Text modality: The BERT model parses key information in the fare description (such as "Beijing to Shanghai high-speed rail ticket").

[0074] Structured data: Obtain the employee's travel budget standard from the ERP system (such as a daily accommodation limit of 800 yuan).

[0075] 2. Feature Fusion

[0076] Tensor fusion (k=64) is used to construct high-order interaction features, for example:

[0077] Invoice amount (numeric value) Travel date (time series) Expense type (classification) → Detect anomalies in “non-working day transportation expenses”.

[0078] Dynamic allocation of attention weights: Image modality accounts for 70% of the weight in anti-counterfeiting detection, and text modality accounts for 60% of the weight in rationality analysis.

[0079] 3. Review decision:

[0080] Rule engine verification: ψ = (invoice amount ≤ 800) ∧ (trip date ∈ project period).

[0081] Anomaly Detection: Calculating Mahalanobis Distance The autoencoder reconstruction error detection (L_AE=1.8>threshold 1.5) was triggered, and traces of invoice PS were found.

[0082] Output result: The application is rejected and a report is generated: "Exception reason: the invoice is suspected of being tampered with, and the amount exceeds the standard by 200 yuan."

[0083] Example 2: Project Procurement Reimbursement Review

[0084] Scenario description: When purchasing equipment across departments, it is necessary to verify the authenticity of invoices and the rationality of budget allocation.

[0085] System innovation application:

[0086] 1. Federated Learning Collaboration:

[0087] Each branch company trains a local invoice classification model (e.g., distinguishing between “office equipment” and “service fees”), and the global parameters are updated by Brownian motion with noise (σ_t 2 =0.01), satisfying differential privacy of ε=0.5.

[0088] 2. Multi-task learning:

[0089] Joint Optimization:

[0090] Task 1: Invoice authenticity classification (cross entropy loss L1).

[0091] Task 2: Budget department allocation (mean square error L2).

[0092] Total loss L_total=0.7L1+0.3L2+0.01||Ω|| 2 _F.

[0093] 3. Explainability Analysis

[0094] The SHAP value shows that the TOP3 features contribute to the rejection decision:

[0095] Invoice supplier blacklist matching degree (φ=0.42)

[0096] Ratio of departmental budget surplus (φ=0.35)

[0097] The correlation between procurement reasons and projects (φ=0.23).

[0098] Output result: Require additional supplier qualification documents and adjust the allocation ratio to 60% for Department A and 40% for Department B.

[0099] Example 3: International Conference Expenses Reimbursement

[0100] Scenario description: An employee submits a foreign currency invoice and exchange rate certificate, involving multi-currency conversion.

[0101] Technical implementation details:

[0102] 1. Multimodal data enhancement:

[0103] Image modality: Generative Adversarial Network (GAN) synthesizes multi-angle invoice samples to improve the robustness of OCR in minority languages.

[0104] Text modality: Converts German invoice descriptions into structured fields (e.g. “Konferenzgebühr” → conference registration fee) based on the Translation API.

[0105] 2. Dynamic rule adaptation:

[0106] When the currency is detected as Euro, the latest foreign exchange policy rules are automatically loaded:

[0107]

[0108] The threshold b2 is updated based on the real-time central bank exchange rate.

[0109] 3. Bayesian reasoning:

[0110] The prior probability P(y=compliant) comes from historical data (82%), and the likelihood function P(xy) is modeled using GMM:

[0111] Foreign currency amount subject to

[0112] The exchange rate proves that the matching degree obeys Beta (α=5,β=2).

[0113] Output result: Automatically converted into RMB based on the exchange rate of the day. Because it exceeds the standard by 10%, but is accompanied by a special approval record, it is marked as "Second-level review required".

[0114] Example 4: Batch review of catering and entertainment expenses

[0115] Scenario description: A department submits 50+ catering invoices for centralized reimbursement each month and needs to detect duplicate reimbursements and excess reimbursements.

[0116] System optimization design:

[0117] 1. Batch processing acceleration:

[0118] Using GPU parallel computing, the image feature extraction speed reaches 200 images per second (ResNet50+FP16 quantization).

[0119] The rule engine is compiled into LLVM intermediate code, and the logic judgment delay is <1ms / order.

[0120] 2. Anomaly Detection Innovation:

[0121] Spatiotemporal correlation analysis: Multiple invoices from the same merchant on the same day → trigger duplicate detection.

[0122] Autoencoder reconstruction: normal invoice L_AE ≈ 0.8, tampered invoice L_AE > 2.0 (threshold 1.5).

[0123] 3. Visual Report:

[0124] Generate a heat map to show the cluster distribution of invoices that exceed the standard (for example, "meal expenses per person > 150 yuan" are concentrated on Friday dinners).

[0125] Output result: 3 duplicate invoices (similarity 98.7%) were automatically rejected, and 7 invoices that exceeded the standards were marked and required the supervisor's signature confirmation.

[0126] The above embodiments demonstrate the technical implementation details and innovations of the system in different business scenarios. During actual deployment, parameter thresholds or module combinations can be adjusted according to the specific needs of the enterprise.

[0127] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent expense reimbursement review system based on multimodal recognition, characterized in that: include: Multimodal data collection module, used to collect invoice images, structured transaction data and text description information; Feature fusion module, which uses attention mechanism to achieve weighted fusion of multimodal features; Audit decision module, used to output audit results based on fusion features; The fusion weight of the feature fusion module is calculated as follows: where h i is the eigenvector of the i-th mode, W i is the trainable weight matrix.

2. The intelligent expense reimbursement review system based on multimodal recognition according to claim 1 is characterized by: The multimodal data acquisition module includes: Image feature extractor based on convolutional neural network, whose loss function adopts improved TripletLoss: where x a is the anchor point sample, x p is a positive sample, x n is a negative sample, and α is the boundary threshold.

3. The intelligent expense reimbursement review system based on multimodal recognition according to claim 1 is characterized by: The feature fusion module uses tensor fusion technology to build high-order feature interactions: Among them||u i ||2=||v i ||2=||w i ||2=1 in represents the tensor product, and k is the decomposition rank.

4. The intelligent expense reimbursement review system based on multimodal recognition according to claim 1 is characterized by: It also includes an anomaly detection module that uses outlier detection based on Mahalanobis distance: Where μ is the sample mean, S is the covariance matrix, when It is judged as abnormal.

5. The intelligent expense reimbursement review system based on multimodal recognition according to claim 4 is characterized by: The anomaly detection module integrates an autoencoder network, and its reconstruction error is calculated as: where f φ is the encoder, g θ is the decoder, and Σ is the covariance matrix of the feature space.

6. The intelligent expense reimbursement review system based on multimodal recognition according to claim 1 is characterized by: Contains the rule engine module, whose logical expression is: where q ij is a regular vector, b ij is the threshold, and m is the total number of rules.

7. The intelligent expense reimbursement review system based on multimodal recognition according to claim 1 is characterized by: Using a multi-task learning framework, its joint loss function is: in is the loss of the t-th task, Ω is the shared parameter matrix, and γ is the regularization coefficient.

8. The intelligent expense reimbursement review system based on multimodal recognition according to claim 1 is characterized by: The review decision module uses Bayesian reasoning: The prior probability P(y) is estimated through historical data, and the likelihood function P(x|y) is modeled using a Gaussian mixture model.

9. The intelligent expense reimbursement review system based on multimodal recognition according to claim 1 is characterized by: Contains the interpretability analysis module, whose SHAP value is calculated as: Where M is the total number of features, S is the feature subset, and f is the model prediction function.

10. The intelligent expense reimbursement review system based on multimodal recognition according to claim 9, characterized in that: The system is deployed as a federated learning architecture, and its global parameter update is implemented by the following differential equations: Where η is the learning rate, K is the number of clients, n k is the data volume of the kth client, and ξ(t) is the Brownian motion noise that satisfies differential privacy.

Images