Financial reimbursement data processing method, device and system

By combining multimodal data processing and a dynamic rule engine with blockchain evidence storage technology, the problems of low efficiency, high error rate, and insufficient security in the financial reimbursement process have been solved, achieving efficient and accurate automated reimbursement processing and data security.

CN120672495AInactive Publication Date: 2025-09-19BEIJING SHIJITAN HOSPITAL CAPITAL MEDICAL UNIVERSITY

Patent Information

Application Number
CN202510813027.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-06-18
Publication Date
2025-09-19
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing financial reimbursement process is inefficient, has a high error rate, and lacks security. Traditional methods rely on manual review and are difficult to adapt to changes in corporate business. Data security is poor, and static compliance verification rules are prone to false positives or omissions.

Method used

It adopts multimodal data processing, dynamic rule engine and blockchain evidence storage technology, uses convolutional neural networks to recognize image text, and natural language processing models to analyze voice and video information, generate dynamic verification rules, and combine blockchain storage and manual review to form an audit traceability chain.

Benefits of technology

It significantly improves the accuracy of parsing complex bills, reduces manual review costs, improves the accuracy of anomaly detection, ensures data security and compliance, shortens reimbursement cycles, and enhances the system's high availability and response speed.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120672495A_ABST
    Figure CN120672495A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of computer data processing, and discloses a financial reimbursement data processing method, device and system.The financial reimbursement data processing method comprises the following steps that multi-modal data of a reimbursement voucher is obtained, and the multi-modal data is stored in a block chain; the multi-modal data comprises pictures, voices and videos of reimbursement vouchers; recognizing characters on the picture based on a convolutional neural network, and extracting and analyzing character information of key frames of the voice and the video by using a natural language processing model; and performing feature extraction on the character information to obtain character feature information, judging whether the character feature information falls into a reimbursement directory, and performing local model fitting according to the character feature information. According to the method, the dynamic verification rule is generated based on the historical reimbursement data, the bill anomaly probability is calculated in real time through the anomaly coefficient formula, and the anomaly detection accuracy is improved by combining the error coefficient and keyword matching degree and the historical record relevance index.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of computer data processing, and in particular to a method, device and system for processing financial reimbursement data. Background Art

[0002] Currently, the financial reimbursement process generally suffers from problems such as low efficiency, high error rate, and insufficient security. Traditional methods mainly rely on manual review of paper or electronic vouchers, resulting in long review cycles, high costs, and easy omissions or misjudgments due to human factors. In addition, compliance verification is mostly based on static rule bases, making it difficult to dynamically adjust thresholds based on historical data, which can easily lead to false positives or omissions and cannot adapt to the changing needs of corporate business. In terms of data security, traditional storage methods have the risk of vouchers being lost, tampered with, or forged, and the audit traceability chain is incomplete, making it difficult to meet internal control and external compliance requirements. Therefore, there is an urgent need for a fully automated solution that integrates multimodal data processing, a dynamic rule engine, and blockchain evidence storage technology to systematically improve the efficiency, accuracy, and security of the reimbursement process.

[0003] An existing patent discloses a method, device, and system for processing financial reimbursement data (publication number CN111079755A). The method comprises obtaining an image of a reimbursement voucher for a pending reimbursement application; segmenting the image of the reimbursement voucher according to identified key reimbursement areas to obtain image slices corresponding to the key reimbursement areas; and performing image text recognition on the image slices to obtain substructured data corresponding to the image slices. Although this existing technology has introduced some automated processing methods, such as basic visual technology to extract invoice information, its analysis accuracy remains low when dealing with multimodal data (such as images, audio, and video), and its recognition capability is particularly limited for complex invoices (such as blurred images or noisy audio). Summary of the Invention

[0004] The present invention provides a financial reimbursement data processing method, device and system to solve the existing technical problems, and solves the problems in the above-mentioned background technology.

[0005] To solve the above technical problems, according to one aspect of the present invention, more specifically a method for processing financial reimbursement data, the method includes the following steps:

[0006] S1. Obtain multimodal data of the reimbursement voucher and store the multimodal data on the blockchain. The multimodal data includes pictures, audio, and video of the reimbursement voucher.

[0007] S2. Recognize the text on the image based on a convolutional neural network, and extract and parse the text information of the key frames of the voice and video using a natural language processing model;

[0008] S3. Extracting features from the text information to obtain text feature information, determining whether the text feature information falls within the reimbursement list, and performing local model fitting based on the text feature information;

[0009] S4. Generate dynamic verification rules based on historical reimbursement data and verify the data using the dynamic verification rules;

[0010] S5. Trigger an alarm for data that fails verification, and retrieve the original certificate stored in the blockchain for manual review.

[0011] Furthermore, the convolutional neural network recognizes the text on the image, including:

[0012] 1) Convert the input image to grayscale and use edge detection algorithm to extract the image contour;

[0013] 2) Generate candidate text regions based on the region proposal network algorithm, and filter high-confidence text regions through non-maximum suppression to obtain the corresponding characters;

[0014] 3) Correct easily confused characters through dictionary matching algorithm and match the recognition results with keywords in the reimbursement rule library.

[0015] Furthermore, the natural language processing model extracts and parses text information from speech and video key frames, including:

[0016] 1) Use an end-to-end speech recognition model to convert speech into a first text sequence, and use OCR technology to recognize text information embedded in video key frames to obtain a second text sequence;

[0017] 2) Align the timestamps of the first text sequence and the second text sequence to extract the text information related to reimbursement.

[0018] Furthermore, in step S3, whether the extracted text information falls into the reimbursement list is determined based on whether the extracted text information contains keywords of the reimbursement list.

[0019] Furthermore, the local model calculates the bill reimbursement anomaly coefficient based on the error coefficient between the amount or date in the text feature information and the actual consumption record information, as well as the relevance index of the reimbursement business, which is:

[0020] ;

[0021] In the formula, g represents the abnormal coefficient of bill reimbursement; Indicates the error coefficient between the amount or date in the bill and the actual consumption record information; Represents the relevance index of reimbursement business.

[0022] Furthermore, in step S4, a dynamic verification threshold is generated based on the relationship between the historical reimbursement data and the abnormality coefficient g of the bill reimbursement, and the financial reimbursement is verified to be abnormal based on the verification threshold.

[0023] Furthermore, in steps S1 to S5, all correction operation records are stored in the blockchain to form an audit traceability chain.

[0024] A financial reimbursement data processing system includes:

[0025] The user terminal module is used to receive multimodal data submitted by users, provide a data upload interface, and pre-process the original data;

[0026] The dynamic rule engine module is used to generate verification thresholds and perform compliance verification; if the verification fails, an alarm is triggered and the blockchain evidence data is called.

[0027] A financial reimbursement data processing device includes: a multimodal data acquisition device, an edge computing device and a dynamic verification controller.

[0028] Compared with the prior art, the financial reimbursement data processing method, device, and system provided by the present invention have the following advantages:

[0029] 1. This invention combines OCR, convolutional neural networks, and natural language processing technologies to extract structured information from multimodal data such as images, voice, and video, significantly improving the accuracy of parsing complex bills and reducing manual review costs.

[0030] 2. The present invention generates dynamic verification rules based on historical reimbursement data, calculates the probability of bill anomalies in real time through the anomaly coefficient formula, and combines the error coefficient with keyword matching and historical record correlation index to improve the accuracy of anomaly detection.

[0031] 3. This invention ensures that all original documents and operation records are stored on the blockchain, ensuring that data cannot be tampered with, forming a complete audit traceability chain, and meeting the compliance requirements of internal control and external audits. Furthermore, when verification fails, the blockchain evidence data is automatically retrieved, combined with a manual review mechanism, to ensure automation efficiency while reducing the risk of system misjudgment.

[0032] 4. The present invention realizes real-time data processing through edge computing devices, reduces the load on cloud servers, and improves the response speed in high-concurrency scenarios.

[0033] 5. This invention automates the entire process, from data collection, identification, verification, to payment and evidence storage, significantly shortening the reimbursement cycle and improving financial processing efficiency. Furthermore, blockchain technology, combined with encryption algorithms, addresses the vulnerability of traditional reimbursement vouchers to loss, tampering, or forgery, enhancing data security and credibility.

[0034] 6. During the data analysis or verification process, the system automatically detects abnormal nodes (such as areas where OCR recognition fails), triggers local reprocessing or prompts users to supplement specific information, and quickly locates problems based on the original data stored in the blockchain, reducing the risk of overall process interruption and ensuring high availability of reimbursement processing.

[0035] 7. The present invention uses timestamp alignment technology and cross-modal feature fusion algorithm to achieve collaborative analysis of image, voice, and video data, ensuring the consistency of multi-source data (such as matching the consumption scene described by voice with the location information in the video key frame), further improving the comprehensiveness and accuracy of anomaly detection, and avoiding misjudgments caused by errors in a single data source. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart of the present invention;

[0037] Figure 2 is a relationship diagram between the abnormal coefficient g and the error coefficient a in the present invention;

[0038] Figure 3 is a relationship diagram between the abnormal coefficient g and the correlation index m in the present invention;

[0039] Figure 4 This is a relationship diagram of the abnormal coefficient g, error coefficient a, and correlation index m of the present invention. DETAILED DESCRIPTION

[0040] In order to make the technical solution of the present invention clearer, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] Example 1

[0042] like Figure 1 As shown, according to one aspect of the present invention, a method for processing financial reimbursement data is provided, comprising acquiring multimodal data from reimbursement vouchers and storing the multimodal data on a blockchain. The multimodal data includes images, audio, and video of the reimbursement vouchers. By collecting multimodal data such as images, audio, and video from the reimbursement vouchers and storing them using blockchain technology, the integrity and immutability of the data are ensured. Multimodal data refers to data in multiple forms, such as images (visual information), audio (audio information), and video (a combination of dynamic visual and audio information), requiring separate parsing and fusion processing using different technologies.

[0043] Blockchain storage is a distributed database technology that stores data through encryption algorithms and a decentralized node network, ensuring that the data cannot be tampered with and is traceable, providing a secure storage solution for reimbursement vouchers that prevents loss and forgery. Blockchain storage enhances data security and provides a traceable audit chain, solving the problem of vouchers being easily lost or tampered with in traditional reimbursement, and laying a trusted data foundation for subsequent automated processing.

[0044] The core of this embodiment is to obtain multimodal data such as images, voice, and video from reimbursement vouchers through the user terminal module and store them using blockchain technology. Furthermore, the multimodal data is used to analyze and integrate various forms of data (such as images, voice, and video) separately.

[0045] Example 2

[0046] The convolutional neural network is used to recognize text on images, and the natural language processing model is used to extract and parse text information from key frames of audio and video. The specific steps for the convolutional neural network to recognize text in images of reimbursement vouchers are as follows:

[0047] 1) Grayscale the input image (grayscale processing is used to reduce computational complexity) and use edge detection algorithms (edge ​​detection such as the Canny algorithm to extract image contours and locate text areas) to extract the image contours;

[0048] 2) Generate candidate text regions based on the region proposal network algorithm (the region proposal network can generate candidate text regions and filter high-confidence regions through non-maximum suppression), and filter high-confidence text regions through non-maximum suppression to obtain the corresponding characters;

[0049] 3) Correct easily confused characters through dictionary matching algorithm (dictionary matching correction is to solve easily confused characters (such as "0" and "O") and match them with the keywords in the reimbursement rule library) and match the recognition results with the keywords in the reimbursement rule library.

[0050] Convolutional neural networks are a type of deep learning model specifically designed for image processing. They automatically extract image features (such as text outlines) through convolutional layers, effectively recognizing text in images of expense reimbursement vouchers. Convolutional neural networks are used to extract and classify features from expense reimbursement voucher images, enhancing the robustness of text recognition.

[0051] The natural language processing model extracts and parses the text information of speech and video key frames in the following steps:

[0052] 1) Use an end-to-end speech recognition model (end-to-end speech recognition is a Transformer model used to convert speech features into text sequences) to convert speech features into a first text sequence. Optical character recognition (OCR) is applied to keyframe images (video keyframe OCR is used to extract text (such as invoice code and amount) from video keyframes) and uses OCR to recognize text embedded in video keyframes to obtain a second text sequence. OCR refers to optical character recognition, which is used to extract reimbursement-related text information (such as amount, date, and merchant name) from images or video keyframes of reimbursement vouchers. Optical character recognition primarily uses image analysis to convert printed or handwritten text in images or video frames into machine-readable text data, which is used to extract information such as bill amount and date.

[0053] 2) Align the timestamps of the first and second text sequences, extract the text information related to reimbursement (timestamp alignment includes integrating voice text and video text, and extracting structured fields (such as consumption time and location)), and then extract the structured fields related to reimbursement.

[0054] Natural language processing (NLP) models are a branch of artificial intelligence technology used to parse and understand speech or text data. For example, they can convert speech to text, extract text from key frames in videos, and structure it into reimbursement fields. Natural language processing is used to parse speech content and text from videos, extracting key fields (such as consumption scenarios and project names).

[0055] Therefore, we use a convolutional neural network to grayscale, detect edges, and filter text regions on reimbursement voucher images, combined with a natural language processing model to analyze text information in audio and video. Multimodal fusion technology improves text recognition accuracy, especially for complex documents (such as blurred images or speech with background noise), significantly reducing manual review costs.

[0056] Example 3

[0057] like Figure 1-4 As shown, whether the extracted text information falls into the reimbursement list is determined based on whether the extracted text information contains keywords of the reimbursement list, and local model fitting is performed based on the text feature information.

[0058] The local model calculates the bill reimbursement anomaly coefficient based on the error coefficient between the amount or date in the text feature information and the actual consumption record information, as well as the reimbursement business correlation index.

[0059] ;

[0060] In the formula, g represents the abnormal coefficient of bill reimbursement. The larger the value of g, the greater the probability of abnormality in the financial reimbursement process of this voucher. Indicates the error coefficient between the amount or date in the bill and the actual consumption record information; Represents the relevance index of reimbursement business.

[0061] Error coefficient: measures the degree of deviation between the amount or date in the reimbursement document and the actual consumption record, such as the amount error ( ) is the relative error, date error ( ) is the percentage of the allowable difference in days, which reflects the overall error after weighting.

[0062] Among them, the error coefficient between the amount or date in the bill and the actual consumption record information The specific calculation steps are:

[0063] 1) The amount error uses the relative error form to calculate the difference between the bill amount and the actual consumption amount:

[0064] ;

[0065] Where, Indicates the error coefficient of amount; Indicates the amount in the bill; Indicates the actual consumption amount.

[0066] 2) Calculation of date error:

[0067] ;

[0068] In the above formula, represents the date error coefficient; Indicates the date in the bill; Indicates the date of actual consumption; Indicates the maximum allowed difference in days.

[0069] Relevance Index: Evaluate the relevance of reimbursement business with historical data, such as by keyword matching ( ) and historical reimbursement frequency ( ) weighted calculation, high correlation may reduce the probability of abnormality.

[0070] Then, the error coefficient between the amount or date in the bill and the actual consumption record information is:

[0071] ;

[0072] In the above formula, 、 Represent the weights of amount error and date error respectively, and the weights .

[0073] Among them, the relevance index of reimbursement business The specific calculation steps are:

[0074] ;

[0075] In the above formula, Indicates the matching degree of keywords; Indicates the total number of keywords; Indicates the number of historical parameter reimbursements for this keyword; Indicates the total number of items.

[0076] The above formula is an empirical analysis conducted to solve practical problems in enterprises. The data obtained in turn and the characteristic relationships between the data are the external manifestations of the empirical formula. The reasoning process is as follows:

[0077] 1) Fit the formula for the abnormal coefficient g of bill reimbursement.

[0078] Among them, the abnormal coefficient g of bill reimbursement can be expressed by the number of reimbursement anomalies in the sample data.

[0079] For example, if 100 reimbursement sample data are collected and the degree of abnormality of the reimbursement voucher information after manual evaluation exceeds the data in the other 50 samples, then it means that the abnormality coefficient g of the bill reimbursement is 50%.

[0080] Other data are calculated and fitted using data collected by the device itself.

[0081] 2) Establish a mathematical model for the relationship between the abnormal coefficient g and the error coefficient a (such as Figure 2 As shown in the figure, the red dots are the distribution of the 100 samples collected), then:

[0082] (Formula 1);

[0083] In the above formula 1, k represents an empirical constant for adjusting the sensitivity of the above model.

[0084] 3) Establish a mathematical model for the relationship between the anomaly coefficient g and the correlation index m (such as Figure 3 As shown in the figure, the blue dots are the distribution of the 100 samples collected), then:

[0085] (Formula 2);

[0086] In the above formula 2, k represents an empirical constant for adjusting the sensitivity of the above model.

[0087] 4) Establish a mathematical model for the relationship between the abnormal coefficient g, error coefficient a, and correlation index m (such as Figure 4 As shown, the model shows that the error coefficient a and the correlation index m are inversely correlated). Combined with the characteristic relationship of the above formula 1 and formula 2, we have:

[0088] g=(Formula 1)×(Formula 2);

[0089] According to the above derivation, we can know that the abnormal coefficient g of bill reimbursement is:

[0090] .

[0091] Among them, the error coefficient between the amount or date in any bill and the actual consumption record information is , the relevance index of reimbursement business is , then we have:

[0092] ;

[0093] Through the above calculation, we can know that the abnormal coefficient of this bill reimbursement The larger the value of g is, the greater the probability of abnormality occurring during the financial reimbursement process of this voucher.

[0094] Example 4

[0095] Dynamic verification rules are generated based on historical reimbursement data (dynamic thresholds, such as tolerance for monetary amount errors, are trained based on historical data). Data is verified using dynamic verification rules, and the verification criteria are dynamically adjusted based on the distribution of the anomaly coefficient g. Dynamic verification rules are automatically generated based on historical reimbursement data. Thresholds (such as tolerance for monetary amount errors) are adjusted through machine learning to adapt to business changes and reduce misjudgments caused by static rules.

[0096] A dynamic verification threshold is generated based on the relationship between historical reimbursement data and the anomaly coefficient g of bill reimbursements. This threshold is then used to verify whether financial reimbursements are abnormal. Dynamic verification rules are trained using historical reimbursement data, and dynamic thresholds are generated based on the distribution of the anomaly coefficient g, allowing for real-time adjustment of verification standards. The dynamic rule engine adapts to changes in the enterprise's business, improving the flexibility of compliance verification and avoiding false positives or negatives caused by static rules. So, there are:

[0097] Table 1 Abnormal status of some implementation data and reimbursement

[0098] Error coefficient#timg# Correlation Index#timg# Abnormal coefficient g Manual review of abnormal status of photo and video credentials Implementation Data 1 7% 73% 73.3% abnormal Implementation Data 2 6% 74% 69.3% abnormal Implementation Data 3 5% 75% 65.1% abnormal Implementation Data 4 4% 76% 60.7% normal Implementation Data 5 3% 77% 56.2% normal

[0099] From the data in Table 1 above, we can know that when the sample data tends to infinity, there will be abnormal states of the photos and video credentials after manual review divided by the value of the abnormal coefficient g. Then we can have, when There will be obvious abnormalities in the process of financial reimbursement.

[0100] Example 5

[0101] Data that fails verification triggers an alarm, and the original document stored on the blockchain is retrieved for manual review. All correction operation records are stored on the blockchain to form a complete audit traceability chain. Audit traceability chain: Utilizing blockchain technology, all operations in the reimbursement process (such as data upload and correction records) are recorded, forming a complete and tamper-proof operation log, which facilitates traceability and verification during internal and external audits.

[0102] When verification fails, an alarm is triggered, and the original certificate stored on the blockchain is called for manual review. All correction records are synchronized on the chain. Combining automation and manual intervention, it ensures the timely processing of abnormal data; the blockchain evidence forms a complete traceability chain to meet audit compliance requirements. The specific process is as follows:

[0103] 1) Trigger an alarm (such as sound and light signals, message push) when verification fails;

[0104] 2) Retrieve the original evidence (pictures, videos) stored on the blockchain for manual review;

[0105] 3) All revision records are synchronized on the chain to form an audit traceability chain that cannot be tampered with.

[0106] Example 6

[0107] like Figure 1 As shown, according to one aspect of the present invention, a financial reimbursement data processing system is provided, including: a user terminal module for receiving multimodal reimbursement data (pictures, voice, video) submitted by users, providing a data upload interface, and preprocessing the original data (such as format conversion, compression).

[0108] The dynamic rules engine module generates dynamic verification thresholds based on an anomaly coefficient calculation model and performs compliance verification. If verification fails, an alarm is triggered and blockchain-stored evidence is retrieved. Original documents, correction operation records, and verification results can also be stored on-chain, forming an unalterable audit traceability chain. Verification results can also be used to automatically assign approval processes (e.g., documents with high anomaly coefficients are routed to manual review, while documents with low anomaly coefficients are automatically approved).

[0109] This financial reimbursement data processing system consists of a user terminal module (data collection and preprocessing) and a dynamic rules engine module (threshold generation and compliance verification), supporting multi-terminal collaboration. The modular design enhances system scalability, the user terminal simplifies operational processes, and the rules engine ensures verification efficiency, achieving seamless integration throughout the entire process.

[0110] Example 7

[0111] like Figure 1 As shown, according to one aspect of the present invention, a financial reimbursement data processing device is provided, including: a multimodal data acquisition device, an edge computing device, and a dynamic verification controller.

[0112] Multimodal data acquisition equipment includes a high-precision OCR scanner and a voice / video acquisition module. The high-precision OCR scanner is used to quickly capture images of reimbursement vouchers, performing grayscale conversion and edge detection. The voice / video acquisition module integrates a microphone and camera, supporting speech-to-text (ASR) and video keyframe extraction.

[0113] Edge computing devices include an embedded AI chip and a rule-matching engine. The embedded AI chip deploys a convolutional neural network and natural language processing model to achieve localized text recognition and feature extraction. The rule-matching engine, with a built-in reimbursement rule library, matches keywords (such as "travel expenses" and "invoice code") in real time to generate preliminary verification results.

[0114] The dynamic verification controller dynamically adjusts verification thresholds (such as the tolerance for amount errors) based on the calculation logic for the bill reimbursement anomaly coefficient, and synchronizes updates with the cloud-based rule engine. Furthermore, when the anomaly coefficient g exceeds the threshold, a notification is sent via audio and visual signals or a push notification to request human intervention.

[0115] As can be seen from the above, this financial reimbursement data processing device integrates multimodal data acquisition devices (such as cameras and microphones), edge computing devices (real-time processing), and a dynamic verification controller (rule execution). Edge computing reduces cloud load and improves response speed; hardware and algorithm collaborative optimization meets the real-time processing requirements in high-concurrency scenarios.

[0116] The above embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A method for processing financial reimbursement data, characterized in that: The following steps are involved: S1. Obtain multimodal data of the reimbursement voucher and store the multimodal data on the blockchain. The multimodal data includes pictures, audio, and video of the reimbursement voucher. S2. Recognize the text on the image based on a convolutional neural network, and extract and parse the text information of the key frames of the voice and video using a natural language processing model; S3. Extracting features from the text information to obtain text feature information, determining whether the text feature information falls within the reimbursement list, and performing local model fitting based on the text feature information; S4. Generate dynamic verification rules based on historical reimbursement data and verify the data using the dynamic verification rules; S5. Trigger an alarm for data that fails verification, and retrieve the original certificate stored in the blockchain for manual review.

2. The financial reimbursement data processing method according to claim 1, characterized in that: The convolutional neural network recognizes the text on the image, including: 1) Convert the input image to grayscale and use edge detection algorithm to extract the image contour; 2) Generate candidate text regions based on the region proposal network algorithm, and filter high-confidence text regions through non-maximum suppression to obtain the corresponding characters; 3) Correct easily confused characters through dictionary matching algorithm and match the recognition results with keywords in the reimbursement rule library.

3. The financial reimbursement data processing method according to claim 1, characterized in that: The natural language processing model extracts and parses text information from speech and video key frames, including: 1) Use an end-to-end speech recognition model to convert speech into a first text sequence, and use OCR technology to recognize text information embedded in video key frames to obtain a second text sequence; 2) Align the timestamps of the first text sequence and the second text sequence to extract the text information related to reimbursement.

4. The financial reimbursement data processing method according to claim 1, characterized in that: In step S3, whether the extracted text information falls into the reimbursement list is determined based on whether the extracted text information contains keywords of the reimbursement list.

5. The financial reimbursement data processing method according to claim 1, characterized in that: The local model calculates the bill reimbursement anomaly coefficient based on the error coefficient between the amount or date in the text feature information and the actual consumption record information, as well as the relevance index of the reimbursement business, which is: ; In the formula, g represents the abnormal coefficient of bill reimbursement; Indicates the error coefficient between the amount or date in the bill and the actual consumption record information; Represents the relevance index of reimbursement business.

6. The financial reimbursement data processing method according to claim 5, characterized in that: In step S4, a dynamic verification threshold is generated based on the relationship between the historical reimbursement data and the abnormality coefficient g of the bill reimbursement, and the financial reimbursement is verified to be abnormal based on the verification threshold.

7. The financial reimbursement data processing method according to claim 1, characterized in that: In steps S1 to S5, all correction operation records are stored in the blockchain to form an audit traceability chain.

8. A financial reimbursement data processing system, characterized in that: Applied to the financial reimbursement data processing method according to any one of claims 1 to 7, the financial reimbursement data processing system comprises: The user terminal module is used to receive multimodal data submitted by users, provide a data upload interface, and pre-process the original data; The dynamic rule engine module is used to generate verification thresholds and perform compliance verification; if the verification fails, an alarm is triggered and the blockchain evidence data is called.

9. A financial reimbursement data processing device, characterized in that: Applied to the financial reimbursement data processing method according to any one of claims 1 to 7, the financial reimbursement data processing device includes: a multimodal data acquisition device, an edge computing device, and a dynamic verification controller.

Citation Information

Patent Citations

  • Financial reimbursement data processing method, device and system

    CN111079755A

Cited By

  • Multi-user multi-scene reimbursement filling processing system

    CN120852081A

  • Artificial intelligence-based financial and tax bill risk intelligent auditing method

    CN122493478A