Multi-source bill processing method and device, electronic equipment and medium

Abstract

The invention provides a multi-source bill processing method and device, electronic equipment and a medium, and relates to the technical field of computer software, and the method comprises the steps: carrying out the preprocessing of obtained bill data, and obtaining target bill image data; inputting the target bill image data into a multi-modal model for feature extraction to obtain structured text information, visual layout information and semantic association information of the bill data; based on the structured text information, the visual layout information and the semantic association information, whether repeated bills exist or not is determined; when it is determined that no repeated bills exist, compliance verification is carried out on the bill data, and a compliance verification result is obtained; processing the bill data of which the compliance verification result is passed and the configured business bill to obtain a bill knowledge graph; and carrying out cost collection processing on the bill data corresponding to the bill knowledge graph to generate accounting document data. The method can greatly improve the bill processing efficiency, and guarantees the credibility and traceability of the whole process of the bill.

Description

Technical Field

[0001] This application relates to the field of computer software technology, and more specifically, to a method, apparatus, electronic device, and medium for processing multi-source invoices. Background Technology

[0002] The sources of existing invoices are complex, encompassing various forms such as paper-based special / general invoices, PDF electronic invoices, OFD digital invoices, and screenshots of taxi and restaurant receipts. Current corporate invoice management faces numerous pain points: manual entry by employees and manual verification by finance departments lead to low data collection efficiency; electronic invoices can be copied infinitely, making it difficult to prevent duplicate reimbursements such as altered numbers; finance departments need to manually compare contracts, orders, and invoices across systems, leading to errors in document verification and difficulty in identifying fraudulent transactions; the identification of sensitive words in product details and the monitoring of supplier blacklists are lagging, resulting in high compliance risks; mixed paper and electronic storage makes document retrieval inconvenient, and electronic documents are easily tampered with or damaged. In existing technologies, traditional reimbursement systems rely on manual input or simple QR code scanning, unable to handle non-standard invoices and with limited deduplication methods; independent OCR services can only convert images to text, lacking business logic understanding and authenticity verification capabilities; traditional image management systems focus on storage, lacking in-depth compliance checks and document association functions, making it difficult to meet the needs of enterprises for efficient and compliant invoice management. Summary of the Invention

[0003] The purpose of this application is to provide a method, apparatus, electronic device and medium for processing multi-source invoices, so as to solve the above-mentioned problems existing in the prior art, which can significantly improve the efficiency of invoice processing, realize the deep integration of business and financial data, reduce the cost of archive management, and ensure the reliability of electronic archives throughout their entire life cycle.

[0004] Firstly, a method for processing multi-source invoices is provided, which may include: Acquire invoice data from multiple heterogeneous data sources, including image-based invoice files and structured electronic invoice files; For any given invoice data, the invoice data is preprocessed to obtain target invoice image data; The target ticket image data is input into a multimodal model for feature extraction to obtain the structured text information, visual layout information, and semantic association information of the ticket data; Based on the structured text information, the visual layout information, and the semantic association information, determine whether there are duplicate tickets; When it is determined that there are no duplicate invoices, the invoice data is subjected to compliance verification to obtain the compliance verification result; The compliance verification results of the invoice data that pass the verification are processed together with the configured business documents to obtain an invoice knowledge graph. The corresponding invoice data in the invoice knowledge graph is processed for cost aggregation to generate accounting voucher data.

[0005] In one possible implementation, after generating accounting voucher data, the method further includes: Based on the invoice data, the business documents, the compliance verification logs corresponding to the compliance verification, and the accounting voucher data, an electronic accounting archive is generated; The electronic accounting records are stored using blockchain technology.

[0006] In one possible implementation, the multimodal model is a pre-trained model based on the Transformer architecture; The structured text information includes invoice identifier, amount, date, product details and transaction party information; the visual layout information includes text coordinate boxes, table structure and font features; and the semantic association information includes the calculation relationship between products and amounts, and the matching relationship between transaction parties and business types.

[0007] In one possible implementation, determining whether duplicate tickets exist based on the structured text information, the visual layout information, and the semantic association information includes: Based on the invoice identifier in the structured text information, a target primary key is generated; The target primary key is compared with the primary key configured in the global index to obtain the first comparison result; Based on the first comparison result, determine whether the target bill is a duplicate bill; If it cannot be determined whether it is a duplicate ticket based on the first comparison result, then the perceptual hash value of the target ticket image data is determined based on the visual layout information. The perceived hash value is compared with the perceived hash value of the existing ticket image to obtain a second comparison result; Based on the second comparison result, the image similarity is determined by a distance threshold to determine whether the target ticket is a duplicate ticket at the image level; If it is impossible to determine whether a ticket is a duplicate based on the first comparison result and the second comparison result, then based on the semantic association information, extract the combined features of the transacting party name, amount, time and consumption details and generate a semantic hash value. The semantic hash value is matched and compared with the semantic hash values ​​of existing tickets to obtain a third comparison result; Based on the third comparison result, it is determined whether the bill is a duplicate bill at the semantic level.

[0008] In one possible implementation, the compliance verification results of the invoice data that pass are processed along with the configured business documents to obtain an invoice knowledge graph, including: Starting with the compliance-verified invoice data, the business document chain associated with the business identification information is determined based on the business identification information contained therein. The business document chain includes one or more of the following: purchase contract, purchase order, logistics document and warehouse receipt document. Align the invoice data with the key elements of each business document in the business document chain. The key elements include supplier information, product details, quantity, amount, and date. Based on the alignment results, a bill knowledge graph centered on transaction events is constructed, in which the bill data, purchase contracts, purchase orders, logistics documents, and warehousing documents serve as entity nodes in the graph, and the alignment relationships of the key elements serve as relational edges connecting the entity nodes.

[0009] In one possible implementation, the alignment relationship includes: belonging, corresponding, and containing.

[0010] In one possible implementation, the ticket data is preprocessed to obtain target ticket image data, including: A GAN-based image enhancement model is used to process image-type ticket files to obtain denoised high-resolution image data. By using perspective transformation technology, angle deviation correction processing is performed on the denoised high-resolution image data to obtain angle-corrected image data; Image layer separation and feature extraction techniques are used to process the angle-corrected image data to obtain image data for restoring the occluded text; By using an image orientation detection algorithm, orientation recognition and rotation processing are performed on the image data of the recovered occluded text to obtain the target ticket image data.

[0011] Secondly, a multi-source ticket processing apparatus is provided, which may include: The acquisition unit is used to acquire invoice data from multiple heterogeneous data sources, including image-based invoice files and structured electronic invoice files; The processing unit is used to preprocess any ticket data to obtain target ticket image data. The extraction unit is used to input the target ticket image data into a multimodal model for feature extraction, thereby obtaining the structured text information, visual layout information, and semantic association information of the ticket data; The determining unit is used to determine whether there are duplicate tickets based on the structured text information, the visual layout information, and the semantic association information; The verification unit is used to perform compliance verification on the invoice data when it is determined that there are no duplicate invoices, and obtain the compliance verification result; The processing unit is also used to process the invoice data with the compliance verification result of passing and the configured business documents to obtain an invoice knowledge graph; In addition, the corresponding invoice data in the invoice knowledge graph is processed for cost aggregation to generate accounting voucher data.

[0012] Thirdly, an electronic device is provided, which includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements any of the steps described in the first aspect above.

[0013] Fourthly, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements the steps of any of the methods described in the first aspect above.

[0014] This application provides a method for processing multi-source invoices. The method includes: acquiring invoice data from multiple heterogeneous data sources; preprocessing any invoice data to obtain target invoice image data; inputting the target invoice image data into a multimodal model for feature extraction to obtain structured text information, visual layout information, and semantic association information of the invoice data; determining whether duplicate invoices exist based on the structured text information, visual layout information, and semantic association information; when no duplicate invoices are determined to exist, performing compliance verification on the invoice data to obtain a compliance verification result; processing the invoice data with passed compliance verification results and configured business documents to obtain an invoice knowledge graph; and performing expense aggregation processing on the corresponding invoice data in the invoice knowledge graph to generate accounting voucher data. This application can significantly improve invoice processing efficiency, eliminate duplicate and fraudulent reimbursements, strengthen compliance and risk control capabilities, achieve deep integration of business and finance, reduce file management costs, and ensure the credibility and traceability of invoices throughout the entire process. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating a method for processing multi-source invoices provided in this application embodiment; Figure 2A schematic diagram of the structure of the multi-source document processing apparatus provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0018] The sources of existing invoices are complex, encompassing various forms such as paper-based special / general invoices, PDF electronic invoices, OFD digital invoices, and screenshots of taxi and restaurant receipts. Current corporate invoice management faces numerous pain points: manual entry by employees and manual verification by finance departments lead to low data collection efficiency; electronic invoices can be copied infinitely, making it difficult to prevent duplicate reimbursements such as altered numbers; finance departments need to manually compare contracts, orders, and invoices across systems, leading to errors in document verification and difficulty in identifying fraudulent transactions; the identification of sensitive words in product details and the monitoring of supplier blacklists are lagging, resulting in high compliance risks; mixed paper and electronic storage makes document retrieval inconvenient, and electronic documents are easily tampered with or damaged. In existing technologies, traditional reimbursement systems rely on manual input or simple QR code scanning, unable to handle non-standard invoices and with limited deduplication methods; independent OCR services can only convert images to text, lacking business logic understanding and authenticity verification capabilities; traditional image management systems focus on storage, lacking in-depth compliance checks and document association functions, making it difficult to meet the needs of enterprises for efficient and compliant invoice management.

[0019] Therefore, this application provides a method for processing multi-source invoices, which solves the above-mentioned problems existing in the prior art, can significantly improve the efficiency of invoice processing, realize deep integration of business and financial data, reduce the cost of archive management, and ensure the reliability of electronic archives throughout their entire life cycle.

[0020] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application. Furthermore, the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0021] Figure 1 This is a flowchart illustrating a method for processing multi-source invoices provided in an embodiment of this application. Figure 1 As shown, the method may include: Step S110: Obtain ticket data from multiple heterogeneous data sources.

[0022] The invoice data includes image-based invoice files and structured electronic invoice files; Image-based invoices include scanned or photographed copies of paper invoices (VAT special / general invoices, fixed-amount invoices, toll invoices, etc.), taxi receipts, restaurant receipts, overseas invoices (Invoice / Receipt), and screenshots of electronic invoices; Structured electronic invoice files include fully electronic invoices in XML / OFD format, electronic invoices in PDF format, bank receipts, and electronic order and invoice association data exported from third-party consumption platforms.

[0023] For paper receipts and image-based receipts, two acquisition modes are used to adapt to different usage scenarios: A. Batch Acquisition Mode: By connecting to the system through a high-speed scanner, batch scanning and acquisition of stacked paper documents can be achieved, directly generating high-definition image data; B. Mobile Acquisition Mode: Single-point acquisition is achieved through the high-speed shooting function integrated in the mobile app. The app has a built-in edge detection algorithm that can automatically identify the edges of the ticket against a cluttered background and perform automatic cropping. It also supports perspective transformation technology to correct the angle deviation caused by tilted shooting and automatically turns on the flash to eliminate shooting shadows, ensuring image clarity.

[0024] For various electronic document formats, data collection without human intervention is achieved through multi-interface adaptation, specifically including: A. API Direct Data Collection: Connects to the VAT invoice integrated service platform and enterprise tax digital account to automatically retrieve XML / OFD format data of input and output invoices, ensuring the integrity of the source data of electronic invoices; at the same time, it collects related business document data such as bank receipts, purchase contracts, purchase orders, and warehouse receipts through direct bank-enterprise connection interface and ERP / SRM system interface. B. Email / SMS Parsing and Collection: The system has a built-in IMAP / POP3 email capture tool that automatically monitors the designated finance invoice receiving email address, parses PDF / OFD format invoices in the email in real time, or extracts invoice download links from the email body and completes automatic download; for invoice links contained in SMS messages, RPA robots simulate manual click operations to realize link parsing and invoice file download; C. Third-party platform integration and data collection: By connecting to mainstream consumer platforms through API interfaces, electronic orders and corresponding electronic invoices generated by employee consumption can be directly obtained, enabling automatic collection without the need for invoices; D. Synchronized Data Collection from Card Wallet: Connects to WeChat and Alipay's electronic invoice card wallet, allowing employees to synchronize electronic invoices stored in their card wallet to the system with a single click, simplifying the personal invoice submission process.

[0025] Step S120: For any ticket data, preprocess the ticket data to obtain the target ticket image data.

[0026] Specifically, step 1 involves using a GAN-based image enhancement model to process image-based ticket files, obtaining denoised high-resolution image data. This step may include: learning the normal texture features of ticket images through the model, accurately removing random noise generated in low-light environments and ambient reflection noise during shooting, and avoiding noise interference with subsequent text recognition; for low-resolution, blurry ticket images, generating high-resolution reconstructed images through the model to restore key details such as ticket text, numbers, and table lines, improving the accuracy of subsequent OCR recognition, and finally obtaining denoised high-resolution image data.

[0027] Step 2: Using perspective transformation technology, perform angle deviation correction processing on the denoised high-resolution image data to obtain angle-corrected image data. This step may include: calculating the tilt angle through image geometric transformation algorithm, adjusting the pixel coordinate mapping of the distorted image based on the principle of perspective projection, correcting the horizontal and vertical deviations of the image, so that the content of the ticket is in a standard positive layout, and obtaining angle-corrected image data to ensure that the ticket elements (text, tables, seals) are arranged neatly.

[0028] Step 3: Using image layer separation and feature extraction techniques, process the angle-corrected image data to obtain image data that restores the occluded text. This step may include: using image feature separation algorithms to identify and peel off the transparent watermark layer in the ticket image to avoid the watermark from obscuring and interfering with text recognition; for ticket images generated by shooting electronic screens, using texture feature extraction and filtering algorithms to remove moiré interference from the screen display and restore a clear ticket surface; using color separation techniques or deep learning models to accurately identify and extract the red stamp layer in the image, peel off the stamp from obscuring the text on the ticket surface, restore the covered text information, and obtain clear image data without interference and restoring the occluded text.

[0029] Step 4: Using an image orientation detection algorithm, perform orientation recognition and rotation processing on the image data of the recovered occluded text to obtain the target ticket image data. This step may include: the image orientation detection algorithm automatically identifies the text arrangement direction of the image (forward, inverted, 90° left / right rotation), and adaptively rotates the image based on the orientation recognition results; all images are uniformly rotated to a forward layout to ensure that the reading direction of the text, tables, and key elements on the ticket is consistent, resulting in a neat, clear, and interference-free target ticket image data.

[0030] In some embodiments, a lightweight preprocessing method is used for structured electronic invoices such as XML / OFD format electronic invoices and PDF format electronic invoices: It directly extracts the original ticket image data from the structured electronic ticket file without any additional enhancement or correction processing, ensuring the original integrity of the ticket information; Synchronously extract the structured metadata (such as invoice code, number, amount, and other native fields) from the file to cross-validate with the structured information extracted by the subsequent multimodal model, thereby improving data accuracy; The original image data is used as the target image data for this type of ticket, and the metadata that is synchronously associated and retained is used for subsequent processing.

[0031] This step, through a targeted layered processing scheme, comprehensively solves the core problems of image-based invoices, such as noise, tilt, occlusion, and interference, while also taking into account the high-efficiency processing requirements of structured electronic invoices. It ensures that all heterogeneous invoice data is transformed into target invoice image data of a unified standard, providing high-quality and highly reliable data support for subsequent core steps such as structured extraction and multi-dimensional deduplication. This meets the requirement that it can be achieved by technical personnel in the relevant technical field without creative effort.

[0032] Step S130: Input the target ticket image data into a multimodal model for feature extraction to obtain the structured text information, visual layout information and semantic association information of the ticket data.

[0033] Among them, the multimodal model is a pre-trained model based on the Transformer architecture; Structured text information includes invoice identifier, amount, date, product details and transaction party information; visual layout information includes text coordinate boxes, table structure and font features; semantic association information includes the calculation relationship between products and amounts, and the matching relationship between transaction parties and business types.

[0034] The multimodal model simultaneously receives three types of core information: textual information, layout information, and visual information of the target invoice image data. This model does not rely on fixed invoice templates and has a strong generalization ability, which can automatically adapt to invoice formats in different provinces and industries, as well as non-standard invoices with various formats.

[0035] Specifically, the target ticket image data is standardized according to the model requirements, including: segmenting and encoding the text initially recognized by OCR, converting the coordinate frame information into a normalized format that the model can recognize, and extracting and normalizing the visual features at the pixel level to ensure the synergy of the three types of input information.

[0036] Using fixed keywords on the invoice (such as invoice code, total amount, Tax, Total, Date, etc.) as anchor points, the semantic meaning of the values ​​around the anchor points is deduced based on the positional relationship of the coordinate frame. For example, by using the position of the keyword "total amount", the corresponding amount value to its right or below can be accurately located and extracted. By leveraging pre-trained entity classification capabilities, the information extracted from the invoice is automatically categorized into predefined entity types, covering core entities such as invoice identifiers (invoice code, number, verification code), amount information (amount excluding tax, tax amount, total amount), date information (invoice date, transaction date), transaction party information (seller's name, tax number, address, bank account holder; buyer's name, tax number, address, bank account holder), and commodity details (commodity name, specifications, unit, quantity, unit price, tax rate, amount).

[0037] For invoices with table structures (such as the commodity details column of VAT invoices and purchase invoices), the model combines the visual features of table lines with the positional relationship of coordinate boxes, and adopts a table structure parsing algorithm to accurately restore the correspondence of each field in the commodity details row. This includes adaptive processing of cross-page tables and misaligned tables, ensuring one-to-one correspondence of fields such as quantity, unit price, amount, and tax rate, and avoiding extraction errors caused by field misalignment.

[0038] To further improve the accuracy of feature extraction, the model integrates a business rule post-processing mechanism, which verifies and corrects the initial extraction results through preset financial business logic constraints: Based on the financial logic of Amount = Unit Price × Quantity, Tax Amount = Amount Excluding Tax × Tax Rate, and Total Amount = Amount Excluding Tax + Tax Amount, the system automatically verifies and corrects minor numerical errors that occur during the extraction process. For fields with fixed formats such as invoice code, tax number, and date, validation is performed according to the standard format. For example, date format errors are corrected (e.g., consistency between 2024.05.20 and 2024-05-20), and the number of digits and characters of the tax number are validated and corrected.

[0039] Through the above extraction process, three types of interrelated feature information are finally output. The specific content of each type of information is as follows: 1. Structured text information: This information is standardized structured data containing the core content of the invoice, which can be directly used for subsequent data comparison and verification. Specifically, it includes: Invoice identification information: invoice code, invoice number, verification code, invoice type (VAT special invoice / general invoice, fixed-amount invoice, overseas invoice, etc.); Amount information: amount excluding tax, tax amount, total amount including tax, and individual item amount in the detailed line; Date information: invoice date, transaction date (for small invoices); Transaction party information: seller's name, seller's tax ID, seller's address and bank, seller's account number; buyer's name, buyer's tax ID, buyer's address and bank, and buyer's account number; Commodity details information: commodity / service name, specifications, unit, quantity, unit price, tax rate, tax amount, and detailed remarks.

[0040] 2. Visual layout information, which reflects the spatial distribution characteristics of the document elements, provides support for image fingerprint calculation and document format recognition. Specifically, it includes: text coordinate frame information: the 2D coordinate positions (x1, y1, x2, y2) corresponding to each structured text information, clarifying the specific position of the field in the document image; table structure information: the coordinates of the table borders, the number of rows, the number of columns, and the correspondence between the detail rows and the table header; font feature information: the font size, font weight (bold / regular), font color (e.g., red for total price and tax, black for detail text), line spacing, and column spacing.

[0041] 3. Semantic Relationship Information: This information reflects the logical relationships and business meanings between the fields of the invoice, providing a basis for semantic deduplication and business document reconciliation. Specifically, it includes: Numerical calculation relationships: In the commodity details line, quantity × unit price = single item amount, single item amount × tax rate = single item tax amount, and the logical relationship of total single item amount = total amount excluding tax, total single item tax amount = total tax amount, and total amount excluding tax + total tax amount = total price and tax; Matching relationship between the transaction party and the business type: Semantic matching relationship based on the seller's industry attributes (such as catering enterprises, technology companies) and commodity details (such as catering services, software products). For example, when the seller is XX Catering Management Co., Ltd., the commodity details should be related to catering services; Relationship between field format and business attributes: For example, the remarks column corresponding to the construction service invoice should include project name and address information, and the amount corresponding to the real estate leasing invoice has an implicit relationship with the lease term.

[0042] When outputting structured text information, visual layout information, and semantic association information, the multimodal model simultaneously calculates a confidence score for each extracted field (based on the matching degree between the model's predicted probability and business rules). When the confidence score of a core field (such as amount, invoice code, or transaction party's tax number) is lower than a preset threshold (e.g., 0.9), the field is marked as pending review and pushed to the manual verification interface. The interface uses a highlighted contrast design, displaying the target invoice image on the left with the corresponding field position selected, and showing the model's extraction results on the right. Finance personnel only need to confirm or correct abnormal fields. After completing the human-machine collaborative review, the feature information is updated and the process proceeds to the subsequent deduplication stage. This step leverages the architectural advantages of the LayoutLMv3 multimodal model, the precise positioning of the anchor point mechanism, the error correction of business rules, and the reliability assurance of human-machine collaboration to achieve high-precision and universal feature extraction for all types of invoices. It solves the technical pain points of traditional OCR such as low recognition rate for non-standard invoices, field misalignment, and lack of logical verification. The output structured text information, visual layout information, and semantic association information fully cover the core data requirements of subsequent steps.

[0043] Step S140: Based on structured text information, visual layout information, and semantic association information, determine whether there are duplicate tickets.

[0044] Specifically, this step employs a three-level deduplication funnel mechanism, sequentially verifying structured text information, visual layout information, and semantic association information to ensure that duplicate tickets are intercepted without omission, as detailed below: Level 1: Metadata deduplication based on structured text information. For invoices with official unique identifiers (such as VAT special invoices / general invoices, train tickets, air tickets, XML / OFD format electronic invoices, bank receipts, etc.), accurate deduplication is achieved by relying on the unique identifier.

[0045] Generate the target primary key based on the invoice identifier in the structured text information; The target primary key is compared with the primary key configured in the global index to obtain the first comparison result; Based on the first comparison result, determine whether the target bill is a duplicate bill; The above process can be understood as follows: extracting the unique identifier combination of the invoice from the structured text information to generate a globally unique primary key; for invoice-type invoices, extracting the invoice code and invoice number as the core unique identifier; for train / airline tickets, extracting the train / flight number, date, and seat / ticket number as the unique primary key; for bank receipts, extracting the receipt number, transaction date, and amount as the unique primary key. The generated target primary key is precisely matched with the invoice primary key already stored in the system's global index library to establish a global deduplication mechanism that spans expense reports, employees, and accounting periods, ensuring that there are no duplicate omissions across different scenarios.

[0046] Judgment result: If a completely identical unique primary key exists in the global index, it is directly identified as a duplicate invoice, triggering the interception process and providing the user with the reason for the duplicate and the associated reimbursement record (such as the person claiming the expense, the reimbursement time, and the reimbursement number). If there is no matching primary key in the global index, or if the unique identifier of the invoice is missing or ambiguous (such as incomplete numbers due to OCR recognition errors or the invoice lacking an official unique identifier), then the second level of deduplication verification will be performed.

[0047] Level 2: Image fingerprint deduplication based on visual layout information. For scenarios such as invoices without unique identifiers, invoices whose unique identifiers have been tampered with (e.g., invoice numbers modified by Photoshop), and invoices that have been photographed / copied multiple times for reimbursement, deduplication is achieved through the actual image features of the invoices.

[0048] If it cannot be determined whether it is a duplicate ticket based on the first comparison result, the perceptual hash value of the target ticket image data is determined based on the visual layout information. The perceptual hash value is compared with the perceptual hash value of the existing ticket image to obtain the second comparison result; Based on the second comparison result, the image similarity is determined by the distance threshold to determine whether the target ticket is a duplicate ticket at the image level. The above process can be understood as follows: Based on the basic image features in the visual layout information, calculate the perceptual hash value (pHash) of the target ticket image data. The specific steps are: ① Convert the ticket image to grayscale to eliminate color interference; ② Downsample the grayscale image to 32×32 pixels to retain core features; ③ Perform DCT (Discrete Cosine Transform) on the downsampled image to extract low-frequency region features (low-frequency features are not affected by shooting angle or scaling); ④ Use the mean of the DCT coefficients as a threshold to binarize the coefficients and generate a 64-bit binary hash fingerprint. Retrieve the perceptual hash value of the stored tickets from the configured image fingerprint database, and calculate the Hamming distance (a measure of the difference between binary strings; the smaller the distance, the higher the similarity) between the target ticket hash value and the stored hash value.

[0049] If the Hamming distance is less than a preset threshold (e.g., 5), it is determined to be a duplicate ticket at the image level, which is essentially the same ticket in different physical forms (shooting angle, scaling, compression, copying, etc.), triggering the interception process. If the Hamming distance is greater than or equal to the threshold, or if the features of the ticket image are incomplete (such as severe blurring or incompleteness), proceed to the third level of semantic deduplication verification.

[0050] Level 3: Semantic fuzzy deduplication based on semantic association information. For non-standard receipts without official unique identifiers (such as fixed-amount invoices, taxi receipts, restaurant receipts, overseas receipts, etc.), such receipts cannot be accurately determined to be duplicated through identifiers or images. Deduplication must be achieved by relying on the semantic features of the consumption scenario.

[0051] If it is impossible to determine whether a bill is a duplicate based on the first comparison result and the second comparison result, then based on semantic association information, extract the combined features of the transacting party name, amount, time and consumption details and generate a semantic hash value. The semantic hash value is matched and compared with the semantic hash values ​​of existing tickets to obtain the third comparison result; Based on the third comparison result, it is determined whether the bill is a duplicate bill at the semantic level.

[0052] The above process can be understood as: extracting core semantic feature combinations from semantic association information, such as merchant name, consumption amount, transaction date, transaction time (fuzzy to hour), and consumption details. This combination can uniquely represent a single consumption behavior, avoiding misjudgment based on a single feature. Perform hash calculation on the above feature combination to generate a semantic hash value; perform similarity matching between the semantic hash value and the non-standard invoice hash values ​​already stored in the system's semantic hash library, and set a similarity threshold (such as 95%).

[0053] If highly similar semantic hash values ​​are found, it is determined to be a suspected duplicate invoice, triggering an early warning process and prompting finance personnel to check for violations such as group reimbursement or duplicate printing of receipts; If the semantic hash value has no matching item, or the similarity is below the threshold, it is ultimately determined that there are no duplicate tickets, and the process proceeds to the subsequent compliance verification stage.

[0054] Once it is determined that there are no duplicate invoices, the invoice data is subjected to compliance verification to obtain the compliance verification result. This process may specifically include: automatically extracting the four elements of invoice code, invoice number, invoice date, and amount / verification code from the structured text information through the configured verification platform interface adapter, and then performing real-time verification to obtain the authenticity conclusion (genuine invoice / fake invoice) and current status (normal / void / red-inked / out of control). The detailed information on the invoice (product name, specifications, amount, tax rate, etc.) extracted by the multimodal model is compared word by word with the returned full invoice information to accurately identify fraudulent behaviors such as altering the amount or changing the product name (e.g., changing a watch to office supplies), ensuring that the invoice information has not been tampered with.

[0055] The seller's tax ID in the structured text information is matched with a preset blacklist database, which includes a list of tax defaulters, absconding companies, and a list of non-compliant suppliers defined internally by the company. If a match is found, the invoice is marked as high-risk and blocked. In some embodiments, a federated learning framework is used to achieve cross-entity data sharing and model training for sensitive word databases and supplier blacklist databases. Each participating party (such as a subsidiary of a group) retains the original invoice data and risk control rules locally, and only uploads the model training gradient updates to the central server. After aggregation, a global risk control model is generated and distributed to each participating party. When a participating party discovers a new non-compliant product variant or a risky supplier, other participating parties can immediately identify the risk through the updated global model, achieving group-level joint prevention and control, and avoiding the risk control lag problem caused by data silos of a single enterprise.

[0056] Based on a pre-set database of sensitive product keywords (such as gift cards, shopping cards, cigarettes, Moutai, gold, game consoles, SPA, KTV, etc.), NLP semantic word segmentation and matching are performed on the product details row. Once sensitive words are identified, the risk of violation is immediately marked and pushed to manual review.

[0057] Verify the consistency between the buyer's name in the invoice header and the company's default standard name, and support error handling (such as differences in full-width / half-width brackets, differences in regional prefixes); at the same time, verify the accuracy of the number of digits and characters of the buyer's tax number, and directly block it if there are name mismatches or incorrect tax numbers; In some embodiments, based on a preset mapping library of business types and rules for remarks columns, the system automatically verifies whether the remarks column for invoices corresponding to specific businesses such as construction services, real estate leasing, and cross-county / city operations is filled with required information such as project name, project address, and cross-regional operation identifier as required. If the information is not filled in or is incomplete, it is directly marked as failing the compliance verification and the violation type is clearly indicated as missing information in the remarks column.

[0058] If all the above verification steps pass, a compliance verification result is generated, and the invoice data enters the subsequent business document association and reconciliation step. If any verification step fails (such as fake ticket, invalid ticket, tampered ticket, sensitive word violation, incorrect header, etc.), a compliance verification failure result will be generated, clearly indicating the type and reason of the violation, triggering the interception process, and allowing users to view the violation details and apply for review.

[0059] This step addresses the pain point of traditional deduplication methods that rely solely on invoice numbers and cannot handle tampered or non-standard invoices through a three-level deduplication mechanism, achieving a near 100% duplicate identification rate. The compliance verification process utilizes online tax verification, full invoice comparison, and multi-dimensional rule filtering to proactively intercept fraudulent invoices and irregular reimbursements.

[0060] Step S150: Process the invoice data that pass the compliance verification and the configured business documents to obtain the invoice knowledge graph.

[0061] Specifically, starting with compliance-verified invoice data, the query process determines the business document chain associated with the business identification information. This business document chain includes one or more of the following: purchase contracts, purchase orders, logistics documents, and warehousing documents. This step may include extracting core business identifiers from the structured text information of the invoice data, including but not limited to purchase order numbers, contract numbers, project numbers, and supplier codes. These identifiers are the key links between related business documents, ensuring a unique match between the document chain and the corresponding transaction event. Based on the extracted business identification information, the system connects to business systems such as ERP, SRM, logistics management systems, and warehouse management systems via API interfaces to automatically retrieve and extract associated business documents, forming the business document chain. The business document chain includes at least one or more of the following: purchase contracts, purchase orders (POs), logistics documents, and warehousing documents / acceptance slips, and the document type can be flexibly adapted according to the enterprise's business scenario (e.g., procurement or service). The system performs an integrity check on the retrieved business documents. If core business documents (such as purchase orders and warehouse receipts) are missing, the system automatically marks the document chain as incomplete and triggers an alert, prompting the user to supplement the related documents before proceeding to the next processing stage. If the documents are complete, the system proceeds to the key element alignment step.

[0062] Align the invoice data with the key elements of each business document in the business document chain. Key elements include supplier information, product details, quantity, amount, and date. Specifically, this step may include extracting key elements from both the invoice data and each business document. Key elements may include supplier information, product details, quantity and amount information, time information, and business attribute information. For scenarios where key element descriptions differ across documents (e.g., the product name is Lenovo laptop in the contract, described as a portable computer in the invoice, and described as a laptop in the warehouse receipt), the BERT semantic similarity algorithm is used to calculate the matching degree between elements, determining whether they point to the same entity (supplier, product), thus achieving entity consistency across documents. Next, the supplier information of the invoices is compared with that of each business document to ensure consistency and that the time information conforms to the business logic (e.g., the warehousing date is later than the logistics receipt date, and the invoice issuance date is later than the warehousing date). Following the logic of matching three documents (purchase order, warehousing order, and invoice) or five documents (new contract and logistics order), the system verifies that the quantity of the purchase order is greater than or equal to the quantity of the warehousing order and the quantity of the invoice, and that the unit price is consistent (allowing a preset small error threshold, such as a difference of 0.01 yuan). The total amount satisfies the logical constraints that the total invoice amount = the sum of the itemized amounts + the tax amount and the total invoice amount ≤ the contract amount. If the deviation of key elements is within the preset threshold, it is automatically determined that the alignment has passed. If the deviation exceeds the threshold (e.g., the quantity difference is more than 10% or the amount difference is more than 50 yuan), manual review is triggered, a difference report is generated, and the inconsistent items are marked.

[0063] Based on the aligned results, a knowledge graph of invoices centered on transaction events is constructed. Invoice data, purchase contracts, purchase orders, logistics documents, and warehousing documents serve as entity nodes in the graph, and the alignment relationships of key elements are represented as relational edges connecting these entity nodes. These alignment relationships include: belonging to, corresponding to, and containing. This step may include: setting all core objects involved in the transaction as entity nodes, specifically including: document-type nodes: invoice data (invoices), purchase contracts, purchase orders (POs), logistics documents, and warehousing documents / acceptance slips; business object-type nodes: suppliers, goods / services (a unified product entity after entity alignment); each entity node is associated with its complete structured information (such as the invoice code, contract signing date, and product specifications), which serves as node attributes. Based on the alignment of key elements, association edges are established between entity nodes. Furthermore, the belonging relationship is used to characterize the attribution relationship between documents and transaction events or business objects. For example, invoice data belongs to the transaction event corresponding to a certain purchase contract, and a commodity entity belongs to the purchase scope of a certain purchase order. The correspondence relationship is used to characterize the direct association relationship between different documents. For example, invoice data corresponds to a certain purchase order, logistics documents correspond to a certain inbound document, and purchase orders correspond to a certain purchase contract. The inclusion relationship is used to characterize the inclusion relationship between documents and business objects. For example, a purchase contract includes a certain commodity entity, invoice data includes a certain itemized commodity detail, and a purchase order includes a certain supplier information.

[0064] Map construction process: Input the entity nodes such as invoice data, business documents, suppliers, and goods, along with their attribute information, into the graph database; Based on the element alignment results, corresponding belonging, corresponding and containing relationship edges are established for each entity node to form a closed-loop document chain with a single transaction event as the core. It supports the identification and construction of complex N:M relationships, such as one order corresponding to multiple invoices or multiple orders combined into one invoice. Through the subgraph isomorphic matching algorithm, it ensures the integrity and accuracy of the transaction chain.

[0065] This step constructs a knowledge graph using a graph database, combining BERT semantic similarity entity alignment technology, flexible three-document / five-document matching logic, and deviation tolerance mechanism. This solves the technical pain points of traditional document comparison where inconsistent fields cannot be matched and complex business relationships are difficult to associate. The constructed invoice knowledge graph enables full-link visualization and traceability of transaction events.

[0066] Step S160: Perform cost collection processing on the corresponding invoice data in the invoice knowledge graph to generate accounting voucher data.

[0067] Specifically, for scenarios where invoice data corresponds to multiple cost centers, departments, or projects, a dual-mode allocation mechanism of rule pre-setting and semantic matching is adopted: For shared expenses such as rent, utilities, and centrally procured office supplies, preset allocation rules are supported. The splitting logic can be configured according to the number of people in the department, office area, budget ratio, etc., and the total cost can be accurately allocated to the corresponding cost center. Using NLP semantic analysis technology, the expense report can be parsed (such as business trip for Project A, server purchase for Project B) or the project association information in the item details of the invoice, and the expenses can be automatically aggregated to the corresponding project number without manual matching. Verify the consistency between the total cost of each dimension after allocation and the total amount of the invoice to ensure that the allocation is without deviation. If any calculation anomalies are found, trigger manual review.

[0068] Then, using XGBoost or BERT classification models, trained with massive amounts of historical voucher data, the mapping relationship between product name, supplier type, reimbursement reason and accounting subject is learned. By taking the product name (e.g., travel expenses, R&D materials), supplier type (e.g., technology company, catering company), reason for reimbursement, and allocation dimension (department / project) of the invoice as input, the model automatically predicts the corresponding debit accounting item (e.g., management expenses and travel expenses, R&D expenditures and material costs, sales expenses and business entertainment expenses, etc.). It automatically identifies tax information in invoices, combines invoice type (special invoice / ordinary invoice) and enterprise tax qualifications to deduce the corresponding taxes payable and VAT payable (input tax) accounts, ensuring tax processing compliance.

[0069] Based on the accounting standards configuration rules, combined with the derived accounting subjects, allocated amounts and tax information, standard debit and credit entries are automatically generated: the debit side is the cost / expense subject + input tax, and the credit side is the accounts payable, bank deposits or other payables, etc. Automatically completes elements such as voucher number, voucher date, number of attachments (number of related invoices and business documents), and summary (integrating key information such as product details and allocation dimensions) to form complete accounting voucher data; The generated accounting voucher data can be directly pushed to the enterprise ERP financial module (such as SAP, Oracle, Yonyou, Kingdee) through the API interface, eliminating the need for manual secondary entry and opening up the automated financial accounting chain.

[0070] During the expense collection process, the enterprise budget management system is simultaneously activated: Based on the allocation results, the budget amount of the corresponding department or project is deducted in real time to ensure that the budget consumption status and expense collection are updated synchronously. If the collected expenses exceed the remaining budget for the corresponding dimension, the system will intercept the expense before the voucher is generated, triggering an alert and indicating insufficient budget, thus achieving pre- and in-process budget control and avoiding over-budget expenditures.

[0071] After generating accounting voucher data, the method also includes: Based on invoice data, business documents, compliance verification logs corresponding to compliance verification, and accounting voucher data, an electronic accounting archive is generated. This step may include: summarizing key data from each stage of invoice processing to form a complete archive data set, including: Core invoice data: original invoice format files (OFD / PDF format), scanned copies (JPG format), and structured metadata (XML / JSON format); Related business documents: electronic images and structured information such as purchase contracts, purchase orders, logistics documents, and warehouse receipts corresponding to the invoices; Compliance verification materials: verification logs, sensitive word filtering records, blacklist screening results, manual review comments, etc., corresponding to compliance verification; Accounting-related data: accounting voucher data, expense allocation details, budget deduction records, etc.

[0072] In accordance with the configured electronic accounting record management specifications, all collected data is packaged into an electronic record package that conforms to the SIP (Submission Information Package) standard to ensure that the record format is compliant. Record core attributes of the archives in XML / JSON format, including archive number, invoice identifier, associated business order number, voucher number, document preparation date, retention period, etc., to facilitate retrieval and management; Store original format files and scanned copies of invoices, business documents, and vouchers to ensure that the files are tamper-proof and can be viewed normally; Organize the relationships between invoices, documents, vouchers, and verification logs according to business logic to form a structured evidence chain directory, which facilitates traceability and verification; Record the entire process of document collection, preprocessing, verification, aggregation and archiving, including the operator, operation time, processing result, etc., to ensure traceability of the process.

[0073] The electronic file package is tested for authenticity, integrity, usability, and security. Authenticity is verified to ensure that the file data has not been tampered with. Integrity is verified to ensure that all related data is complete. Usability is verified to ensure that the file format can be opened and viewed normally. Security is verified to ensure that the file data has not been illegally accessed or tampered with. After the inspection is passed, an electronic signature from the electronic record management system is affixed, clearly indicating the subject and time of record creation.

[0074] Next, blockchain technology is used to preserve the electronic accounting records. This step may include: extracting the hash digest of the electronic record package (including the overall hash of the package and the individual hashes of key documents), and summarizing the records to form operation log summaries of each key node (such as collection time, verification time, voucher generation time, and archiving time); using a cryptographic hash algorithm to calculate the unique hash value of the above data, ensuring that the hash value will change significantly if the data is tampered with, providing a basis for subsequent verification. A consortium blockchain (such as Hyperledger Fabric) is used as the evidence preservation platform to ensure the security and privacy of the evidence preservation data; the generated hash value, file number, evidence preservation timestamp, and other information are packaged and uploaded to the blockchain nodes, and the consensus mechanism is used to achieve synchronous storage across all nodes in the network, forming an immutable evidence preservation record; the evidence preservation number returned by the blockchain is bound to the electronic record package, facilitating quick querying of on-chain evidence preservation records using the evidence preservation number. The system monitors the status of archived documents (e.g., whether they have been reversed or voided). If any abnormal status changes are detected, it immediately links to the blockchain evidence record, generates an anomaly alert, and retains the basis for verification. In tax audits or legal disputes, the system can generate an electronic document authenticity verification report. By comparing the current hash value of the electronic document package with the hash value stored on the blockchain, it verifies whether the document has been tampered with, providing judicial-level evidentiary value. Based on a full-text search engine (such as Elasticsearch), it supports multi-dimensional queries by supplier name, product keywords, amount, document number, evidence number, etc., locating the target electronic document and its corresponding on-chain evidence record in milliseconds.

[0075] This step, through intelligent allocation and subject derivation models, achieves full automation of cost collection and voucher generation, significantly reducing manual operation costs and error rates; electronic records are packaged according to SIP standards and tested to ensure compliance with national archives management standards; blockchain evidence storage technology solves the pain points of electronic records being easily tampered with and lacking legal validity, realizing reliable traceability of the entire life cycle of archives.

[0076] This application provides a method for processing multi-source invoices. The method includes: acquiring invoice data from multiple heterogeneous data sources; preprocessing any invoice data to obtain target invoice image data; inputting the target invoice image data into a multimodal model for feature extraction to obtain structured text information, visual layout information, and semantic association information of the invoice data; determining whether duplicate invoices exist based on the structured text information, visual layout information, and semantic association information; when no duplicate invoices are determined to exist, performing compliance verification on the invoice data to obtain a compliance verification result; processing the invoice data with passed compliance verification results and configured business documents to obtain an invoice knowledge graph; and performing expense aggregation processing on the corresponding invoice data in the invoice knowledge graph to generate accounting voucher data. This application can significantly improve invoice processing efficiency, eliminate duplicate and fraudulent reimbursements, strengthen compliance and risk control capabilities, achieve deep integration of business and finance, reduce file management costs, and ensure the credibility and traceability of invoices throughout the entire process.

[0077] Corresponding to the above method, embodiments of this application also provide a multi-source invoice processing apparatus, such as... Figure 2 As shown, the device includes: The acquisition unit 210 is used to acquire invoice data from multiple heterogeneous data sources, the invoice data including image-type invoice files and structured electronic invoice files; Processing unit 220 is used to preprocess any ticket data to obtain target ticket image data; Extraction unit 230 is used to input the target ticket image data into a multimodal model for feature extraction, and obtain the structured text information, visual layout information and semantic association information of the ticket data; The determining unit 240 is used to determine whether there are duplicate tickets based on the structured text information, the visual layout information, and the semantic association information; The verification unit 250 is used to perform compliance verification on the invoice data when it is determined that there are no duplicate invoices, and obtain a compliance verification result; The processing unit 220 is also used to process the invoice data with the compliance verification result of passing and the configured business documents to obtain an invoice knowledge graph; In addition, the corresponding invoice data in the invoice knowledge graph is processed for cost aggregation to generate accounting voucher data.

[0078] The functions of each functional unit of the multi-source invoice processing device provided in the above embodiments of this application can be implemented through the above method steps. Therefore, the specific working process and beneficial effects of each unit in the multi-source invoice processing device provided in the embodiments of this application will not be repeated here.

[0079] This application also provides an electronic device, such as... Figure 3 As shown, it includes a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other through the communication bus 340.

[0080] Memory 330 is used to store computer programs; When the processor 310 executes the program stored in the memory 330, it performs the following steps: Acquire invoice data from multiple heterogeneous data sources, including image-based invoice files and structured electronic invoice files; For any given invoice data, the invoice data is preprocessed to obtain target invoice image data; The target ticket image data is input into a multimodal model for feature extraction to obtain the structured text information, visual layout information, and semantic association information of the ticket data; Based on the structured text information, the visual layout information, and the semantic association information, determine whether there are duplicate tickets; When it is determined that there are no duplicate invoices, the invoice data is subjected to compliance verification to obtain the compliance verification result; The compliance verification results of the invoice data that pass the verification are processed together with the configured business documents to obtain an invoice knowledge graph. The corresponding invoice data in the invoice knowledge graph is processed for cost aggregation to generate accounting voucher data.

[0081] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0082] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0083] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0084] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0085] The implementation methods and beneficial effects of the various components of the electronic device in the above embodiments for solving the problem can be found in [reference needed]. Figure 1 The steps in the illustrated embodiments are used to implement the electronic device. Therefore, the specific working process and beneficial effects of the electronic device provided in this application will not be repeated here.

[0086] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform a multi-source ticket processing method as described in any of the above embodiments.

[0087] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform a multi-source ticket processing method as described in any of the above embodiments.

[0088] Those skilled in the art will understand that the embodiments in this application can be provided as methods, systems, or computer program products. Therefore, the embodiments in this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments in this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0089] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0090] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0091] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0092] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected," "coupled," or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0093] Although preferred embodiments have been described in this application, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the embodiments in this application are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments in this application.

[0094] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the embodiments of this application and their equivalents, then these modifications and variations are also intended to be included in the embodiments of this application.

Claims

1. A method for processing multi-source invoices, characterized in that, The method includes: Acquire invoice data from multiple heterogeneous data sources, including image-based invoice files and structured electronic invoice files; For any given invoice data, the invoice data is preprocessed to obtain target invoice image data; The target ticket image data is input into a multimodal model for feature extraction to obtain the structured text information, visual layout information, and semantic association information of the ticket data; Based on the structured text information, the visual layout information, and the semantic association information, determine whether there are duplicate tickets; When it is determined that there are no duplicate invoices, the invoice data is subjected to compliance verification to obtain the compliance verification result; The compliance verification results of the invoice data that pass the verification are processed together with the configured business documents to obtain an invoice knowledge graph. The corresponding invoice data in the invoice knowledge graph is processed for cost aggregation to generate accounting voucher data.

2. The method as described in claim 1, characterized in that, After generating accounting voucher data, the method further includes: Based on the invoice data, the business documents, the compliance verification logs corresponding to the compliance verification, and the accounting voucher data, an electronic accounting archive is generated; The electronic accounting records are stored using blockchain technology.

3. The method as described in claim 1, characterized in that, The multimodal model is a pre-trained model based on the Transformer architecture; The structured text information includes invoice identifier, amount, date, product details and transaction party information; the visual layout information includes text coordinate boxes, table structure and font features; and the semantic association information includes the calculation relationship between products and amounts, and the matching relationship between transaction parties and business types.

4. The method as described in claim 1, characterized in that, Based on the structured text information, the visual layout information, and the semantic association information, determining whether duplicate tickets exist includes: Based on the invoice identifier in the structured text information, a target primary key is generated; The target primary key is compared with the primary key configured in the global index to obtain the first comparison result; Based on the first comparison result, determine whether the target bill is a duplicate bill; If it cannot be determined whether it is a duplicate ticket based on the first comparison result, then the perceptual hash value of the target ticket image data is determined based on the visual layout information. The perceived hash value is compared with the perceived hash value of the existing ticket image to obtain a second comparison result; Based on the second comparison result, the image similarity is determined by a distance threshold to determine whether the target ticket is a duplicate ticket at the image level; If it is impossible to determine whether a ticket is a duplicate based on the first comparison result and the second comparison result, then based on the semantic association information, extract the combined features of the transacting party name, amount, time and consumption details and generate a semantic hash value. The semantic hash value is matched and compared with the semantic hash values ​​of existing tickets to obtain a third comparison result; Based on the third comparison result, it is determined whether the bill is a duplicate bill at the semantic level.

5. The method as described in claim 1, characterized in that, The compliance verification results for the invoice data that passed are processed along with the configured business documents to obtain an invoice knowledge graph, including: Starting with the compliance-verified invoice data, the business document chain associated with the business identification information is determined based on the business identification information contained therein. The business document chain includes one or more of the following: purchase contract, purchase order, logistics document and warehouse receipt document. Align the invoice data with the key elements of each business document in the business document chain. The key elements include supplier information, product details, quantity, amount, and date. Based on the alignment results, a bill knowledge graph centered on transaction events is constructed, in which the bill data, purchase contracts, purchase orders, logistics documents, and warehousing documents serve as entity nodes in the graph, and the alignment relationships of the key elements serve as relational edges connecting the entity nodes.

6. The method as described in claim 5, characterized in that, The alignment relationships include: belonging to, corresponding to, and containing.

7. The method as described in claim 1, characterized in that, The invoice data is preprocessed to obtain target invoice image data, including: A GAN-based image enhancement model is used to process image-type ticket files to obtain denoised high-resolution image data. By using perspective transformation technology, angle deviation correction processing is performed on the denoised high-resolution image data to obtain angle-corrected image data; Image layer separation and feature extraction techniques are used to process the angle-corrected image data to obtain image data for restoring the occluded text; By using an image orientation detection algorithm, orientation recognition and rotation processing are performed on the image data of the recovered occluded text to obtain the target ticket image data.

8. A multi-source document processing device, characterized in that, The device includes: The acquisition unit is used to acquire invoice data from multiple heterogeneous data sources, including image-based invoice files and structured electronic invoice files; The processing unit is used to preprocess any ticket data to obtain target ticket image data. The extraction unit is used to input the target ticket image data into a multimodal model for feature extraction, thereby obtaining the structured text information, visual layout information, and semantic association information of the ticket data; The determining unit is used to determine whether there are duplicate tickets based on the structured text information, the visual layout information, and the semantic association information; The verification unit is used to perform compliance verification on the invoice data when it is determined that there are no duplicate invoices, and obtain the compliance verification result; The processing unit is also used to process the invoice data with the compliance verification result of passing and the configured business documents to obtain an invoice knowledge graph; In addition, the corresponding invoice data in the invoice knowledge graph is processed for cost aggregation to generate accounting voucher data.

9. An electronic device, characterized in that, The electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-7.

Images