Invoice verification method and device, computer device, and storage medium
By using multimodal data fusion and blockchain technology, and leveraging a pre-trained visual language model to generate confidence information for invoice images and text, the accuracy and efficiency issues of existing invoice verification methods are resolved, achieving efficient and reliable invoice verification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA PING AN PROPERTY INSURANCE CO LTD
- Filing Date
- 2026-03-06
- Publication Date
- 2026-06-12
Smart Images
Figure CN122200686A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and specifically to an invoice verification method, apparatus, computer equipment, and computer-readable storage medium. Background Technology
[0002] Currently, in today's business transactions and financial management, invoices serve as crucial proof of transactions, and their authenticity and accuracy are essential for tax compliance, financial auditing, and business operations. However, with the increasing use of invoices and the continuous advancement of forgery techniques, invoice forgery has become a growing problem, posing significant challenges to businesses and tax authorities. Traditional invoice verification methods primarily rely on manual review and simple database queries, which are not only inefficient but also prone to errors and ill-suited to dealing with complex invoice forgery techniques.
[0003] In the healthcare sector, invoice verification is equally crucial. Medical invoices not only affect patients' medical expense reimbursement but are also closely related to the management and use of medical insurance funds. Forged medical invoices can lead to the loss of medical insurance funds, harm patients' legitimate rights and interests, and create management difficulties for medical institutions and medical insurance departments. In the fintech sector, invoice verification is a vital component of financial transactions and risk management. Financial institutions need to verify the authenticity of relevant invoices when processing loans, insurance claims, and investments to ensure the legality and compliance of transactions. Forged invoices can lead to financial losses for financial institutions and affect the stability of the financial market. However, currently, whether in healthcare, fintech, or other application areas, existing invoice verification methods need improvement in both accuracy and efficiency, failing to meet users' actual needs.
[0004] Therefore, how to provide an invoice verification method, apparatus, computer equipment, and computer-readable storage medium that can effectively improve the accuracy and efficiency of invoice verification is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide an invoice verification method, apparatus, computer device and computer-readable storage medium, aiming to solve the problem of how to effectively improve the accuracy and efficiency of invoice verification.
[0006] To achieve the above objectives, the present invention adopts the following technical solution:
[0007] In a first aspect, the present invention provides an invoice verification method, comprising: Obtain the invoice verification request input by the target user, parse the content of the invoice verification request, and extract the text data and the invoice image data to be verified. The text data is preprocessed to generate target text, and the invoice image data is preprocessed to generate target invoice image. Optical character recognition technology is used to perform text recognition on the target invoice image to obtain the invoice text information of the invoice image data. Based on the target text and the target invoice image, a first confidence information of the target identifier in the target invoice image is generated using a pre-trained visual language large model, and a second confidence information of the invoice text information is generated using the visual language large model based on the target text and the invoice text information. Based on the first confidence information and the second confidence information, a verification result of the invoice image data is generated according to a preset invoice authenticity judgment strategy, and the verification result is stored on the blockchain using blockchain technology.
[0008] Secondly, the present invention provides an invoice verification device, comprising: The acquisition module is used to acquire the invoice verification request input by the target user, parse the content of the invoice verification request, and extract the text data and the invoice image data to be verified. The preprocessing module is used to perform text preprocessing on the text data to generate target text, and to perform image preprocessing on the invoice image data to generate target invoice image. The module also uses optical character recognition technology to perform text recognition on the target invoice image to obtain the invoice text information of the invoice image data. The first generation module is used to generate first confidence information of the target identifier in the target invoice image based on the target text and the target invoice image using a pre-trained visual language large model, and to generate second confidence information of the invoice text information based on the target text and the invoice text information using the visual language large model. The second generation module is used to generate a verification result of the invoice image data based on the first confidence information and the second confidence information, according to a preset invoice authenticity judgment strategy, and to save the verification result on the blockchain using blockchain technology.
[0009] Thirdly, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the invoice verification method as described above.
[0010] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the invoice verification method as described above.
[0011] Compared to existing technologies, this invention provides an invoice verification method, apparatus, computer device, and computer-readable storage medium. The method involves acquiring an invoice verification request input by a target user, parsing the request to extract text data and invoice image data to be verified, performing text preprocessing on the text data to generate target text, and performing image preprocessing on the invoice image data to generate a target invoice image. Optical character recognition (OCR) technology is then used to perform character recognition on the target invoice image to obtain the invoice text information of the invoice image data. Based on the target text and the target invoice image, a pre-trained visual language model is used to generate first confidence information of the target identifier in the target invoice image, and based on the target text and the invoice text information, the visual language model is used to generate second confidence information of the invoice text information. Based on the first and second confidence information, a verification result of the invoice image data is generated according to a preset invoice authenticity judgment strategy, and the verification result is stored on the blockchain using blockchain technology. Therefore, this invention effectively improves the accuracy and efficiency of invoice verification. Attached Figure Description
[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0013] Figure 1 This is a schematic diagram illustrating the application environment of an invoice verification method provided in an embodiment of the present invention.
[0014] Figure 2 This is a flowchart illustrating an invoice verification method according to an embodiment of the present invention.
[0015] Figure 3 This is a schematic diagram of the program modules of an invoice verification device provided in an embodiment of the present invention.
[0016] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention.
[0017] Figure 5 This is another structural schematic diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation
[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0019] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.
[0020] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0021] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [the described condition or event] is detected," or "in response to detection of [the described condition or event]."
[0022] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0023] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.
[0024] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0025] To illustrate the technical solution of the present invention, specific embodiments are described below.
[0026] An embodiment of the present invention provides an invoice verification method that can be applied to, for example... Figure 1 In the application environment shown, the client and server communicate via a network. The client includes, but is not limited to, handheld computers, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, and personal digital assistants (PDAs). The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.
[0027] Please see Figure 2 An embodiment of the present invention provides an invoice verification method, wherein the method includes the following steps: S100: Obtain the invoice verification request input by the target user, parse the content of the invoice verification request, and extract the text data and the invoice image data to be verified. S200. Perform text preprocessing on the text data to generate target text, and perform image preprocessing on the invoice image data to generate target invoice image. Use optical character recognition technology to perform text recognition on the target invoice image to obtain the invoice text information of the invoice image data. S300. Based on the target text and the target invoice image, a first confidence information of the target identifier in the target invoice image is generated using a pre-trained visual language large model, and based on the target text and the invoice text information, a second confidence information of the invoice text information is generated using the visual language large model. S400. Based on the first confidence information and the second confidence information, according to the preset invoice authenticity judgment strategy, a verification result of the invoice image data is generated, and the verification result is stored on the blockchain using blockchain technology.
[0028] In practical implementation, the invoice verification method of this embodiment achieves the technical effect of effectively improving the accuracy and efficiency of invoice verification by comprehensively utilizing a variety of advanced technologies. Specifically: 1. Multimodal Data Fusion: In steps S100 and S200, the method not only extracts the text data input by the user, but also preprocesses and performs OCR text recognition on the invoice image input by the user, obtaining both the image and text information of the invoice. This multimodal data fusion approach allows the verification process to analyze the invoice from multiple perspectives. Compared to methods that rely solely on images, it can more comprehensively capture the features of the invoice, thereby improving the accuracy of the verification.
[0029] 2. Application of Deep Learning Models: In step S300, a pre-trained visual language model is used to generate first confidence information for the target identifier and second confidence information for the invoice text information. The pre-trained visual language model possesses powerful feature extraction and analysis capabilities, enabling it to automatically learn complex patterns in invoice images and text and provide high-confidence judgments. This not only improves the accuracy of verification but also reduces the complexity and uncertainty of manually set rules, thereby enhancing verification efficiency.
[0030] 3. Introduction of Blockchain Technology: In step S400, the verification results are stored on the blockchain using blockchain technology. The immutability and traceability of the blockchain ensure the authenticity and reliability of the verification results, preventing the risk of tampering. Simultaneously, storing the results on the blockchain allows for rapid querying and sharing, improving the transparency and efficiency of the verification process and facilitating subsequent auditing and traceability.
[0031] 4. Preset Invoice Authenticity Verification Strategy: Based on first and second confidence level information, verification results are generated according to the preset invoice authenticity verification strategy. This strategy can be flexibly set according to actual needs, enabling the verification process to adapt to different invoice types and business scenarios, further improving the accuracy and adaptability of verification.
[0032] As described above, this method achieves high accuracy, high efficiency, and high reliability in invoice verification by integrating multimodal data, utilizing the intelligent analysis capabilities of deep learning models, and leveraging the trusted storage of blockchain technology. It effectively solves the problems of low efficiency, error-proneness, and difficulty in dealing with complex forgery methods in traditional invoice verification methods.
[0033] Understandably, the invoice verification method provided in this embodiment of the invention can be applied to invoice verification scenarios related to the medical and health field. The following is a specific example: background: In the healthcare field, patients incur substantial medical expenses during their treatment, which typically require reimbursement through medical insurance or claims from commercial insurance. Invoices serve as crucial documentation for reimbursement and claims, making their authenticity and accuracy paramount. Forged medical invoices can lead to the loss of medical insurance funds, harm patients' legitimate rights, and create management difficulties for medical institutions and medical insurance departments.
[0034] Specific examples: Suppose a patient, Zhang San, underwent wound suturing surgery at the First Hospital of XX City, with a surgery costing 1000.00 yuan. Zhang San needs to claim part of the cost through medical insurance, therefore, the authenticity of the medical invoice needs to be verified. The following is the specific invoice verification process: 1. Obtain invoice verification request: Zhang San uploaded an image of a medical invoice through an invoice verification system on a certain platform, and entered some prompts as text data, such as: "This is the invoice for my wound suturing surgery at the First Hospital of XX City. The invoice number is 123456789, the amount is 1000.00 yuan, and the invoice date is 2025-12-22. Please help me verify the authenticity of the invoice." The system receives an invoice verification request, parses the content, and extracts text data (prompt text) and invoice image data.
[0035] 2. Preprocessing and OCR recognition: The text data is cleaned and standardized to generate the target text. For example, extra spaces are removed and date formats are standardized.
[0036] The invoice image is preprocessed with noise reduction and cropping to generate the target invoice image.
[0037] OCR technology is used to recognize the text in the target invoice image and extract the invoice text information, such as invoice number, amount, invoice date, patient name, treatment items, etc.
[0038] 3. Confidence information generation: By using a pre-trained visual language model, combined with the target text (prompt) and the target invoice image, first confidence information of the target identifier (such as the invoice stamp) is generated.
[0039] By combining the target text and the invoice text information recognized by OCR, a second confidence level of the invoice text information is generated.
[0040] 4. Validation result generation and on-chain storage: Based on a pre-defined invoice authenticity assessment strategy, a verification result is generated by combining the first and second confidence levels. For example, if the overall confidence level is higher than 0.8, the verification result is "true"; otherwise, it is "false" or "requires verification".
[0041] The verification results are structured, hash values are generated, and the verification results are stored on the blockchain using blockchain technology.
[0042] result: The verification results show that the invoice is genuine and valid, with a comprehensive confidence level of 0.95.
[0043] The hash value and detailed information of the verification results are recorded on the blockchain to ensure that the verification results are tamper-proof and traceable.
[0044] Understandably, the invoice verification method provided in this embodiment of the invention can also be applied to invoice verification scenarios related to the fintech field. The following is a specific example: background: In the fintech sector, businesses and individuals typically need to provide invoices as financial documentation when applying for loans, making insurance claims, or investing. Financial institutions need to verify the authenticity and accuracy of these invoices to ensure the legality and compliance of transactions. Forged invoices can lead to financial losses for financial institutions and affect the stability of financial markets.
[0045] Specific examples: Suppose a company, XX Technology Co., Ltd., applies for a loan from a financial institution to purchase server equipment. It needs to submit an invoice from a previous purchase record for 50,000.00 yuan. The following is the specific invoice verification process: 1. Obtain invoice verification request: The company uploaded an invoice image through an invoice verification system on a certain platform and entered some prompts as text data, such as: "This is the invoice we purchased server equipment from XX Sales Company. The invoice number is 987654321, the amount is 50,000.00 yuan, and the invoice date is 2025-12-20. Please help me verify the authenticity of the invoice." The system receives an invoice verification request, parses the content, and extracts text data (prompt text) and invoice image data.
[0046] 2. Preprocessing and OCR recognition: The text data is cleaned and standardized to generate the target text. For example, extra spaces are removed and monetary amounts are formatted correctly.
[0047] The invoice image is preprocessed with noise reduction and cropping to generate the target invoice image.
[0048] OCR technology is used to recognize the text in the target invoice image and extract the invoice text information, such as invoice number, amount, invoice date, supplier name, and purchased items.
[0049] 3. Confidence information generation: By using a pre-trained visual language model, combined with the target text (prompt) and the target invoice image, first confidence information of the target identifier (such as the invoice stamp) is generated.
[0050] By combining the target text and the invoice text information recognized by OCR, a second confidence level of the invoice text information is generated.
[0051] 4. Validation result generation and on-chain storage: Based on a pre-defined invoice authenticity assessment strategy, a verification result is generated by combining the first and second confidence levels. For example, if the overall confidence level is higher than 0.8, the verification result is "true"; otherwise, it is "false" or "requires verification".
[0052] The verification results are structured, hash values are generated, and the verification results are stored on the blockchain using blockchain technology.
[0053] result: The verification results show that the invoice is genuine and valid, with a comprehensive confidence level of 0.92.
[0054] The hash value and detailed information of the verification results are recorded on the blockchain to ensure that the verification results are tamper-proof and traceable.
[0055] The two specific examples above demonstrate the significant application value of the invoice verification method of this invention in the fields of healthcare and fintech. This method effectively improves the accuracy and efficiency of invoice verification through multimodal data fusion, intelligent analysis using deep learning models, and trusted storage via blockchain technology, providing reliable technical support for business processes in related fields.
[0056] Furthermore, in one embodiment, the invoice verification method, wherein obtaining the invoice verification request input by the target user, parsing the content of the invoice verification request, and extracting the text data and the invoice image data to be verified, specifically includes: Obtain the invoice verification request input by the target user through the client, and perform format validation and integrity checks on the invoice verification request; When both format validation and integrity checks pass, the content of the invoice verification request is parsed, and the text data and the invoice image data to be verified in the invoice verification request are obtained based on the parsing results. If the format validation and / or integrity check fails, a prompt message will be returned.
[0057] Furthermore, the invoice verification method, wherein obtaining the invoice verification request input by the target user through the client, and performing format validation and integrity checks on the invoice verification request, specifically includes: The client responds to the target user's interface activation command to open and enter the invoice verification interface. On the invoice verification interface, the identity information of the target user is verified through at least one authentication method, namely biometrics, dynamic verification code or digital certificate; When the target user's identity information is verified, the invoice verification request input by the target user is received through the invoice verification interface, and the format verification and integrity check of the invoice verification request are performed.
[0058] In practice, the specific implementation process of this embodiment is roughly as follows: 1. Command to enable client response interface: Users trigger the interface opening command through a client (such as a web browser, mobile application, etc.) (e.g., clicking the "Invoice Verification" button).
[0059] After the client detects the interface opening command, it sends a request to the server to open the invoice verification interface.
[0060] The server responds to the request, returning the HTML, CSS, and JavaScript code for the invoice verification interface. The client then renders and displays the invoice verification interface.
[0061] 2. User authentication: The invoice verification interface offers users a choice of authentication methods such as biometrics (e.g., fingerprint recognition, facial recognition), dynamic verification codes, or digital certificates.
[0062] Users select one or more authentication methods, and the server calls the corresponding API or the client's local function to perform authentication.
[0063] If authentication is successful, proceed to the next step; if authentication fails, the message "Authentication failed, please try again" will be returned.
[0064] 3. Receive invoice verification request: Users upload an image of the invoice (such as by taking a photo or selecting a local file) and enter the necessary text data (prompts) on the invoice verification interface, which is then packaged into an invoice verification request.
[0065] Perform format validation on invoice verification requests to check whether the requests conform to preset format specifications (such as whether the fields of text data are complete, and whether the format and size of image files meet the requirements).
[0066] At the same time, perform an integrity check to ensure that the request contains all necessary information (such as invoice images and key prompt text).
[0067] 4. Processing of verification and inspection results: If both the format validation and integrity checks pass, the content of the invoice verification request is parsed, and the text data and the invoice image data to be verified are obtained from the parsing results. If the format validation and / or integrity check fails, a prompt message will be returned to the user, indicating the specific error or missing information (such as "Invoice image format is not supported, please upload an image in JPG or PNG format").
[0068] Through the above process, the invoice verification method of the present invention can effectively process user-inputted invoice verification requests and perform strict format verification and integrity checks to ensure the accuracy of the invoice verification process.
[0069] Further, in one embodiment, the invoice verification method, wherein the steps of performing text preprocessing on the text data to generate target text, performing image preprocessing on the invoice image data to generate a target invoice image, and performing character recognition on the target invoice image using optical character recognition technology to obtain the invoice text information of the invoice image data, specifically include: After cleaning, segmenting, and tagging the text data, the target text is generated using a context-aware algorithm. After denoising, image enhancement, and size normalization of the invoice image data, geometric correction is performed based on invoice edge detection and perspective transformation. Then, the effective area of the invoice is extracted through image cropping technology to generate the target invoice image. According to a preset region division strategy, the target invoice image is divided into several invoice sub-regions; Optical character recognition technology is used to perform text recognition on each of the invoice sub-regions to obtain the sub-text information corresponding to each of the invoice sub-regions; Integrate all the sub-text information to generate the invoice text information of the invoice image data.
[0070] In practice, the specific implementation process of this embodiment is roughly as follows: 1. Text data cleaning: Clean the text data entered by the user, remove irrelevant characters (such as extra spaces and special symbols), unify the text encoding format (such as converting all characters to UTF8 format), and correct obvious spelling errors.
[0071] 2. Text data segmentation and part-of-speech tagging: Natural Language Processing (NLP) techniques are used to segment the cleaned text into individual lexical units.
[0072] The word segmentation results are tagged with part-of-speech tags to identify the part of speech of each word (such as noun, verb, number, etc.) in order to better understand the semantic structure of the text.
[0073] 3. Target text generation: By utilizing context-aware algorithms and combining the results of word segmentation and part-of-speech tagging, target text is generated. Context-aware algorithms can intelligently identify and correct ambiguous information in the text based on contextual information, ensuring the accuracy and consistency of the text.
[0074] 4. Image denoising and enhancement: The uploaded invoice image data undergoes preprocessing, starting with denoising to remove random noise and improve image quality. Algorithms such as Gaussian filtering can be used.
[0075] Enhance the image by adjusting contrast, sharpening, etc., to improve its readability.
[0076] 5. Image size normalization and cropping: Adjust the images to a uniform size for subsequent processing.
[0077] Geometric correction is performed based on invoice edge detection and perspective transformation. Then, image cropping technology is used to remove edge parts in the invoice image that are not related to the invoice content, extract the effective area of the invoice, and generate the target invoice image.
[0078] Example: The cropped image contains only the invoice content, with the surrounding blank areas removed.
[0079] 6. Invoice image area division: Based on a preset region segmentation strategy, the target invoice image is divided into several sub-regions. The region segmentation strategy can be based on fixed region segmentation of the invoice template or dynamic region segmentation based on machine learning.
[0080] Example: Divide the target invoice image into areas such as invoice number, amount, and invoice date.
[0081] 7. OCR Text Recognition: Optical character recognition (OCR) technology is used to recognize text in each sub-region of the invoice, thereby obtaining the sub-text information corresponding to each sub-region.
[0082] 8. Subtext information integration: All sub-text information is integrated to generate complete invoice text information. A second verification process can be performed during integration to ensure the accuracy and consistency of the information.
[0083] Through the above process, the invoice verification method of the present invention can effectively preprocess text data and invoice image data, and accurately extract invoice text information using OCR technology. These steps not only improve the accuracy and consistency of the data, but also provide a reliable data foundation for subsequent verification and blockchain storage.
[0084] Further, in one embodiment, the invoice verification method, wherein generating first confidence information of a target identifier in the target invoice image using a pre-trained visual language large model based on the target text and the target invoice image, and generating second confidence information of the invoice text information using the visual language large model based on the target text and the invoice text information, specifically includes: Load a pre-trained large visual language model; The target text and the target invoice image are input into the visual language big data model for identifier confidence recognition processing to generate first confidence information of the target identifier in the target invoice image; wherein, the target identifier includes a QR code and a seal; The target text and the invoice text information are input into the visual language big data model for invoice text confidence recognition processing to generate the second confidence information of the invoice text information.
[0085] In practice, the specific implementation process of this embodiment is roughly as follows: 1. Model loading: Load pre-trained large-scale visual language models on the server side. These models are typically pre-trained on large-scale datasets and are capable of handling joint features of images and text.
[0086] 2. Identifier confidence recognition: The target text and the target invoice image are used as input data and fed into the visual language model for identifier confidence recognition processing.
[0087] The model outputs confidence information for target identifiers (such as QR codes and seals) in the target invoice image.
[0088] 3. Invoice text confidence recognition: The target text and invoice text information are used as input data and fed into a large visual language model to perform confidence recognition processing of invoice text information.
[0089] The model will output confidence information for the invoice text information.
[0090] 4. Confidence information generation: Based on the model's output information, the first confidence level of the target identifier and the second confidence level of the invoice text information are obtained.
[0091] The first and second confidence level information are normalized to ensure that their values are between 0 and 1.
[0092] Through the above process, the invoice verification method of the present invention can effectively utilize a pre-trained visual language model to generate confidence information for target identifiers and invoice text information. This confidence information provides important reference for subsequent invoice authenticity judgment, improving the accuracy and reliability of verification.
[0093] Furthermore, in one embodiment, the invoice verification method, wherein generating a verification result of the invoice image data based on the first confidence information and the second confidence information, according to a preset invoice authenticity judgment strategy, and storing the verification result on the blockchain using blockchain technology, specifically includes: The first confidence information and the second confidence information are fused using a weighted fusion algorithm to generate comprehensive confidence information; Based on a preset invoice authenticity judgment strategy, a verification result of the invoice image data is generated according to the comprehensive confidence information; The verification result is structured, the detailed information of the verification result is extracted, and a unique hash value is generated. Using blockchain technology, the hash value of the verification result and the detailed information are stored on the blockchain.
[0094] Furthermore, the invoice verification method, wherein the step of structuring the verification result, extracting the detailed information of the verification result, generating a unique hash value, and then using blockchain technology to store the hash value and the detailed information of the verification result on the blockchain, further includes: The verification result is formatted according to the configuration information of the client corresponding to the target user. The formatted verification result is sent to the client and displayed to the target user on the client according to a preset result display strategy.
[0095] In practice, the specific implementation process of this embodiment is roughly as follows: 1. Application of weighted fusion algorithm: A weighted fusion algorithm is used to fuse the first and second confidence levels. Based on a preset weight allocation principle, the overall confidence level is calculated.
[0096] For example, assuming the weight of the first confidence level information is 0.6 and the weight of the second confidence level information is 0.4, the calculation formula is: Overall confidence level information = (0.6 × first confidence level information) + (0.4 × second confidence level information).
[0097] 2. Generate verification results based on preset strategies: Based on a pre-defined invoice authenticity assessment strategy, a verification result is generated using comprehensive confidence level information. For example, if a threshold of 0.8 is set, the verification result is "true" if the comprehensive confidence level is higher than 0.8; otherwise, it is "false" or "requires verification".
[0098] 3. Structured processing of validation results: The verification results are structured to extract detailed information such as invoice number, amount, overall confidence level, and verification conclusion.
[0099] 4. Generate hash values and save them on the blockchain: Use a hash algorithm (such as SHA-256) to generate a unique hash value for the structured verification results.
[0100] By using blockchain technology, the hash value and detailed information of the verification results are stored on the chain.
[0101] 5. Adjust the format according to the client configuration: The verification results are formatted according to the configuration information of the target user's client. For example, the text language is adjusted according to the client's language settings, and the data format is adjusted according to display preferences.
[0102] 6. Send and display the verification result: The formatted verification results are sent to the client and displayed to the target user on the client according to the preset result display strategy.
[0103] Through the above process, the invoice verification method of the present invention can not only effectively generate verification results and securely and reliably store them on the blockchain, but also adjust the format of the verification results according to the client's configuration information and display the results in a user-friendly manner. This ensures the accuracy, efficiency, security, and optimized user experience of the verification process.
[0104] As can be seen from the above method embodiments, the invoice verification method provided by the present invention includes: obtaining an invoice verification request input by a target user; parsing the content of the invoice verification request to extract text data and invoice image data to be verified; performing text preprocessing on the text data to generate target text, and performing image preprocessing on the invoice image data to generate a target invoice image; performing character recognition on the target invoice image using optical character recognition technology to obtain invoice text information of the invoice image data; generating first confidence information of the target identifier in the target invoice image using a pre-trained visual language model based on the target text and the target invoice image, and generating second confidence information of the invoice text information using the visual language model based on the target text and the invoice text information; generating a verification result of the invoice image data based on the first confidence information and the second confidence information, according to a preset invoice authenticity judgment strategy, and storing the verification result on the blockchain using blockchain technology. Thus, the method of the present invention can effectively improve the accuracy and efficiency of invoice verification.
[0105] It should be understood that although this application provides the method operation steps as described in the embodiments or flowcharts, conventional or non-inventive labor may include more or fewer operation steps, and these operation steps are not necessarily executed sequentially according to the order of the embodiments or flowcharts. The order of steps listed in the embodiments or flowcharts is merely one way of executing many steps and does not represent the only execution order. It should be noted that there is no necessary sequential order between the above steps. Those skilled in the art can understand from the description of the embodiments of the present invention that the above steps may have different execution orders in different embodiments, that is, they may be executed in parallel or in exchange, etc. Moreover, at least some steps in the embodiments or flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but may be executed in turn, alternately, or synchronously with other steps or at least a part of the sub-steps or stages of other steps.
[0106] Based on the above method embodiments, please refer to Figure 3 Another embodiment of the present invention also provides an invoice verification device, wherein the device includes: The acquisition module 11 is used to acquire the invoice verification request input by the target user, parse the content of the invoice verification request, and extract the text data and the invoice image data to be verified. The preprocessing module 12 is used to perform text preprocessing on the text data to generate target text, and to perform image preprocessing on the invoice image data to generate target invoice image. It also uses optical character recognition technology to perform text recognition on the target invoice image to obtain the invoice text information of the invoice image data. The first generation module 13 is used to generate first confidence information of the target identifier in the target invoice image based on the target text and the target invoice image using a pre-trained visual language large model, and to generate second confidence information of the invoice text information based on the target text and the invoice text information using the visual language large model. The second generation module 14 is used to generate a verification result of the invoice image data based on the first confidence information and the second confidence information, according to a preset invoice authenticity judgment strategy, and to save the verification result on the blockchain using blockchain technology.
[0107] Furthermore, in one embodiment, the invoice verification device, wherein obtaining the invoice verification request input by the target user, parsing the content of the invoice verification request, and extracting the text data and the invoice image data to be verified, specifically includes: Obtain the invoice verification request input by the target user through the client, and perform format validation and integrity checks on the invoice verification request; When both format validation and integrity checks pass, the content of the invoice verification request is parsed, and the text data and the invoice image data to be verified in the invoice verification request are obtained based on the parsing results. If the format validation and / or integrity check fails, a prompt message will be returned.
[0108] Furthermore, in the aforementioned invoice verification device, the step of acquiring the invoice verification request input by the target user through the client and performing format verification and integrity checks on the invoice verification request specifically includes: The client responds to the target user's interface activation command to open and enter the invoice verification interface. On the invoice verification interface, the identity information of the target user is verified through at least one authentication method, namely biometrics, dynamic verification code or digital certificate; When the target user's identity information is verified, the invoice verification request input by the target user is received through the invoice verification interface, and the format verification and integrity check of the invoice verification request are performed.
[0109] Further, in one embodiment, the invoice verification device, wherein the steps of performing text preprocessing on the text data to generate target text, performing image preprocessing on the invoice image data to generate a target invoice image, and performing text recognition on the target invoice image using optical character recognition technology to obtain the invoice text information of the invoice image data, specifically include: After cleaning, segmenting, and tagging the text data, the target text is generated using a context-aware algorithm. After denoising, image enhancement, and size normalization of the invoice image data, geometric correction is performed based on invoice edge detection and perspective transformation. Then, the effective area of the invoice is extracted through image cropping technology to generate the target invoice image. According to a preset region division strategy, the target invoice image is divided into several invoice sub-regions; Optical character recognition technology is used to perform text recognition on each of the invoice sub-regions to obtain the sub-text information corresponding to each of the invoice sub-regions; Integrate all the sub-text information to generate the invoice text information of the invoice image data.
[0110] Further, in one embodiment, the invoice verification device, wherein the step of generating first confidence information of a target identifier in the target invoice image using a pre-trained visual language large model based on the target text and the target invoice image, and generating second confidence information of the invoice text information using the visual language large model based on the target text and the invoice text information, specifically includes: Load a pre-trained large visual language model; The target text and the target invoice image are input into the visual language big data model for identifier confidence recognition processing to generate first confidence information of the target identifier in the target invoice image; wherein, the target identifier includes a QR code and a seal; The target text and the invoice text information are input into the visual language big data model for invoice text confidence recognition processing to generate the second confidence information of the invoice text information.
[0111] Furthermore, in one embodiment, the invoice verification device, wherein the step of generating a verification result of the invoice image data based on the first confidence information and the second confidence information, according to a preset invoice authenticity judgment strategy, and storing the verification result on the blockchain using blockchain technology, specifically includes: The first confidence information and the second confidence information are fused using a weighted fusion algorithm to generate comprehensive confidence information; Based on a preset invoice authenticity judgment strategy, a verification result of the invoice image data is generated according to the comprehensive confidence information; The verification result is structured, the detailed information of the verification result is extracted, and a unique hash value is generated. Using blockchain technology, the hash value of the verification result and the detailed information are stored on the blockchain.
[0112] Furthermore, the invoice verification device, wherein the step of structuring the verification result, extracting the detailed information of the verification result, generating a unique hash value, and then using blockchain technology to store the hash value and the detailed information of the verification result on the blockchain, further includes: The verification result is formatted according to the configuration information of the client corresponding to the target user. The formatted verification result is sent to the client and displayed to the target user on the client according to a preset result display strategy.
[0113] It should be noted that, in the device embodiments of the present invention, the information interaction and execution process between the above modules are based on the same concept as in the method embodiments of the present invention. For details on their specific functions and the resulting technical effects, please refer to the aforementioned method embodiments section, which will not be repeated here.
[0114] Based on the above method embodiments, another embodiment of the present invention also provides a computer device, which can be a server, and its internal structure diagram can be as follows. Figure 4 As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the functions or steps of the invoice verification method server-side as described in any of the above method embodiments.
[0115] Based on the above method embodiments, another embodiment of the present invention also provides a computer device, which can be a client, and its internal structure diagram can be as follows. Figure 5As shown, the computer device includes a processor, memory, network interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements the functions or steps of the invoice verification method on the client side as described in any of the above method embodiments.
[0116] Those skilled in the art will understand that Figure 4 and Figure 5 The structural schematic diagram shown is only a schematic diagram of a part of the structure related to the present invention and does not constitute a limitation on the computer device on which the present invention is applied. The specific computer device may include more components than shown in the figure, or combine certain components, or have different component arrangements.
[0117] The processor referred to herein can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.
[0118] The memory includes readable storage media, internal memory, etc., where internal memory can be the RAM of a computer device. Internal memory provides an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of the computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory can include both internal storage units and external storage devices of the computer device. The memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. The memory can also be used to temporarily store data that has been output or will be output.
[0119] Based on the above method embodiments, another embodiment of the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the invoice verification method as described in any of the above method embodiments. The computer-readable storage medium may be non-volatile or volatile.
[0120] It should be noted that the functions or steps that can be achieved by the computer-readable storage medium or computer device, and the technical effects brought about by the functions / steps, can be referred to the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.
[0121] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc. The disclosed memory components or memories of the operating environment described herein are intended to include one or more of these and / or any other suitable types of memory.
[0122] Those skilled in the art will understand that, for the sake of convenience and brevity, the embodiments of the device of the present invention are only illustrated by the division of the above-mentioned functional units and modules. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of the present invention. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
[0123] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.
[0124] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0125] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0126] It should be noted that if any software tools or components not belonging to this company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use. The above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. An invoice verification method, characterized in that, include: Obtain the invoice verification request input by the target user, parse the content of the invoice verification request, and extract the text data and the invoice image data to be verified. The text data is preprocessed to generate target text, and the invoice image data is preprocessed to generate target invoice image. Optical character recognition technology is used to perform text recognition on the target invoice image to obtain the invoice text information of the invoice image data. Based on the target text and the target invoice image, a first confidence information of the target identifier in the target invoice image is generated using a pre-trained visual language large model, and a second confidence information of the invoice text information is generated using the visual language large model based on the target text and the invoice text information. Based on the first confidence information and the second confidence information, a verification result of the invoice image data is generated according to a preset invoice authenticity judgment strategy, and the verification result is stored on the blockchain using blockchain technology.
2. The invoice verification method according to claim 1, characterized in that, The process of obtaining the invoice verification request input by the target user, parsing the content of the invoice verification request, and extracting the text data and the invoice image data to be verified includes: Obtain the invoice verification request input by the target user through the client, and perform format validation and integrity checks on the invoice verification request; When both format validation and integrity checks pass, the content of the invoice verification request is parsed, and the text data and the invoice image data to be verified in the invoice verification request are obtained based on the parsing results. If the format validation and / or integrity check fails, a prompt message will be returned.
3. The invoice verification method according to claim 2, characterized in that, The step of obtaining the invoice verification request input by the target user through the client and performing format validation and integrity checks on the invoice verification request includes: The client responds to the target user's interface activation command and enters the invoice verification interface. On the invoice verification interface, at least one authentication method selected from biometrics, dynamic verification code, or digital certificate is used to verify the identity information of the target user. When the target user's identity information is verified, the invoice verification request input by the target user is received through the invoice verification interface, and the format verification and integrity check of the invoice verification request are performed.
4. The invoice verification method according to claim 1, characterized in that, The process involves preprocessing the text data to generate target text, preprocessing the invoice image data to generate a target invoice image, and using optical character recognition (OCR) technology to perform character recognition on the target invoice image to obtain the invoice text information of the invoice image data, including: After cleaning, word segmentation, and part-of-speech tagging of the text data, the target text is generated using a context-aware algorithm. After denoising, image enhancement, and size normalization of the invoice image data, geometric correction is performed based on invoice edge detection and perspective transformation. Then, the effective area of the invoice is extracted through image cropping technology to generate the target invoice image. According to a preset region division strategy, the target invoice image is divided into several invoice sub-regions; Optical character recognition technology is used to perform text recognition on each of the invoice sub-regions to obtain the sub-text information corresponding to each of the invoice sub-regions; Integrate all the sub-text information to generate the invoice text information of the invoice image data.
5. The invoice verification method according to claim 1, characterized in that, The step of generating first confidence information of the target identifier in the target invoice image using a pre-trained visual language large model based on the target text and the target invoice image, and generating second confidence information of the invoice text information using the visual language large model based on the target text and the invoice text information, includes: Load a pre-trained large visual language model; The target text and the target invoice image are input into the visual language big data model for identifier confidence recognition processing to generate first confidence information of the target identifier in the target invoice image; wherein, the target identifier includes a QR code and a seal; The target text and the invoice text information are input into the visual language big data model for invoice text confidence recognition processing to generate the second confidence information of the invoice text information.
6. The invoice verification method according to claim 1, characterized in that, The step of generating a verification result for the invoice image data based on the first confidence information and the second confidence information, according to a preset invoice authenticity judgment strategy, and storing the verification result on the blockchain using blockchain technology includes: The first confidence information and the second confidence information are fused using a weighted fusion algorithm to generate comprehensive confidence information; Based on a preset invoice authenticity judgment strategy, a verification result of the invoice image data is generated according to the comprehensive confidence information; The verification result is structured, the detailed information of the verification result is extracted, and a unique hash value is generated. Using blockchain technology, the hash value of the verification result and the detailed information are stored on the blockchain.
7. The invoice verification method according to claim 6, characterized in that, The process of structuring the verification result, extracting detailed information from the verification result, generating a unique hash value, and then storing the hash value and detailed information of the verification result on the blockchain using blockchain technology further includes: The verification result is formatted according to the configuration information of the client corresponding to the target user. The formatted verification result is sent to the client and displayed to the target user on the client according to a preset result display strategy.
8. An invoice verification device, characterized in that, include: The acquisition module is used to acquire the invoice verification request input by the target user, parse the content of the invoice verification request, and extract the text data and the invoice image data to be verified. The preprocessing module is used to perform text preprocessing on the text data to generate target text, and to perform image preprocessing on the invoice image data to generate target invoice image. The module also uses optical character recognition technology to perform text recognition on the target invoice image to obtain the invoice text information of the invoice image data. The first generation module is used to generate first confidence information of the target identifier in the target invoice image based on the target text and the target invoice image using a pre-trained visual language large model, and to generate second confidence information of the invoice text information based on the target text and the invoice text information using the visual language large model. The second generation module is used to generate a verification result of the invoice image data based on the first confidence information and the second confidence information, according to a preset invoice authenticity judgment strategy, and to save the verification result on the blockchain using blockchain technology.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the invoice verification method as described in any one of claims 1-7.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the invoice verification method as described in any one of claims 1-7.