Method and device for carrying out anomaly identification on voucher image

Abstract

The embodiment of the invention provides a method and device for carrying out anomaly recognition on a voucher image, and the method comprises the steps: extracting a plurality of text segments from the voucher image; and inputting the plurality of text segments into the first large model, selecting a plurality of key texts from the plurality of text segments, performing semantic intention recognition on each key text, and determining each corresponding target knowledge base according to a recognition result. And for any first key text in the plurality of key texts, according to a first retrieval mode matched with the corresponding first target knowledge base, retrieving the first target knowledge base to obtain first target knowledge matched with the first key text. And inputting each key text and each corresponding target knowledge into a second large model, performing anomaly detection on each key text based on each target knowledge, and determining an anomaly recognition result for the voucher image according to a detection result.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and more particularly to a method and apparatus for anomaly recognition of voucher images. Background Technology

[0002] In application scenarios such as government and enterprise financial management, invoice compliance review, and tax risk control, the current task of anomaly recognition for voucher images mainly relies on two common technical paths: First, the method of "OCR text extraction + rule engine judgment", which first uses optical character recognition technology to extract key field information from the voucher image, and then performs automated comparison and anomaly screening through preset business rules (such as amount range, invoice number rules, date logic, etc.); Second, the method of visual judgment based directly on tampering traces, which analyzes the physical or digital features of the voucher image itself, such as whether there are anomalies in handwriting, seals, background patterns, erasures, splicing, etc., thereby identifying possible forgery or alteration.

[0003] However, the two methods mentioned above still have certain limitations when dealing with complex and hidden anomalies. For example, the rules are difficult to cover all anomaly patterns, OCR recognition errors affect the accuracy of judgment, and tampering traces are difficult to detect automatically and efficiently. Therefore, there is a need to provide a more efficient and accurate method for anomaly recognition of voucher images. Summary of the Invention

[0004] This specification describes one or more embodiments of a method and apparatus for anomaly identification of voucher images, which can identify anomalies in voucher images more efficiently and accurately.

[0005] Firstly, a method for anomaly identification of voucher images is provided, including:

[0006] Extract multiple text fragments from a voucher image;

[0007] The multiple text fragments are input into the first large model, which selects several key texts from the multiple text fragments, performs semantic intent recognition on each key text, and determines the corresponding target knowledge base based on the recognition results.

[0008] For any first key text among the plurality of key texts, the first target knowledge base is retrieved according to a first retrieval method that matches the corresponding first target knowledge base, and the first target knowledge matching the first key text is obtained.

[0009] Each key text and its corresponding target knowledge are input into the second major model, which then performs anomaly detection on each key text based on the target knowledge. Based on the detection results, the anomaly recognition result for the voucher image is determined.

[0010] Secondly, an apparatus for anomaly detection of voucher images is provided, comprising:

[0011] An extraction unit is used to extract multiple text fragments from a voucher image;

[0012] The input unit is used to input the multiple text fragments into the first large model, so that it selects several key texts from the multiple text fragments, performs semantic intent recognition on each key text, and determines the corresponding target knowledge base based on the recognition results.

[0013] The retrieval unit is used to retrieve the first target knowledge base for any first key text among the plurality of key texts, according to a first retrieval method that matches the corresponding first target knowledge base, and to obtain the first target knowledge that matches the first key text.

[0014] The input unit is also used to input each key text and the corresponding target knowledge into the second model, so that it can perform anomaly detection on each key text based on the target knowledge, and determine the anomaly recognition result for the voucher image based on the detection result.

[0015] Thirdly, a computer storage medium is provided on which a computer program is stored, which, when executed in a computer, causes the computer to perform the method of the first aspect.

[0016] Fourthly, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the method of the first aspect.

[0017] The method and apparatus for anomaly recognition of voucher images provided in one or more embodiments of this specification first utilize a large model to perform semantic intent recognition on each key text included in the voucher image to determine the corresponding target knowledge bases; then, each target knowledge base is retrieved to obtain target knowledge; finally, the large model is used to perform anomaly detection on the corresponding key texts based on the retrieved target knowledge, thereby determining the anomaly recognition result of the voucher image. In other words, this solution can leverage the knowledge and capabilities of a large model to perform anomaly recognition on voucher images, thereby significantly improving recognition efficiency. Furthermore, this solution retrieves corresponding target knowledge from different knowledge bases for different key texts to perform anomaly detection, which can greatly improve the accuracy of key text detection, and thus improve the accuracy of anomaly recognition of voucher images. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram illustrating an implementation scenario of one embodiment disclosed in this specification;

[0020] Figure 2 A schematic diagram of an anomaly detection system is shown in one example of this specification;

[0021] Figure 3 A flowchart of a method for anomaly identification of a voucher image according to an embodiment of this specification is shown;

[0022] Figure 4 A schematic diagram of a voucher image is shown in one example of this specification;

[0023] Figure 5 A schematic diagram of an apparatus for anomaly identification of a credential image according to one embodiment of this specification is shown. Detailed Implementation

[0024] The solution provided in this specification will now be described with reference to the accompanying drawings.

[0025] As mentioned earlier, existing methods for detecting anomalies in voucher images suffer from poor generalization and accuracy. With the widespread adoption of multimodal large-scale models, some solutions propose using these models for voucher image anomaly detection. This involves inputting the voucher image into a multimodal large-scale model (e.g., Donut, LayoutLMv3, etc.) for classification, and then obtaining a result indicating whether the voucher image is normal or abnormal. However, this method has the following drawbacks: 1. It can only perform classification and cannot identify the abnormal fields and their causes; 2. Its generalization ability depends on the distribution of the training set; 3. It cannot flexibly introduce new rules / scene knowledge; 4. It is completely uncontrollable in the intermediate process, making debugging and optimization difficult.

[0026] To address this, this solution proposes combining Retrieval Augmented Generation (RAG) technology with a large-scale model to perform anomaly detection on voucher images. Specifically, the large-scale model first performs semantic intent recognition on the key texts included in the voucher image to determine the corresponding target knowledge bases; then, it retrieves the target knowledge from each target knowledge base to obtain the target knowledge; finally, the large-scale model uses the retrieved target knowledge to perform anomaly detection on the corresponding key texts, thereby determining the anomaly detection result of the voucher image.

[0027] Figure 1 This is a schematic diagram illustrating an implementation scenario of one of the embodiments disclosed in this specification. Figure 1 First, multiple text fragments are extracted from the voucher image. These include, for example, the field name and value of the invoice code field, the field name and value of the invoice number field, etc. These text fragments are then input into a large model, which selects key texts from them and performs semantic intent recognition on each key text. Based on the recognition results, the corresponding target knowledge bases are determined. For example, key text 1 could be "91110000717862580H" (Unified Social Credit Code), and its corresponding target knowledge base could be: Entity Archives; key text 2 could be "Purchase Office Tea Breaks" (Remarks), and its corresponding target knowledge base could be: Expert Rule Vector Library. Based on each key text, the target knowledge bases are retrieved using a retrieval method that matches each key text, obtaining the target knowledge matching each key text. Finally, each key text and its matching target knowledge are input into the large model, which performs anomaly detection on each key text based on the target knowledge. Based on the detection results, the anomaly recognition results for the voucher image are determined.

[0028] The method for anomaly detection of voucher images described in this solution can be executed by an anomaly detection system, wherein the anomaly detection system can, as follows: Figure 2 As shown, Figure 2 In this system, the system may include:

[0029] The OCR module is used to extract multiple text fragments and their bounding box coordinates from a voucher image.

[0030] The structured parsing module is used to select several key texts from multiple text fragments using a large model, and to determine the target knowledge base corresponding to each key text.

[0031] The RAG knowledge supply module is used to retrieve the target knowledge base corresponding to each key text and obtain the target knowledge that matches each key text.

[0032] A multimodal large model is used to perform anomaly detection on key texts based on knowledge of each target, and then determine the anomaly recognition result for the voucher image.

[0033] Figure 3 This diagram illustrates a method for anomaly identification of a voucher image according to one embodiment of this specification. It will be understood that this method can be executed by any apparatus, device, platform, or cluster of devices with computing and processing capabilities. Figure 3 As shown, the method includes the following steps:

[0034] Step S302: Extract multiple text fragments from the voucher image.

[0035] The aforementioned document images may include any of the following: invoices, receipts, shopping slips, and delivery notes, etc.

[0036] In one embodiment, Optical Character Recognition (OCR) technology can be used to perform text recognition on the voucher image, resulting in multiple text fragments. Furthermore, the bounding box coordinates (also known as position information) of each text fragment can be obtained, which can be represented as: [x_min, y_min, x_max, y_max]. Where (x_min, y_min) are the x and y coordinates of the top-left vertex of the bounding box, and (x_max, y_max) are the x and y coordinates of the bottom-right vertex.

[0037] by Figure 4 Taking the voucher image shown as an example, the multiple text fragments extracted from it and their bounding box coordinates can be as follows:

[0038] "Invoice Date": [36, 102, 96, 117];

[0039] "June 2, 2024": [105, 102, 226, 117];

[0040] "Seller": [46, 128, 80, 142];

[0041] Shanghai Weimei Catering Management Co., Ltd.: [114, 128, 298, 142];

[0042] Project Name: [36, 291, 84, 305];

[0043] "Food and Beverage Services": [394, 128, 445, 142];

[0044] "Total price including tax": [272, 291, 317, 305];

[0045] “850.00”: [374, 270, 417, 284].

[0046] The bounding box coordinates mentioned above can be corrected using spatial analysis.

[0047] In another embodiment, commonly used end-to-end deep learning models (such as CRNN, Mask R-CNN, EAST, etc.) can also be used to extract multiple text fragments from the voucher image.

[0048] Step S304: Input multiple text fragments into the large model, and let it select several key texts from the multiple text fragments. Perform semantic intent recognition on each key text, and determine the corresponding target knowledge base based on the recognition results.

[0049] Among these, the aforementioned large models can be, for example, BERT, GPT series, etc.

[0050] Additionally, a trained object detection model can be used to detect the category labels of at least some of the text segments mentioned above. This object detection model can be implemented as an R-CNN series, YOLO series, SSD series, Anchor-Free model, etc.

[0051] In one example, the category labels here could be: "Amount", "Invoice Code", "Invoice Number", etc.

[0052] In practice, the input to the aforementioned large model may also include the location information of each of the multiple text segments and the category labels of at least some of the text segments.

[0053] In one example, the prompt word "prompt1" can be input into a large model, instructing it to select key text and perform semantic intent recognition on the key text. Here, "prompt1" can include: the model's role (e.g., "You are an intelligent information routing assistant"), instruction descriptions (e.g., 1. Key text filtering: Select key texts that meet the "filtering rules" from multiple text fragments; 2. Intent recognition and routing: Determine the semantic intent of the selected key texts and match them to the corresponding "knowledge base"), input data (including multiple text fragments, location information, and category labels, etc.), candidate knowledge bases and their definitions (e.g., enterprise system database: a proprietary knowledge base storing internal enterprise rules, regulations, process documents, etc., used to standardize operations; online business database: a dynamic database connected to the official business system, providing enterprise registration and operation services). Real-time public information such as status; Entity archive: a relational database recording static archives and historical data of enterprises, institutions, or individuals throughout their entire lifecycle; Tax law knowledge vector library: a semantic retrieval library that transforms tax law provisions into high-dimensional vectors, supporting intelligent tax Q&A and clause association analysis; Regional regulations database: an authoritative legal provisions database containing normative documents such as regional regulations and departmental rules at all levels, filtering rules (e.g., prioritizing texts with the most specific information and clearest direction), intent recognition rules (e.g., strictly classifying based on the literal meaning and implicit needs of key texts; referring to the definition of a knowledge base, selecting the most matching and specific category), and output format.

[0054] Generally, when the semantic intent identified from the key text requires entity existence verification, the corresponding target knowledge base is an SQL database or a key-value database. Entities here can include, but are not limited to, companies, personal names, regional divisions, or tax rate values. Conversely, when the semantic intent identified from the key text requires verification of corporate expense reimbursement standards, the corresponding target knowledge base is a vector database.

[0055] For example, for the key text "91110000717862580H" (Unified Social Credit Code), the target knowledge base can be: Entity Archives; and for the key text "Purchase Office Tea Breaks" (Remarks), the target knowledge base can be: Expert Rule Vector Library.

[0056] In summary, this solution allows for the acquisition of different knowledge bases for different key texts, thus enabling refined detection of key texts and improving detection accuracy.

[0057] Step S306: For any key text T1 among the above key texts, retrieve the target knowledge base KB1 according to the retrieval method that matches the corresponding target knowledge base KB1, and obtain the target knowledge kb1 that matches the key text T1.

[0058] In one example, the above retrieval methods include structured retrieval and vectorized retrieval.

[0059] Specifically, when the target knowledge base KB1 is an SQL database or a key-value database (e.g., an entity archive), the corresponding retrieval method can be structured retrieval. When the target knowledge base KB1 is a vector database (e.g., an expert rule vector library), the corresponding retrieval method can be vectorized retrieval.

[0060] In practice, after identifying the target knowledge base for a specific key text, routing instructions containing the corresponding search methods can be generated. For example, for the key text "91110000717862580H", the following routing instruction can be generated: Query_Type: EXACT, Key: "91110000717862580H", KB: ... Here, "EXACT" represents structured search, also known as precise search. And for the key text "purchase office tea breaks", the following routing instruction can be generated: Query_Type: VECTOR, Content: "Office tea break reimbursement regulations, KB: ... Here, "VECTOR" represents vectorized search. Then, based on this routing instruction, a hierarchical hybrid search can be performed to obtain the target knowledge. The following explains this hierarchical hybrid search process:

[0061] When the target knowledge base KB1 is an SQL database, SQL code can be generated based on the key text T1 and the target knowledge base KB1. Executing the SQL code will yield the target knowledge kb1.

[0062] In one example, a large model can be used to generate SQL code based on the key text T1 and the target knowledge base KB1.

[0063] Taking the key text T1 as the company name and the target knowledge base KB1 as an online business database as an example, the generated SQL code can be as follows:

[0064] SELECT Business Status, Tax ID, Registered Location FROM Online Business Database WHERE Company Name = 'XX Technology Company'.

[0065] It should be understood that in this example, the "business status, tax number and place of registration" retrieved is the target knowledge kb1 mentioned above.

[0066] When the target knowledge base KB1 is a key-value database, the key value can be determined based on the key text T1. The target knowledge base KB1 is then queried based on this key value, and the resulting value is used as the target knowledge kb1.

[0067] Taking the key text T1 as "91110000717862580H" as an example, the corresponding Key="91110000717862580H" and the corresponding Value can include: registration information, registration agency information, credit rating and risk information, business information, shareholder and investment information, etc.

[0068] When the target knowledge base KB1 is a vector database, the target vector is determined based on the key text T1. Then, the similarity between the target vector and each vector in the target knowledge base KB1 can be calculated, and the knowledge represented by the vector with the highest similarity is taken as the target knowledge kb1.

[0069] Among them, representation models such as GPT series and BERT can be used to process key text T1, thereby obtaining the corresponding target vector.

[0070] In this scheme, the target knowledge kb1 retrieved for the key text T1 may include one or more of the following information types: entity description information, legal norms, industry standards, fuzzy expert experience, and historical anomaly cases, etc.

[0071] Similarly, target knowledge that matches each key text can be retrieved.

[0072] In practice, there may be conflicts in the retrieved target knowledge. In such cases, you can filter the target knowledge according to priority. For example, prioritize content from "laws and regulations" or "latest updates".

[0073] Additionally, the target knowledge retrieved for each key text in this solution can be organized into any of the following data formats: YAML, JSON, and XML.

[0074] Taking an organization in JSON format as an example, in one example, the target knowledge can be as follows:

[0075] <rag> [

[0077] {

[0078] "Target Knowledge Base": "Enterprise System Base"

[0079] Key fields: "Invoice date & Project name",

[0080] "Target Knowledge": "Employees' 'meal' expenses incurred on non-working days (Saturdays, Sundays, and public holidays) must be linked to the [Overtime Application Form] in the OA system; otherwise, they will be considered personal expenses and will not be reimbursed."

[0081] },

[0082] {

[0083] "Target Knowledge Base": "Networked Business Database"

[0084] Key field: "Seller Name",

[0085] "Target Knowledge": {

[0086] Entity: Shanghai Weimei Catering Management Co., Ltd.

[0087] Status: "Cancelled (January 15, 2024)" / / Key point: Cancelled before invoicing.

[0088] Risk Level: High

[0089] }

[0090] } ]

[0092] < / rag>

[0093] The aforementioned "key fields" are the fields to which the key text belongs. The first target knowledge is retrieved based on the two key fields "invoice date" and "item name", while the second target knowledge is retrieved based on the key field "seller name".

[0094] In another example, the target knowledge can also be as follows:

[0095] {

[0096] "scenario": "Geo_Consistency_Check", / / Geographic consistency check

[0097] "ocr_observation": { / / Optical character recognition detection}

[0098] "invoice_code_prefix": "031001", / / First six digits of the invoice code

[0099] "seller_address": "Zhongguancun Street, Haidian District, Beijing..." / / Address for invoice stamp

[0100] },

[0101] "rag_knowledge_supply": { / / RAG knowledge supply

[0102] "code_definition": "Shanghai (0310)", / / Truth value retrieved from RAG

[0103] "logic_rule": "The first four digits of the invoice code must match the geographical region of the invoice issuer".

[0104] "source": "Administrative_Code_DB" / / Region code database

[0105] }

[0106] }

[0107] {

[0108] "scenario": "Legal_Compliance_Check", / / Compliance check

[0109] "ocr_observation": {

[0110] "item_name": "Baijiu / Holiday Gift Box", / / Product Name

[0111] "tax_usage": "Input tax credit", / / Tax purpose

[0112] "buyer_type": "General taxpayer" / / Buyer type

[0113] },

[0114] "rag_knowledge_supply": {

[0115] "law_reference": "Article 10 of the Provisional Regulations on Value-Added Tax", / / Legal basis

[0116] "law_content": "Input tax on goods purchased for collective welfare or personal consumption (such as entertainment or gifts) shall not be deducted from output tax."

[0117] "source": "Tax_Law_Vector_DB" / / Tax Law Vector Database

[0118] }

[0119] }

[0120] {

[0121] "scenario": "Entity_Risk_Web_Search", / / Entity risk web search

[0122] "ocr_observation": {

[0123] "seller_name": "Hangzhou XX Network Technology Co., Ltd." / / Seller Name

[0124] "invoice_date": "2024-11-20", / / The invoice date is very recent.

[0125] "amount": "100,000.00" / / Amount

[0126] },

[0127] "rag_knowledge_supply": {

[0128] "official_status": "Revoked", / / Official status

[0129] "revoke_date": "2023-05-15", / / Cancellation date is before the invoice was issued.

[0130] "risk_flag": "High", / / Risk flag

[0131] "source": "Web_Search_Supplement" / / Highlight: The source is web-based autocomplete (web search for supplementary information).

[0132] }

[0133] }

[0134] The above explains the scenarios where target knowledge can be retrieved based on key text. In practice, there may be cases where target knowledge cannot be retrieved. For example, if the key text T1 is a company name, and that company is newly established, then target knowledge kb1 will not be retrieved. Furthermore, if the similarity between the target vector of key text T1 and each vector in the target knowledge base KB1 is less than a predetermined threshold, target knowledge kb1 will also not be retrieved.

[0135] If the target knowledge base KB1 cannot be retrieved, a search engine (e.g., the official website of the tax bureau, Qichacha, the national legal database, etc.) can be invoked based on the key text T1 to obtain the target webpage content. A large model is then used to extract the target information from the target webpage content. This target information is then stored in the target knowledge base KB1.

[0136] In a more specific embodiment, search terms can first be generated for the key text T1, whereby the search terms need to reflect the retrieval requirements determined based on the semantic intent recognition results. Then, a search engine is invoked based on these search terms. For example, if the key text T1 is "XX Technology Co., Ltd.", the generated search term could be: "XX Technology Co., Ltd. Business Registration Information". And if the key text T1 is "Drone Repair Fee", the generated search term could be: "2025 Drone Repair Fee VAT Rate".

[0137] Step S308: Input each key text and its corresponding target knowledge into the large model, and let it perform anomaly detection on each key text based on the target knowledge. Based on the detection results, determine the anomaly recognition result for the voucher image.

[0138] Among them, the aforementioned large models can be multimodal large models (large language models with joint understanding and reasoning capabilities of images and text), such as GPT-4V / GPT-4o (OpenAI), Gemini (Google), or Qwen-VL / Qwen2-VL (Alibaba), etc.

[0139] Additionally, the voucher image can also be input into the large model so that anomaly detection can be performed on the voucher image based on non-textual information, such as Photoshop traces or text style (size, color, or format).

[0140] It should be noted that since the text fragments extracted by OCR technology may be inaccurate, for example, there may be missing words or misidentification, it is necessary to cross-verify the text fragments and the voucher image. Thus, this solution can also achieve cross-modal verification.

[0141] In addition, the type of voucher image can also be entered into the large model. Taking an invoice as an example, the type of voucher image can include, for example, "Value-Added Tax Special Invoice", "Value-Added Tax General Invoice", "Motor Vehicle Sales Uniform Invoice", "Used Car Sales Uniform Invoice", etc.

[0142] In one example, the prompt word "prompt2" can be input into a large model, which will then output the anomaly detection results for the voucher image. The content of the prompt word "prompt2" can be as follows:

[0143] You are an "AI Auditor for Document Image Compliance Audits". Your task is to receive document images, multiple key texts, and their matching target knowledge, and conduct an in-depth review of the authenticity, compliance, and logical consistency of the document image.

[0144] <image_input>[Loading voucher image_Invoice_001.jpg]< / image_input>

[0145] <ocr>

[0146] {"Invoice Type": "General VAT Invoice", "Key Text Description": { "Invoice Date": { "text": "June 2, 2024", "position": [450, 120, 550, 140]}, / / Key Point: This is a Sunday, "Seller Name": { "text": "Shanghai Weimei Catering Management Co., Ltd.", "position": [100,200, 300, 220]}, "Project Name": { "text": "Catering Services", "position": [100, 350,200, 370]}, "Total Amount": { "text": "¥850.00", "position": [600, 500, 700, 520]}}}

[0147] < / ocr>

[0148] <rag> [

[0150] {

[0151] "Target Knowledge Base": "Enterprise System Base"

[0152] Key fields: "Invoice date & Project name",

[0153] "Target Knowledge": "Employees' 'meal' expenses incurred on non-working days (Saturdays, Sundays, and public holidays) must be linked to the [Overtime Application Form] in the OA system; otherwise, they will be considered personal expenses and will not be reimbursed."

[0154] },

[0155] {

[0156] "Target Knowledge Base": "Networked Business Database"

[0157] Key field: "Seller Name",

[0158] "Target Knowledge": {

[0159] Entity: Shanghai Weimei Catering Management Co., Ltd.

[0160] Status: "Cancelled (January 15, 2024)" / / Key point: Cancelled before invoicing.

[0161] Risk Level: High

[0162] }

[0163] } ]

[0165] < / rag>

[0166] In this example, the prompt word prompt2 includes: voucher image, invoice type, key text description (including key fields, text content, and position information) and target knowledge related to the key text.

[0167] In practice, the above instruction description can also be replaced with: "Your task is to perform semantic understanding of key text based on target knowledge, and to perform cross-field logical judgments (i.e., judging the reasoning relationships between multiple fields), rule matching, and common sense judgments, etc." Furthermore, the above anomaly identification results include not only the conclusion of whether an anomaly exists, but also the anomaly field, the violation logic, the anomaly location, and explanations.

[0168] In summary, the anomaly identification method for voucher images provided in this specification can combine the text content of the voucher image to identify anomalies, and it is independent of layout and templates, supporting different types of voucher formats. Furthermore, since the knowledge in each target knowledge base supports hot updates, it can flexibly adapt to different scenarios. Finally, this solution can output field-level anomaly prompts, rule violation explanations, and visual annotations of abnormal areas, and has strong traceability.

[0169] Example effect:

[0170] A reimbursement invoice shows a tax rate of 17%, but the product category is "books". The big model points out that "books should be subject to tax exemption / 3% tax rate, which is a logical inconsistency", and locates the "tax rate field" and gives the violation rule number.

[0171] The innovations of this solution are summarized below: 1. A large-model-driven logic-based anomaly reasoning and judgment mechanism replaces traditional rule-based judgment. 2. A RAG-assisted dynamic rule / specification supply mechanism enables cross-field document adaptation. 3. A cross-modal field-image space joint structure alignment method supports field location and anomaly interpretability. 4. Anomaly cause explanation capabilities provide multi-dimensional outputs such as fields, rules, and locations, improving usability and trustworthiness. 5. A highly compatible structured organizational process that can be reused in other document analysis scenarios (such as contract review and delivery note verification).

[0172] Corresponding to the above-described method for anomaly recognition of voucher images, one embodiment of this specification also provides an apparatus for anomaly recognition of voucher images, such as... Figure 5 As shown, the device may include:

[0173] Extraction unit 502 is used to extract multiple text fragments from the voucher image.

[0174] Input unit 504 is used to input multiple text fragments into the first large model, so that it can select several key texts from the multiple text fragments, perform semantic intent recognition on each key text, and determine the corresponding target knowledge base based on the recognition results.

[0175] The retrieval unit 506 is used to retrieve the first target knowledge base for any first key text among several key texts, according to a first retrieval method that matches the corresponding first target knowledge base, and obtain the first target knowledge that matches the first key text.

[0176] The input unit 504 is also used to input each key text and the corresponding target knowledge into the second model, so that it can perform anomaly detection on each key text based on the target knowledge, and determine the anomaly recognition result for the voucher image based on the detection result.

[0177] In one embodiment, the input to the first major model described above also includes one or more of the following:

[0178] Location information of multiple text fragments;

[0179] The category labels of at least some of the multiple text segments; the category labels of a single text segment are obtained by processing the text segment using an object detection model.

[0180] In one embodiment, the extraction unit 502 is specifically used for:

[0181] Optical Character Recognition (OCR) technology is used to perform text recognition on voucher images, resulting in multiple text fragments.

[0182] In one embodiment, the first target knowledge base is an SQL database / KV database, and the first retrieval method is structured retrieval;

[0183] Retrieval unit 506 is specifically used for:

[0184] When the first target knowledge base is an SQL database, SQL code is generated based on the first key text and the first target knowledge base; the SQL code is executed to obtain the first target knowledge.

[0185] When the target knowledge base is a key-value database, the key value is determined based on the first key text; the first target knowledge base is queried based on the key value, and the obtained value is used as the first target knowledge.

[0186] In another embodiment, the first target knowledge base is a vector database, and the first retrieval method is vectorized retrieval;

[0187] Retrieval unit 506 is specifically used for:

[0188] Determine the target vector based on the first key text;

[0189] The first target knowledge is determined by calculating the similarity between the target vector and each vector in the first target knowledge base.

[0190] In one embodiment, the second large model is a multimodal large model, and the input of the second large model also includes the voucher image.

[0191] In one embodiment, the device further includes:

[0192] Calling unit 508 is used to call the search engine based on the first key text to obtain the target webpage content when the first target knowledge is not retrieved.

[0193] Extraction unit 510 is used to extract target information from the content of the target webpage using a large model;

[0194] Storage unit 512 is used to store target information into the first target knowledge base.

[0195] In one embodiment, the first target knowledge includes one or more of the following: entity description information, legal norms, industry standards, fuzzy expert experience, and historical anomaly cases.

[0196] In one embodiment, the aforementioned credential image includes any of the following: invoice, receipt, shopping slip, and delivery note, etc.

[0197] In one embodiment, the aforementioned first target knowledge is organized in any of the following data formats: YAML, JSON, and XML, etc.

[0198] The functions of each functional module of the apparatus in the above embodiments of this specification can be implemented through the steps of the above method embodiments. Therefore, the specific working process of the apparatus provided in one embodiment of this specification will not be repeated here.

[0199] This specification provides an embodiment of an apparatus for anomaly recognition of voucher images, which can perform anomaly recognition of voucher images more efficiently and accurately.

[0200] According to another embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed in a computer, causes the computer to perform a combination Figure 3 The method described.

[0201] According to another embodiment, a computing device is also provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements a combination... Figure 3 The method described.

[0202] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the medium or device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0203] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0204] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this specification. It should be understood that the above description is only a specific embodiment of this specification and is not intended to limit the scope of protection of this specification. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of this specification should be included within the scope of protection of this specification.

Claims

1. A method for anomaly detection of voucher images, comprising: Extract multiple text fragments from a voucher image; The multiple text fragments are input into the first large model, which selects several key texts from the multiple text fragments, performs semantic intent recognition on each key text, and determines the corresponding target knowledge base based on the recognition results. For any first key text among the plurality of key texts, the first target knowledge base is retrieved according to a first retrieval method that matches the corresponding first target knowledge base, and the first target knowledge matching the first key text is obtained. Each key text and its corresponding target knowledge are input into the second major model, which then performs anomaly detection on each key text based on the target knowledge. Based on the detection results, the anomaly recognition result for the voucher image is determined.

2. The method according to claim 1, wherein, The input to the first large model also includes one or more of the following: The location information of the multiple text fragments; The category labels of at least some of the multiple text segments; the category labels of a single text segment are obtained by processing the text segment using an object detection model.

3. The method according to claim 1, wherein, The extraction of multiple text fragments from the voucher image includes: The document image is subjected to text recognition using optical character recognition (OCR) technology to obtain the multiple text fragments.

4. The method according to claim 1, wherein, The first target knowledge base is an SQL database / KV database, and the first retrieval method is structured retrieval; The step of retrieving the first target knowledge base according to a first retrieval method that matches the corresponding first target knowledge base includes: When the first target knowledge base is an SQL database, SQL code is generated based on the first key text and the first target knowledge base; the SQL code is executed to obtain the first target knowledge. When the target knowledge base is a KV database, a Key value is determined based on the first key text; the first target knowledge base is queried based on the Key value, and the obtained Value is used as the first target knowledge.

5. The method according to claim 1, wherein, The first target knowledge base is a vector database; the first retrieval method is vectorized retrieval; The step of retrieving the first target knowledge base according to a first retrieval method that matches the corresponding first target knowledge base includes: Based on the first key text, determine the target vector; The first target knowledge is determined by calculating the similarity between the target vector and each vector in the first target knowledge base.

6. The method according to claim 1, wherein, The second large model is a multimodal large model, and the input of the second large model also includes the voucher image.

7. The method according to claim 1, further comprising: If the first target knowledge is not found, the search engine is invoked based on the first key text to obtain the target webpage content; Target information is extracted from the content of the target webpage using a large model; The target information is stored in the first target knowledge base.

8. The method according to claim 1, wherein, The first target knowledge includes one or more of the following: entity description information, legal norms, industry standards, fuzzy expert experience, and historical anomaly cases.

9. The method according to claim 1, wherein, The document image includes any of the following: invoice, receipt, shopping slip, and delivery note.

10. The method according to claim 1, wherein, The first target knowledge is organized in any of the following data formats: YAML, JSON, and XML.

11. An apparatus for anomaly detection of voucher images, comprising: An extraction unit is used to extract multiple text fragments from a voucher image; The input unit is used to input the multiple text fragments into the first large model, so that it selects several key texts from the multiple text fragments, performs semantic intent recognition on each key text, and determines the corresponding target knowledge base based on the recognition results. The retrieval unit is used to retrieve the first target knowledge base for any first key text among the plurality of key texts, according to a first retrieval method that matches the corresponding first target knowledge base, and to obtain the first target knowledge that matches the first key text. The input unit is also used to input each key text and the corresponding target knowledge into the second model, so that it can perform anomaly detection on each key text based on the target knowledge, and determine the anomaly recognition result for the voucher image based on the detection result.

12. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed in the computer, it causes the computer to perform the method according to any one of claims 1-10.

13. A computing device comprising a memory and a processor, wherein, The memory stores executable code, and when the processor executes the executable code, it implements the method of any one of claims 1-10.

Images