Document field extraction method and device, computer device and readable storage medium

Abstract

The application relates to a document field extraction method and device, computer equipment and a readable storage medium. A plurality of source candidate extraction results corresponding to a target field in a to-be-recognized document image are obtained. If at least two conflicting extraction results exist in the plurality of source candidate extraction results, consistency verification is performed on each conflicting extraction result based on the to-be-recognized document image to obtain a target verification result corresponding to each conflicting extraction result. Multi-source credibility analysis is performed based on each conflicting extraction result and the target verification result corresponding to each conflicting extraction result to obtain a multi-source credibility score corresponding to each conflicting extraction result. A target multi-source credibility score meeting a preset credibility decision condition is determined from the plurality of multi-source credibility scores. A target conflicting extraction result corresponding to the target multi-source credibility score is determined as a target extraction result corresponding to the target field. The accuracy and credibility of document field extraction are improved.

Description

Technical Field

[0001] This application relates to the field of document field extraction technology, and in particular to a document field extraction method, apparatus, computer device, and readable storage medium. Background Technology

[0002] In the information extraction process of document images (such as invoices, ID cards, contracts, etc.), single text extraction models are easily affected by factors such as image quality, layout differences, font variations, and noise interference, leading to unstable recognition results and insufficient accuracy for the same target field. To improve system robustness, traditional techniques typically employ multi-source strategies (such as multiple heterogeneous OCR (Optical Character Recognition) engines or information extraction models) to generate multiple candidate extraction results in parallel. However, different sources often output conflicting extraction results, and existing methods generally rely on confidence fusion strategies such as linear weighting or simple summation for result decision-making. Such strategies lack the ability to discriminate the quality of sources, easily allowing a large number of low-accuracy sources to generate a "false high credibility" phenomenon through score superposition or numerical advantage, thus significantly reducing the accuracy and credibility of the final field extraction results.

[0003] There is currently no effective solution to the problems of low accuracy and poor reliability in document field extraction in traditional technologies. Summary of the Invention

[0004] Therefore, it is necessary to provide a document field extraction method, apparatus, computer device, and readable storage medium to address the aforementioned technical problems.

[0005] Firstly, this application provides a method for extracting document fields, the method comprising:

[0006] Obtain the multi-source candidate extraction results corresponding to the target fields in the image of the document to be identified;

[0007] If at least two conflicting extraction results are detected among the multi-source candidate extraction results, then based on the document image to be identified, a consistency check is performed on each of the conflicting extraction results to obtain the target check result corresponding to each of the conflicting extraction results.

[0008] Based on the conflict extraction results and their corresponding target verification results, multi-source credibility analysis is performed to obtain the multi-source credibility score corresponding to each conflict extraction result.

[0009] From the multiple multi-source credibility scores, a target multi-source credibility score that meets the preset credibility decision conditions is determined. Based on the target multi-source credibility score and the target conflict extraction result corresponding to the target multi-source credibility score, the target extraction result corresponding to the target field is determined.

[0010] In one embodiment, the step of performing consistency verification on each of the conflict extraction results based on the document image to be identified, to obtain the target verification result corresponding to each of the conflict extraction results, includes:

[0011] The document image to be identified and each of the conflict extraction results are input into multiple heterogeneous verification models in parallel. Visual consistency verification is performed on each of the conflict extraction results to obtain the first verification result corresponding to each of the conflict extraction results.

[0012] Based on the preset rigid verification rules, the rule consistency verification is performed on each of the conflict extraction results to obtain the second verification result corresponding to each of the conflict extraction results.

[0013] Based on each of the first verification results and each of the second verification results, the target verification result corresponding to each of the conflict extraction results is determined.

[0014] In one embodiment, the step of inputting the document image to be identified and each of the conflict extraction results into multiple heterogeneous verification models in parallel, performing visual consistency verification on each of the conflict extraction results, and obtaining a first verification result corresponding to each of the conflict extraction results includes:

[0015] The document image to be identified and each of the conflict extraction results are input into multiple heterogeneous verification models in parallel. The multiple heterogeneous verification models are used to perform visual consistency verification on each of the conflict extraction results to obtain multiple visual consistency results corresponding to each of the conflict extraction results.

[0016] For each of the conflict extraction results, based on the credibility weight coefficients corresponding to the heterogeneous verification models, a weighted average is performed on the multiple visual consistency results corresponding to the conflict extraction results to obtain the first verification result corresponding to the conflict extraction results.

[0017] In one embodiment, the step of performing rule consistency verification on each of the conflict extraction results based on preset rigid verification rules to obtain a second verification result corresponding to each of the conflict extraction results includes:

[0018] Identify the document feature information of the document image to be identified;

[0019] Based on the document feature information, at least one target rigid verification rule that matches the document feature information is determined from the preset rigid verification rules;

[0020] If the conflict extraction result does not conform to any of the target rigid verification rules, then the second verification result corresponding to the conflict extraction result is set to a preset value.

[0021] In one embodiment, the conflict extraction result includes a conflict candidate field value and at least one initial confidence level corresponding to the conflict candidate field value; the step of performing multi-source credibility analysis based on each of the conflict extraction results and their corresponding target verification results to obtain a multi-source credibility score corresponding to each of the conflict extraction results includes:

[0022] For each initial confidence level corresponding to the conflict candidate field value, the single-source credibility probability of the conflict candidate field value on the associated text extraction model associated with the initial confidence level is calculated based on the initial confidence level and the target verification result.

[0023] Based on the credibility probabilities of each single source, a joint probability analysis is performed to obtain the multi-source credibility score corresponding to the conflict extraction result.

[0024] In one embodiment, determining the target multi-source credibility score that satisfies the preset credibility decision conditions from a plurality of multi-source credibility scores, and determining the target extraction result corresponding to the target field based on the target multi-source credibility score and the target conflict extraction result corresponding to the target multi-source credibility score, includes:

[0025] The multi-source credibility score with the highest score among the multiple multi-source credibility scores is determined as the candidate multi-source credibility score;

[0026] If the candidate multi-source credibility score is greater than or equal to a preset credibility threshold, and the conflict extraction result corresponding to the candidate multi-source credibility score satisfies the preset multi-source consensus condition, then the candidate multi-source credibility score is determined as the target multi-source credibility score.

[0027] The target conflict candidate field value in the target conflict extraction result corresponding to the target multi-source credibility score is determined as the target field value corresponding to the target field; the target multi-source credibility score is determined as the target confidence level corresponding to the target field value.

[0028] Secondly, this application provides a document field extraction apparatus, the apparatus comprising:

[0029] The acquisition module is used to acquire the multi-source candidate extraction results corresponding to the target fields in the document image to be recognized;

[0030] The verification module is used to perform consistency verification on each of the conflicting extraction results based on the document image to be identified if at least two conflicting extraction results are detected among the multi-source candidate extraction results, so as to obtain the target verification result corresponding to each of the conflicting extraction results.

[0031] The multi-source credibility analysis module is used to perform multi-source credibility analysis based on each of the conflict extraction results and their corresponding target verification results, and to obtain the multi-source credibility score corresponding to each of the conflict extraction results.

[0032] The decision module is used to determine the target multi-source credibility score that meets the preset credibility decision conditions from multiple multi-source credibility scores, and to determine the target extraction result corresponding to the target field based on the target multi-source credibility score and the target conflict extraction result corresponding to the target multi-source credibility score.

[0033] Thirdly, this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0034] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0035] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0036] The aforementioned document field extraction method, apparatus, computer equipment, and readable storage medium acquire multi-source candidate extraction results of target fields in a document image to be identified. When at least two conflicting extraction results are detected among the multi-source candidate extraction results, a consistency check is performed on each conflicting extraction result based on the document image to be identified, generating target check results corresponding to each conflicting extraction result, providing an analytical basis for subsequent multi-source credibility analysis. Furthermore, multi-source credibility analysis is performed based on the conflicting extraction results and their corresponding target check results to obtain the corresponding multi-source credibility scores. This effectively suppresses the "false high credibility" phenomenon caused by the superposition of original confidence scores or the output of a single high-scoring model due to low-quality extraction results, thereby avoiding blind trust in low-quality sources. Finally, based on preset credibility decision conditions, further decision analysis is performed to select the target conflicting extraction results corresponding to the target credibility scores that meet the preset credibility decision conditions, and the target extraction results corresponding to the target fields are determined accordingly, improving the accuracy and credibility of document field extraction. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is an application environment diagram of a document field extraction method in one embodiment;

[0039] Figure 2 This is a flowchart illustrating a document field extraction method in one embodiment;

[0040] Figure 3 This is a flowchart illustrating the steps for determining the target verification result in one embodiment;

[0041] Figure 4 This is a flowchart illustrating the first verification result determination step in one embodiment;

[0042] Figure 5 This is a flowchart illustrating the second verification result determination step in one embodiment;

[0043] Figure 6 This is a flowchart illustrating the steps for determining the credibility score of multiple information sources in one embodiment;

[0044] Figure 7 This is a flowchart illustrating the steps for determining the target extraction result in one embodiment;

[0045] Figure 8 This is a structural block diagram of a document field extraction device in one embodiment;

[0046] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] In scenarios such as financial KYC (Know Your Customer), expense reimbursement, and document management, it is necessary to extract key fields (such as name, date, and amount) from images of ID cards, invoices, and contracts. With the development of Large Language Models (LLM), intelligent extraction methods based on LLM are gradually replacing traditional template matching methods. Currently, the mainstream technical solutions mainly include three categories: First, rule-based methods (such as regular expressions or fixed templates), which have high accuracy but poor generalization ability and are difficult to handle distorted, blurry, or formatted documents; second, end-to-end visual large language model (Visual LLM) methods, which have strong generalization ability but are prone to "illusions," i.e., fabricating non-existent field values; and third, linear combination strategies that simply weight and vote on the results of multiple extractors (such as summing or averaging scores). However, these related solutions have significant drawbacks: First, when there are many low-quality extractors, linear weighting is prone to the problem of "pseudo-high scores (i.e., falsely high credibility)": for example, the sum of the scores of three models with an accuracy of only 40% may easily exceed the passing threshold (such as 90 points), causing erroneous fields to be misjudged as valid, seriously damaging data accuracy; second, there is a lack of an effective consensus mechanism. When a single high-weighted LLM outputs an incorrect result due to illusion, its high score can directly trigger the passing judgment, forming "isolated evidence misjudgment," lacking the constraint of multi-source cross-validation; finally, when faced with result conflicts between different extractors (such as regular expression extraction being "A" and LLM outputting "B"), traditional methods can only rely on preset weights to rigidly choose one of the two, unable to combine the visual context of the image for dynamic and intelligent arbitration. The above problems constitute the core bottleneck of current intelligent document information extraction, and are also the key issues that this application aims to solve.

[0049] The document field extraction method provided in this application embodiment can be applied to, for example, Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, and tablets. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0050] In one embodiment, such as Figure 2 As shown, Figure 2 This is a flowchart illustrating a document field extraction method in one embodiment. This embodiment uses the method applied to a terminal as an example; it is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0051] Step S201: Obtain the multi-source candidate extraction results corresponding to the target field in the document image to be identified.

[0052] The document image to be identified refers to a document image containing structured or semi-structured text information. The document image to be identified may include, but is not limited to, identity documents, financial and business invoices, and legal and administrative documents; identity documents may include, but are not limited to, resident ID cards, passports, driver's licenses, residence permits, and visa pages; financial and business invoices may include, but are not limited to, value-added tax invoices, bank transfer receipts, shopping receipts, and logistics waybills; legal and administrative documents may include, but are not limited to, commercial contracts, business licenses, property ownership certificates, insurance policies, medical records, and university application forms.

[0053] In one exemplary embodiment, the document image to be identified can be acquired by an optical acquisition device; the optical acquisition device may include, but is not limited to, a smartphone camera, a high-speed document scanner, a flatbed scanner, etc.; the document image to be identified may also be generated from electronic files, such as by taking a screenshot from a PDF (Portable Document Format) reader, exporting directly from an electronic invoice system, or generating a document image to be identified from a webpage screenshot.

[0054] The document image to be identified includes document feature information; document feature information may include, but is not limited to, textual semantic information and visual layout information; textual semantic information refers to the character content in the document that carries specific business meaning; textual semantic information may be presented in the form of key-value pairs (such as "Name: Zhang San") or tables. Visual layout information includes document version type, etc.

[0055] The target field refers to the key information items in the document image to be identified; for example, the target field may be, but is not limited to, name, ID number, reimbursement amount, date, etc.

[0056] Here, the multi-source candidate extraction results corresponding to the target field refer to the set of candidate extraction results generated by multiple heterogeneous text extraction models independently extracting the target field from the same document image to be recognized. These multiple heterogeneous text extraction models may include, but are not limited to, regular rule extractors, OCR engines, Visual Large Language Models (Visual LLM), and specialized field recognition models.

[0057] The multi-source candidate extraction results include multiple candidate extraction results; each candidate extraction result includes a candidate field value and at least one initial confidence level corresponding to the candidate field value.

[0058] It is understandable that the same candidate field value may be output by multiple heterogeneous text extraction models. For example, if three different heterogeneous text extraction models (such as regular rule extractors, large language model text extractors, and visual large language models) independently extract the same candidate field value "A" for a certain target field, then the candidate field value "A" corresponds to three initial confidence scores, which come from the evaluation of each model itself.

[0059] In an exemplary embodiment, obtaining the multi-source candidate extraction results corresponding to the target field in the document image to be identified includes the following steps:

[0060] Step 1: Obtain the image of the document to be identified and multiple heterogeneous text extraction models.

[0061] Step 2: Based on multiple heterogeneous text extraction models, perform multi-source parallel field extraction on the document image to be identified, and obtain the multi-source candidate extraction results corresponding to each target field in the document image to be identified.

[0062] In an exemplary embodiment, step 2, based on multiple heterogeneous text extraction models, performs multi-source parallel field extraction on the document image to be identified, obtaining multi-source candidate extraction results corresponding to each target field in the document image to be identified. This can be achieved by: performing multi-source parallel field extraction on the document image to be identified based on multiple heterogeneous text extraction models, obtaining multiple corresponding extraction result output lists, and then performing bucketing processing on these multiple extraction result output lists to obtain multi-source candidate extraction results corresponding to each target field in the document image to be identified. The extraction result output lists include candidate extraction results extracted by the corresponding heterogeneous text extraction models for all target fields in the document image to be identified. Bucketing processing refers to aggregating multiple extraction result output lists according to the field name of the target field: traversing the outputs of all heterogeneous models, all candidate field values ​​(and their sources and initial confidence levels) under the same field name are grouped into the same logical group (i.e., a "bucket"), thereby forming structured data centered on the field. For example, all outputs related to the "name" field are grouped into the "name bucket," and all outputs related to "amount" are grouped into the "amount bucket."

[0063] For example, suppose the output list of all heterogeneous text extraction model providers is as follows: Provider A reports: {Name: "Zhang San", Age: 20}; Provider B reports: {Name: "Zhang Sanfeng", Age: 20}; Based on the output list of all heterogeneous text extraction model providers, the data is binned using the field name as the key and the list of candidate values ​​and their source information under that field name as the value. The multi-source candidate extraction results corresponding to each target field in the document image to be identified are as follows: Name bin: ["Zhang San" (from A), "Zhang Sanfeng" (from B)]; Age bin: [20 (from A), 20 (from B)].

[0064] It should be noted that the inputs of different heterogeneous text extraction models vary. A heterogeneous text extraction model can be the original image of the document to be recognized, or it can be a list of unstructured text (such as a sequence of text blocks arranged by position or line order) corresponding to the image of the document to be recognized, based on an OCR engine. The specific form depends on the architecture and design requirements of each heterogeneous text extraction model. For example, regular rule extractors rely on structured or semi-structured OCR text output, while Visual Large Language Models (Visual LLM) can directly use the original image as input for end-to-end inference. Since different models have different requirements for input modalities and formats, this application does not specifically limit the input format of each heterogeneous text extraction model.

[0065] For example, taking a regular expression rule extractor as an example, the OCR engine identifies the corresponding unstructured text list from the document image to be recognized; wherein, the unstructured text list includes multiple text blocks; each text block includes a text fragment and the position information of the text fragment in the image coordinate system. For example, the unstructured text list is: "Block 1: text="Name", box=[10,10, 50,30];Block 2: text="Zhang San", box=[60,10, 100,30];Block 3: text="Birth", box=[10,50, 50,70];Block 4: text="January 1, 1990", box=[60,50,200,70];Block 5: text="Address", box=[10,90, 50,110];Block 6: text="District B, City A...",box=[60,90, 300,110]。 Secondly, the list of unstructured text is distributed to the regular expression rule extractor. For example, the input is the above unstructured list, and the regular expression (?<=Name)[\u4e00-\u9fa5]+ is run. The output is "Zhang San", with a confidence score of 100 (successful match). The corresponding candidate extraction result is obtained as: {field: "name", value: "Zhang San", score: 100}.

[0066] Step S202: If at least two conflicting extraction results are detected among the multi-source candidate extraction results, then based on the document image to be identified, a consistency check is performed on each conflicting extraction result to obtain the target check result corresponding to each conflicting extraction result.

[0067] Conflict extraction results refer to candidate extraction results where candidate field values ​​conflict with each other (i.e., are inconsistent). Conflict extraction results include conflicting candidate field values ​​and at least one initial confidence level corresponding to each conflicting candidate field value.

[0068] Among them, consistency verification is used to compare the conflict extraction results with the document image to be identified in order to evaluate the degree of consistency between each conflict extraction result and the document image to be identified at the visual and semantic levels, thereby determining the target verification result of each conflict extraction result.

[0069] The target verification result is determined jointly by the first and second verification results. The first verification result characterizes the degree of visual consistency between the conflict extraction result and the document image to be identified. For example, multiple heterogeneous verification models are used to evaluate whether the conflict candidate field values ​​in the conflict extraction result match the actual pixel content. The second verification result characterizes whether the conflict extraction result conforms to preset rigid verification rules. These preset rigid verification rules refer to inviolable verification conditions set based on business logic, format specifications, or mathematical constraints; for example, the date format must be logically valid (e.g., "February 30th" does not exist); the invoice code / number must match the coding structure defined by the tax system, etc. When a conflict extraction result violates any preset rigid verification rule, even if the corresponding first verification result is high, it should be directly rejected. By combining the first and second verification results to obtain the target verification result, a more reliable conflict resolution and final field determination can be achieved.

[0070] In an exemplary embodiment, suppose we need to identify a slightly worn invoice whose actual "Total Amount" column shows "¥1,008.00". The number "8" appears as a "0" due to blurring, leading to discrepancies between different heterogeneous text extraction models. A comparison of the output results of the heterogeneous text extraction models is shown in Table 1.

[0071] Table 1

[0072]

[0073] It should be noted that the reason why the initial confidence level of the candidate field value "1,000.00" extracted by the regular expression extractor is 100 is that the original text recognized by the OCR engine is "Total: 1,000.00 (of which the actual amount is " The number "8" in "1,008.00" was misidentified as "0". Since the input to the regular expression extractor comes directly from the OCR output, its processing logic relies solely on predefined regular expressions for pattern matching within the OCR text. In this case, the string "1,000.00" perfectly matches the rule definition, and the regular expression extractor successfully and unambiguously completes the extraction, thus determining its result to be highly reliable and assigning an initial confidence level of 100. However, this high confidence level only reflects the determinism of the rule matching and does not perceive underlying OCR errors or true semantics. Essentially, it makes a correct but invalid inference based on incorrect input, which is precisely the limitation of single-rule methods in complex real-world scenarios.

[0074] Although the input to the large language model text extractor also comes from the OCR output, it is not limited to literal matching. Instead, it recognizes the detailed line information "unit price 504.00 × quantity 2" above the invoice through contextual semantic understanding. Based on this, the large language model text extractor performs a simple arithmetic logic check (504 × 2 = 1008), thereby inferring that the correct total amount should be "1,008.00", and correcting the explicit recognition error of the OCR accordingly. However, since this result is inconsistent with "1,000.00" in the original OCR text, the large language model text extractor maintains a certain degree of caution regarding its reasoning result in the lack of direct visual evidence, and therefore assigns an initial confidence score of 90, slightly below the maximum. This confidence score reflects both the advantages of the large language model in semantic reasoning and logical consistency, and its self-calibration mechanism against input noise and potential hallucination risks.

[0075] Furthermore, based on Table 1, by clustering all candidate field values ​​corresponding to "Total Amount," i.e., grouping identical candidate values ​​from multiple heterogeneous extraction model outputs into the same group, two sets of conflicting candidate extraction results can be detected in the multi-source candidate extraction results: The first set of conflicting candidate extraction results: the conflicting candidate field value is "1,000.00," originating from the regular rule extractor, with an initial confidence level of 100; the second set of conflicting candidate extraction results: the conflicting candidate field value is "1,008.00," originating from the large language model text extractor and the visual large language model, corresponding to two initial confidence levels of 90 and 95, respectively. Further, based on the document image to be identified (i.e., the invoice image), consistency verification is performed on the above conflicting extraction results to obtain the target verification result corresponding to each conflicting extraction result.

[0076] Step S203: Based on each conflict extraction result and its corresponding target verification result, perform multi-source credibility analysis to obtain the multi-source credibility score corresponding to each conflict extraction result.

[0077] Among them, the multi-source credibility score is used to characterize the overall reliability of the conflict candidate field values ​​in the conflict candidate results.

[0078] It should be noted that the "multi-source candidate extraction results" in this application refer to multiple text extraction models from different sources and with heterogeneous principles, each independently analyzing and extracting target fields from the document image to be recognized, thereby providing diverse candidate field values ​​and corresponding initial confidence levels. Based on this, it can be understood that multi-source credibility analysis is used to comprehensively evaluate the overall credibility of conflict candidate field values ​​by integrating conflict extraction results (including conflict field values ​​and at least one initial confidence level corresponding to conflict candidate field values) and corresponding target verification results, generating a multi-source credibility score that characterizes the overall reliability of conflict candidate field values.

[0079] Understandably, by conducting multi-source credibility analysis on conflict candidate field values, it is possible to effectively suppress the "false high credibility" phenomenon caused by the superposition of original confidence levels or the output of a single high-scoring model in low-quality extraction results, thereby avoiding blind trust in low-quality information sources.

[0080] Step S204: Determine the target multi-source credibility score that meets the preset credibility decision conditions from multiple multi-source credibility scores; determine the target extraction result corresponding to the target field based on the target multi-source credibility score and the target conflict extraction result corresponding to the target multi-source credibility score.

[0081] Among them, the preset credible decision conditions refer to the decision criteria for determining whether the extracted conflict candidates can be adopted as the final target extraction result. The preset credible decision conditions need to be pre-set based on business accuracy requirements, risk tolerance, etc., and are not specifically limited here.

[0082] The target extraction results include the target field value and the target confidence score; the target field value is the target conflict field value in the target conflict extraction results; and the target confidence score is the target multi-source credibility score.

[0083] In an exemplary embodiment, preset credible decision conditions may include, but are not limited to, a confidence threshold condition and a preset multi-source consensus condition. The confidence threshold condition is used to filter conflict extraction results whose multi-source confidence scores are greater than or equal to the preset confidence threshold. The preset multi-source consensus condition is used to avoid the risk of "isolated evidence," that is, to prevent a single model (especially a high-weight model) from directly passing a decision due to its high confidence when generating illusions or misjudgments. The preset multi-source consensus condition requires that the conflict candidate field value in a conflict extraction result must be supported by at least two heterogeneous text extraction models to be considered a valid consensus. It is understood that by setting preset multi-source consensus conditions, it can be ensured that even if the multi-source confidence score of a conflict extraction result is high, if the conflict extraction result is proposed by only a single source and does not receive independent support from at least one other heterogeneous text extraction model, the conflict extraction result will be rejected automatically and will be manually reviewed to ensure the accuracy of the field extraction.

[0084] By combining a preset confidence threshold with multi-source consensus as a dual constraint, it can effectively prevent the "high-score misjudgment" phenomenon caused by illusions or misjudgments from a single source (such as a large language model). Even if the confidence of a certain extraction result is inflated, it cannot pass the decision without cross-validation from other heterogeneous models. This effectively improves the accuracy, reliability, and robustness of field extraction, making it suitable for high-risk business scenarios with stringent data quality requirements, such as finance and government.

[0085] For example, the process involves obtaining multi-source candidate extraction results corresponding to the target field in the document image to be identified; detecting whether there are conflicting extraction results among the multi-source candidate extraction results; if at least two conflicting extraction results are detected, then based on the document image to be identified, performing consistency verification on each conflicting extraction result to obtain the target verification result corresponding to each conflicting extraction result; performing multi-source credibility analysis based on each conflicting extraction result and its corresponding target verification result to obtain the multi-source credibility score corresponding to each conflicting extraction result; determining the target multi-source credibility score that meets the preset credibility decision conditions from multiple multi-source credibility scores; determining the target conflict candidate field value in the target conflict extraction result corresponding to the target multi-source credibility score as the target field value corresponding to the target field; and determining the target multi-source credibility score as the target confidence level corresponding to the target field value.

[0086] If no conflicting extraction results are detected among the multi-source candidate extraction results, it indicates that the candidate field values ​​output by all heterogeneous text extraction models form only one unique candidate group after clustering. At this point, it can be considered that all heterogeneous text extraction models have reached a consensus, and there is no need to perform consistency verification. Multi-source credibility analysis can be performed directly based on the multi-source candidate extraction results, and the corresponding multi-source credibility score can be obtained by combining the initial confidence of each source. If the multi-source credibility score meets the preset credibility decision conditions, the multi-source candidate extraction results are determined as the target extraction results corresponding to the target field. Otherwise, even if there is no conflict, it is still regarded as a low-confidence result, and manual review or automatic output rejection is required to ensure data quality and system reliability.

[0087] In this embodiment, multi-source candidate extraction results of target fields in the document image to be identified are obtained. When at least two conflicting extraction results are detected among the multi-source candidate extraction results, consistency verification is performed on each conflicting extraction result based on the document image to be identified, generating target verification results corresponding to each conflicting extraction result, providing an analytical basis for subsequent multi-source credibility analysis. Furthermore, multi-source credibility analysis is performed based on the conflicting extraction results and their corresponding target verification results to obtain the corresponding multi-source credibility scores. This effectively suppresses the "false high credibility" phenomenon caused by the superposition of original confidence scores or the output of a single high-scoring model due to low-quality extraction results, thereby avoiding blind trust in low-quality sources. Finally, based on preset credibility decision conditions, further decision analysis is performed to select the target conflicting extraction results corresponding to the target credibility scores that meet the preset credibility decision conditions, and the target extraction results corresponding to the target fields are determined accordingly, improving the accuracy and credibility of document field extraction.

[0088] In one embodiment, Figure 3This is a flowchart illustrating the target verification result determination step in one embodiment; based on the document image to be identified, consistency verification is performed on each conflict extraction result to obtain the target verification result corresponding to each conflict extraction result, including the following steps:

[0089] Step S301: Input the document image to be recognized and each conflict extraction result into multiple heterogeneous verification models in parallel, perform visual consistency verification on each conflict extraction result, and obtain the first verification result corresponding to each conflict extraction result.

[0090] Among them, the heterogeneous verification model is used to independently evaluate the visual consistency between each conflict extraction result and the document image to be identified, so as to quantify the degree of consistency between each conflict extraction result and the document image to be identified at the visual and semantic levels.

[0091] The heterogeneous verification model can include, but is not limited to, multimodal large language models (MLMs) or dedicated visual understanding models, such as GPT-4o and Qwen-VL. It should be noted that the heterogeneous verification model possesses end-to-end image-text joint understanding capabilities, and can directly receive the document image to be recognized and the conflict extraction results to complete visual consistency verification.

[0092] The first verification result is used to characterize the degree of visual consistency between the conflict extraction result and the document image to be identified. It should be noted that the first verification result is not derived from the judgment of a single heterogeneous verification model, but is obtained by fusing and calculating multiple visual consistency results generated by multiple heterogeneous verification models (such as GPT-4o, Qwen-VL, etc.) for the same conflict extraction result.

[0093] Multiple visual consistency results are independently output by multiple heterogeneous verification models; there is a one-to-one correspondence between the multiple visual consistency results and the multiple heterogeneous verification models. A visual consistency result refers to the quantitative evaluation value generated by the corresponding heterogeneous verification model for the degree of visual consistency between the conflict candidate field values ​​in the conflict extraction results and the actual content of the corresponding region in the document image to be identified. Visual consistency results can be quantified, but are not limited to, using confidence scores (e.g., 0-100 points), reflecting the strength of a single heterogeneous verification model's judgment on whether the conflict candidate field values ​​in the conflict extraction results "appear to truly exist in the image."

[0094] Understandably, the first verification result is essentially a "majority opinion" formed collaboratively by multiple heterogeneous verification models to answer the question of "which conflict candidate field value in the conflict extraction result looks more real and credible in the original document image to be identified". By integrating the independent visual consistency judgments of different heterogeneous verification models on the same conflict extraction result, it can effectively smooth out the misjudgments that a single heterogeneous verification model may cause due to illusion, training bias, image noise, blur or layout interference, and provide a reliable basis for subsequent credibility analysis and decision-making.

[0095] Step S302: Based on the preset rigid verification rules, perform rule consistency verification on each conflict extraction result to obtain the second verification result corresponding to each conflict extraction result.

[0096] The preset rigid validation rules include multiple rigid validation rules. Rigid validation rules refer to validation conditions that cannot be violated, set based on business logic, format specifications, or mathematical constraints. For example: date logic is valid (e.g., "February 30th" does not exist); invoice code / number must match the coding structure defined by the tax system, etc.

[0097] Understandably, by performing rule-based consistency checks on each conflict extraction result, the problem of "which value absolutely cannot be true" can be effectively solved. If a conflict extraction result violates any rigid validation rule, its corresponding second validation result should be deemed unsuccessful. In this case, regardless of the visual consistency level of the conflict extraction result (i.e., the first validation result), it is considered an invalid candidate and should be directly eliminated or have its confidence level reduced to zero, and should not enter the final decision-making process. This ensures strict adherence to business compliance and data legality in the extraction of key fields, fundamentally avoiding the risks of logical errors or illegal formats.

[0098] Step S303: Based on each first verification result and each second verification result, determine the target verification result corresponding to each conflict extraction result.

[0099] In an exemplary embodiment, the method for determining the target verification result corresponding to each conflict extraction result based on each first verification result and each second verification result may be: multiplying the first verification result and the second verification result corresponding to the conflict extraction result to obtain the target verification result corresponding to the conflict extraction result. For example: assuming the first verification result corresponding to the conflict extraction result is... The second verification result is The target verification result corresponding to the conflict extraction result is obtained as follows: This ensures that any violation of the preset rigid verification rules ( The target verification result corresponding to the conflict extraction result is directly cleared to zero.

[0100] It should be noted that, compared to traditional arbitration mechanisms, this embodiment has a significant advantage in handling the model "illusion" problem. Traditional methods typically rely on multiple voting by models with similar origins or structures. When these models develop a systematic illusion due to shared training data, architecture, or cue bias, the arbitration mechanism may mistakenly regard erroneous but consistent outputs as "consensus," thus adopting false results with high confidence. However, this application, by introducing a dual verification strategy based on multiple heterogeneous verification models and preset rigid verification rules, can effectively avoid the false consensus formed by the illusion of origin. Even if multiple heterogeneous text extraction models consistently output a candidate field value, as long as they violate any rigid verification rule, a rejection mechanism will be triggered, and the result will be reliably filtered out.

[0101] In this embodiment, by introducing a dual verification strategy of visual consistency verification and rule consistency verification, it can not only effectively smooth out misjudgments that may be caused by illusions, training bias, image noise, blurring or layout interference in a single heterogeneous verification model, providing a more reliable and robust basis for subsequent credibility analysis and decision-making; it can also effectively avoid the risk of logical errors or illegal formats, laying the foundation for improving the accuracy of field extraction and business compliance.

[0102] In one embodiment, such as Figure 4 As shown, Figure 4 This is a flowchart illustrating the first verification result determination step in one embodiment; the document image to be identified and each conflict extraction result are input in parallel to multiple heterogeneous verification models, and visual consistency verification is performed on each conflict extraction result to obtain the first verification result corresponding to each conflict extraction result, including the following steps:

[0103] Step S401: The document image to be identified and each conflict extraction result are input into multiple heterogeneous verification models in parallel. Through multiple heterogeneous verification models, visual consistency verification is performed on each conflict extraction result to obtain multiple visual consistency results corresponding to each conflict extraction result.

[0104] Among them, multiple visual consistency results are independently output by multiple heterogeneous verification models; there is a one-to-one correspondence between multiple visual consistency results and multiple heterogeneous verification models.

[0105] The visual consistency result refers to the quantitative evaluation value generated by the corresponding heterogeneous verification model for the degree of visual consistency between the conflict candidate field values ​​in the conflict extraction results and the actual content of the corresponding region in the document image to be identified. The visual consistency result can be quantified by confidence scores (such as 0 to 100 points), reflecting the strength of the judgment of a single heterogeneous verification model on whether the conflict candidate field values ​​in the conflict extraction results "appear to exist in the image".

[0106] In an exemplary embodiment, step S401 further includes: constructing verification prompts based on each conflict extraction result and the document image to be identified; encapsulating the constructed verification prompts into a standard API (Application Programming Interface) request packet (containing messages and image fields); and sending the packet in parallel to multiple heterogeneous verification models via HTTPS protocol, so that the multiple heterogeneous verification models can independently perform inference and scoring according to the instructions of the verification prompts, thereby outputting a visual consistency result (such as a confidence score of 0 to 100) reflecting whether the conflict candidate field value is visually consistent with the content of the document image to be identified.

[0107] The verification prompts may include, but are not limited to, the URL (Uniform Resource Locator) of the document image to be identified, field definitions, and a list of candidate conflict field values. The URL of the document image to be identified points to its storage location. The list of candidate conflict field values ​​includes at least two candidate conflict field values.

[0108] Understandably, the verification prompts are used to guide the attention mechanism of the heterogeneous verification model to focus on specific regions in the image, rather than describing the entire image in general terms. They are also used to constrain the divergent natural language generation capabilities of the heterogeneous verification model into a convergent, structured numerical scoring task, so that its results can be directly calculated by downstream algorithms.

[0109] In one exemplary embodiment, the visual consistency verification process of the heterogeneous verification model may include: preprocessing the document image to be identified: converting the document image to be identified into standard RGB format, adaptively scaling it to the maximum resolution supported by the heterogeneous verification model (e.g., the long side does not exceed 2048 pixels), and encoding it into a Base64 string; if necessary, performing privacy desensitization on the target area where the non-target field is located. Subsequently, based on the preprocessed document image to be identified, the values ​​of each conflict candidate field, and the field definition, a structured verification prompt word is constructed, and the verification prompt word is input in parallel to multiple heterogeneous verification models, so that multiple heterogeneous verification models focus on the target field area in the image according to the verification prompt word instructions, and perform visual consistency evaluation on each conflict candidate field value by combining character shape, language habits, and contextual clues, and perform quantitative scoring according to preset rules: the scoring logic is as follows: perfect match (100 points): the conflict candidate field value is completely consistent with the image content in terms of characters, spelling, and format. High confidence (80-95 points): Slight OCR noise exists (e.g., '0' is recognized as 'O') but the semantics are unambiguous, or although the spelling is different, it meets specific language transliteration rules (e.g., the correct English spelling of a Thai name). Mismatch (0-20 points): The conflicting candidate field value is completely absent in the image, or seriously conflicts with the image content (e.g., language mismatch, numerical error). Finally, each heterogeneous verification model will output a strictly structured JSON result (e.g., {"name": [{"value": "Candidate A", "score": 0}, {"value": "Candidate B", "score": 95}]}).

[0110] Step S402: For each conflict extraction result, based on the credibility weight coefficients corresponding to each of the multiple heterogeneous verification models, perform a weighted average of the multiple visual consistency results corresponding to the conflict extraction results to obtain the first verification result corresponding to the conflict extraction results.

[0111] The credibility weight coefficients corresponding to the heterogeneous verification model can be dynamically adjusted based on the performance of the heterogeneous verification model, and no specific limitations are made here.

[0112] For example, the confidence weight coefficients corresponding to each heterogeneous verification model are denoted as follows: For each conflict extraction result, the credibility weight coefficients corresponding to each of the multiple heterogeneous verification models are used. Multiple visual consistency results corresponding to conflict extraction results A weighted average is performed to obtain the first verification result corresponding to the conflict extraction result. The first verification result is... ;in, , is the visual consistency result output by the i-th heterogeneous verification model for the conflict extraction result; , where is the credibility weight coefficient corresponding to the i-th heterogeneous verification model.

[0113] For example, for the conflict candidate field value "A" corresponding to the target field "Name", the heterogeneous validation model GPT-4o (denoted as GPT-4o) The visual consistency results given are: The confidence weight coefficient of the heterogeneous verification model GPT-4o is: Heterogeneous verification model Qwen-VL (denoted as The visual consistency results given are: The confidence weight coefficient of the heterogeneous validation model Qwen-VL is: The first verification result corresponding to the conflict candidate field value "A" is obtained by weighted averaging of multiple visual consistency results. .

[0114] In this embodiment, based on the credibility weight coefficients corresponding to each of the multiple heterogeneous verification models, a weighted average is performed on the multiple visual consistency results corresponding to the conflict extraction result to obtain the first verification result corresponding to the conflict extraction result. This can effectively solve the problem of "which conflict candidate field value in the conflict extraction result looks more real and credible in the original document image to be identified". By integrating the independent visual consistency judgments of different heterogeneous verification models on the same conflict extraction result, it can effectively smooth out the misjudgments that may be caused by a single heterogeneous verification model due to illusion, training bias, image noise, blur or layout interference, and provide a reliable basis for subsequent multi-source credibility analysis and decision-making.

[0115] In one embodiment, such as Figure 5 As shown, Figure 5 This is a flowchart illustrating the second verification result determination step in one embodiment; based on preset rigid verification rules, rule consistency verification is performed on each conflict extraction result to obtain the second verification result corresponding to each conflict extraction result, including the following steps:

[0116] Step S501: Identify the document feature information of the document image to be identified.

[0117] Document feature information may include, but is not limited to, textual semantic information and visual layout information. Textual semantic information refers to the character content in the document that carries specific business meaning; this information can be presented in key-value pairs (e.g., "Name: Zhang San") or tables. Visual layout information includes version type, etc.

[0118] It should be noted that the document feature information recognition method can be implemented using traditional feature recognition techniques, which will not be elaborated here.

[0119] Step S502: Based on the document feature information, determine at least one target rigid verification rule that matches the document feature information from the preset rigid verification rules.

[0120] The preset rigid validation rules include multiple rigid validation rules; these rigid validation rules refer to inviolable validation conditions set based on business logic, format specifications, or mathematical constraints. Rigid validation rules may include, but are not limited to, regional language rigid validation rules, format rigid validation rules, and logical rigid validation rules.

[0121] In an exemplary embodiment, if the document image to be identified contains a "VAT general invoice from country A", then based on the document feature information, the target rigid verification rule associated with it is automatically determined from the preset rigid verification rules, such as "the verification code is a 6-digit number" or "the amount field must retain two decimal places". If the document image to be identified contains a "ID card from country B", then based on the document feature information, the target rigid verification rule associated with it is automatically determined from the preset rigid verification rules, such as "the semantics are in the language of country B".

[0122] Understandably, dynamically matching target rigid verification rules based on document feature information can ensure the accuracy and scenario adaptability of rule consistency verification, avoid invalid verification of irrelevant rules, and ensure that key business constraints are strictly enforced under the corresponding document type.

[0123] Step S503: If the conflict extraction result does not conform to any target rigid verification rule, then the second verification result corresponding to the conflict extraction result is set to a preset value.

[0124] The preset value is used to explicitly identify conflict extraction results that are deemed invalid due to violation of the target rigid verification rules. The preset value can be set to an extremely low score (such as 0 points) to ensure that it is directly excluded in subsequent multi-source credibility analysis, and even if its first verification result is high, it should not be used as a candidate for the final output.

[0125] For example, the formula for calculating the second verification result is as follows: ;in, This represents the rule verification result corresponding to the conflict candidate field value under the rigid verification rule for the i-th target; if the conflict candidate field value fully conforms to the rigid verification rule for the i-th target, then... If the conflict candidate field value does not conform to the rigid verification rule of the i-th target, then .

[0126] For example, by identifying the document feature information of the document image to be identified, at least one target rigid verification rule matching the document feature information is determined from the preset rigid verification rules. Then, it is determined whether the conflict candidate field value in the conflict extraction result conforms to each target rigid verification rule. If the conflict candidate field value in the conflict extraction result completely conforms to all target rigid verification rules, then the second verification result is obtained. If the conflict candidate field values ​​in the conflict extraction results do not conform to any target rigid verification rule (i.e., there exists any conflict candidate field value), then the conflict candidate field values ​​do not conform to any target rigid verification rule. This leads to the entire If the second verification result is 0, the conflict candidate field value needs to be forcibly removed. It is understandable that only conflict candidate field values ​​that simultaneously satisfy all the target rigid verification rules can obtain a non-zero second verification result, thus providing a legitimate and compliant basic input for subsequent multi-source credibility analysis.

[0127] In this embodiment, by introducing preset rigid verification rules, it is ensured that business compliance and data legality are strictly followed in the extraction of key fields, fundamentally avoiding the risk of logical errors or illegal formats, and laying the foundation for improving the accuracy of field extraction and business compliance.

[0128] In one embodiment, such as Figure 6 As shown, Figure 6 This is a flowchart illustrating the steps for determining the multi-source credibility score in one embodiment. Based on each conflict extraction result and its corresponding target verification result, multi-source credibility analysis is performed to obtain the multi-source credibility score corresponding to each conflict extraction result, including the following steps:

[0129] Step S601: For each initial confidence level corresponding to the conflict candidate field value, calculate the single-source credibility probability of the conflict candidate field value on the associated text extraction model associated with the initial confidence level, based on the initial confidence level and the target verification result.

[0130] Among them, the associated text extraction model refers to the heterogeneous text extraction model that outputs conflict candidate field values ​​and provides corresponding initial confidence scores.

[0131] It is understandable that the same conflict candidate field value may be jointly output by multiple heterogeneous text extraction models. For example, taking Table 1 described in the above embodiment as an example, the conflict candidate field value "1,008.00" is output by the large language model text extractor and the visual large language model, with initial confidence levels of 90 and 95, respectively. Therefore, it can be determined that the large language model text extractor is the associated text extraction model associated with the initial confidence level of 90, and the visual large language model is the associated text extraction model associated with the initial confidence level of 95.

[0132] Among them, the single source credibility probability is determined by the corresponding initial confidence level and the target verification result, and is used to characterize the reliability of the conflict candidate field value under the single source path.

[0133] In one example embodiment, for each initial confidence level corresponding to a conflict candidate field value, based on the initial confidence level and the target verification result, the single-source credibility probability of the conflict candidate field value on the associated text extraction model associated with the initial confidence level is calculated, including: for each initial confidence level corresponding to a conflict candidate field value... Obtain the initial confidence level The single-source credibility weight coefficient corresponding to the associated text extraction model Based on the single source credibility weight coefficient Initial confidence level and target verification results Calculate the single-source confidence probability of conflict candidate field values ​​on the associated text extraction model associated with the initial confidence level. :

[0134] ;

[0135] Among them, the single source credibility weight coefficient A value less than 1 is used to prevent the risk of "isolated evidence." Specifically, if a conflict candidate field value is extracted by only one heterogeneous text extraction model (i.e., lacking multi-source consensus), even if its target verification result gives a perfect score (i.e., 100 points), the system will still pass the test if the single-source credibility weight coefficient is less than 1. This suppresses the credibility of the heterogeneous text extraction model. Therefore, it ensures that the highest achievable single-source credibility probability under a single source will not exceed the single-source credibility weight coefficient. For example, when the single source confidence weight coefficient is set to 0.9, even with an initial confidence level... Verification results with the target All scores are 100, and the final upper limit of the credibility probability of a single source is 0.9, in order to avoid "high-score misjudgment" caused by illusion or misjudgment due to a single model.

[0136] Step S602: Perform joint probability analysis based on the credibility probabilities of each individual source to obtain the multi-source credibility score corresponding to the conflict extraction result.

[0137] Joint probability analysis, which can be implemented using methods such as Bayesian probability, treats the credibility probability of each individual source as an independent event to evaluate the overall credibility of conflict candidate field values.

[0138] In an exemplary embodiment, the method for obtaining the multi-source credibility score corresponding to the conflict extraction result by performing joint probability analysis based on the credibility probabilities of each individual source can be as follows: performing joint probability analysis based on the credibility probabilities of each individual source to obtain the multi-source joint probability corresponding to the conflict candidate field value in the conflict extraction result; and performing numerical quantization processing on the multi-source joint probability to obtain the corresponding multi-source credibility score.

[0139] The joint probability of multiple sources is used to characterize the joint probability that at least one of the multiple associated text extraction models corresponding to conflicting candidate field values ​​is correct. The formula for calculating the joint probability of multiple sources is as follows:

[0140] ;

[0141] The formula for calculating the credibility score of multiple sources is as follows:

[0142] ;

[0143] in, , where is the joint probability of multiple sources corresponding to the conflict extraction result; , which is the value of the conflict candidate field in the conflict extraction results; Let P1 be the single-source confidence probability of the conflict candidate field value output by the i-th associated text extraction model. For example, if there is only one source (i.e., there is only one associated text extraction model corresponding to the conflict candidate field value), such as P1=0.9, then... If there are two information sources (i.e., two related text extraction models corresponding to conflict candidate field values), such as P1=0.9 and P2=0.95, then This significantly improved overall credibility.

[0144] In this embodiment, joint probability analysis can naturally amplify consensus signals and suppress isolated high scores, thereby enhancing sensitivity to multi-source consistency while maintaining high accuracy, and providing a robust and quantifiable decision basis for final field determination.

[0145] In one embodiment, such as Figure 7 As shown, Figure 7 This is a flowchart illustrating the target extraction result determination step in one embodiment; the target multi-source credibility score that meets the preset credibility decision conditions is determined from multiple multi-source credibility scores, and the target conflict extraction result corresponding to the target multi-source credibility score is determined as the target extraction result corresponding to the target field, including the following steps:

[0146] Step S701: The multi-source credibility score with the highest score among multiple multi-source credibility scores is determined as the candidate multi-source credibility score.

[0147] It is understandable that the highest score does not necessarily mean that the corresponding conflict candidate field value is credible enough. Further confidence threshold determination is required by combining the preset confidence threshold.

[0148] Step S702: If the candidate multi-source credibility score is greater than or equal to the preset credibility threshold, and the conflict extraction result corresponding to the candidate multi-source credibility score satisfies the preset multi-source consensus condition, then the candidate multi-source credibility score is determined as the target multi-source credibility score.

[0149] The preset credibility threshold refers to a pre-set lower limit standard for credibility, used to determine whether the conflict extraction results corresponding to the credibility scores of candidate multi-source sources have sufficient reliability. The preset credibility threshold is between 0 and 1 (e.g., 0.90, 0.95, or 0.98), and its specific value needs to be set according to the accuracy requirements and risk tolerance of the actual business scenario, and is not specifically limited here.

[0150] The pre-set multi-source consensus condition is designed to avoid the risk of "isolated evidence," preventing a single model (especially a high-weight model) from making a decision directly due to its high confidence level when it generates illusions or misjudgments. The pre-set multi-source consensus condition requires that the conflict candidate field value in a conflict extraction result must be supported by at least two heterogeneous text extraction models to be considered a valid consensus. Understandably, by setting the pre-set multi-source consensus condition, it ensures that even if a conflict extraction result has a high multi-source confidence score, if the conflict extraction result is proposed by only a single source and does not receive independent support from at least one other heterogeneous text extraction model, the conflict extraction result will be rejected automatically.

[0151] Step S703: Determine the target conflict candidate field value in the target conflict extraction result corresponding to the target multi-source credibility score as the target field value corresponding to the target field; determine the target multi-source credibility score as the target confidence level corresponding to the target field value.

[0152] In one specific embodiment, assuming a preset credibility threshold of 90 points, taking the extraction of the "name" field from a certain type of official ID document as an example, assume there are two conflict candidate field values: Conflict candidate field value A is in Latin letter form and is extracted by four heterogeneous text extraction models; conflict candidate field value B is in local text form and is only output by the regular expression rule extractor. First, based on multiple heterogeneous verification models, the first verification result of each conflict candidate field value is calculated, and a second verification result is generated by combining it with preset rigid verification rules (such as language rules). Then, the first verification result and the second verification result are multiplied to obtain the target verification result. Specifically: the first verification result of conflict candidate field value A is full marks, but because it does not contain local text, it violates the language rules, and the second verification result is 0, resulting in its target verification result. =0, based on The calculated single-source credibility probability is 0, so it is directly eliminated without needing to perform downstream joint probability analysis. The first verification result of the conflict candidate field value B is 95 points, the second verification result is 1.0, and the target verification result is 95; however, since the conflict candidate field value B is only output by a single heterogeneous text extraction model, its single-source credibility weight coefficient is set to 0.9. The calculated single-source credibility probability is 1 × 0.9 × 0.95 = 0.855, and the multi-source credibility score is 85.5. Although the conflict candidate field value B has the highest multi-source credibility score in the conflict, it does not reach the preset credibility threshold (90 points). Therefore, the system determines that it cannot be automatically adopted, triggering a manual review process, effectively balancing compliance, accuracy, and risk control.

[0153] In this embodiment, by combining a preset confidence threshold with multi-source consensus as a dual constraint, the phenomenon of "high-score misjudgment" caused by illusions or misjudgments from a single source (such as a large language model) can be effectively prevented. Even if the confidence of a certain extraction result is artificially high, it cannot pass the decision without cross-validation from other heterogeneous models. This effectively improves the accuracy, reliability, and robustness of field extraction, making it suitable for high-risk business scenarios with stringent data quality requirements, such as finance and government.

[0154] In other embodiments, by introducing a full-link structured backtracking and a hierarchical consistency verification visualization report, the "black box" dilemma of traditional intelligent document processing solutions is effectively broken, and pixel-level error location and decision interpretability are achieved.

[0155] Specifically, end-to-end structured backtracking means that the system not only outputs the final field results but also breaks down the entire processing flow into multiple standardized independent steps. Key data for each step, including inputs (such as original image slices or OCR segments with coordinates), outputs (such as candidate field values ​​extracted by each model), and intermediate parameters (such as the single-source credibility weight coefficients of each heterogeneous text extraction model, the credibility weight coefficients of each heterogeneous verification model, and the first / second verification results), are all structured, serialized, and persistently stored. When an error occurs, maintenance personnel can retrieve a complete execution snapshot of the task using a unique Trace ID. For example, if the final amount field is incorrect, the source of the problem can be precisely traced: the OCR engine misidentifies "8" as "3," there is a flaw in the regularization rule logic, or a large model experiences a hallucination due to a deviation in the prompt word.

[0156] During the decision-making phase, the system automatically generates a consistency verification report, presenting the conflict decision-making process in a hierarchical and visual manner. For each conflicting field, the report clearly lists the detailed scores obtained from multiple heterogeneous verification models for the values ​​of each candidate conflict field, resolving the inexplicable question of "why the system chose A instead of B".

[0157] Furthermore, in other embodiments, the system supports manual correction of erroneous fields directly on the web. The corrected values ​​are marked as manually reviewed (with an output confidence level considered 100%) and written back to the processing pipeline, forming a closed-loop feedback mechanism of "error-correction-verification". More importantly, these manually intervened samples are automatically accumulated into a hard sample library, used for subsequent targeted fine-tuning of the heterogeneous text extraction model, thereby fundamentally suppressing the recurrence of similar errors and continuously improving the system's robustness and business adaptability.

[0158] In one specific embodiment, the document field extraction method includes the following steps:

[0159] Step 1: Submit a document field extraction request based on the image of the document to be identified to the server via the client.

[0160] Step 2: The server responds to the document field extraction request, calls the OCR engine to recognize the document image to be recognized, and generates an unstructured text list.

[0161] Step 3: In parallel, call multiple heterogeneous text extraction models, taking the unstructured text list and / or the document image to be identified as the input of the heterogeneous text extraction models. Through multiple heterogeneous text extraction models, perform multi-source parallel field extraction on the document image to be identified, and obtain the multi-source candidate extraction results corresponding to each target field in the document image to be identified.

[0162] Step 4: If at least two conflicting extraction results are detected in the multi-source candidate extraction results, the document image to be identified and the values ​​of each conflicting candidate field are input in parallel into multiple heterogeneous verification models to perform visual consistency verification on each conflicting candidate field value, and obtain the first verification result corresponding to each conflicting candidate field value.

[0163] The conflict extraction results include conflict candidate field values ​​and at least one initial confidence level corresponding to the conflict extraction results.

[0164] Step 5: Based on the preset rigid verification rules, perform rule consistency verification on the values ​​of each conflict candidate field to obtain the second verification result corresponding to each conflict candidate field value.

[0165] Step 6: Based on the first verification results and the second verification results, determine the target verification result corresponding to each conflict candidate field value.

[0166] Step 7: For each initial confidence level corresponding to the conflict candidate field value, calculate the single-source credibility probability of the conflict candidate field value on the associated text extraction model associated with the initial confidence level, based on the initial confidence level and the target verification result.

[0167] Step 8: Perform joint probability analysis based on the credibility probabilities of each individual source to obtain the multi-source credibility score corresponding to the conflict candidate field value.

[0168] Step 9: Determine the target multi-source credibility score that meets the preset credibility decision conditions from multiple multi-source credibility scores, determine the target conflict candidate field value corresponding to the target field as the target field value, and determine the target multi-source credibility score as the target confidence level corresponding to the target field value.

[0169] In one specific embodiment, if no conflicting extraction results are detected among the multi-source candidate extraction results, it can be assumed that all heterogeneous text extraction models have reached a consensus, and no consistency verification is required (i.e., the target verification result defaults to a full score of 100). Multi-source credibility analysis can be directly performed based on the multi-source candidate extraction results, and the corresponding multi-source credibility score is obtained by combining the initial confidence levels of each source. If the multi-source credibility score meets the preset credibility decision conditions, the multi-source candidate extraction result is determined as the target extraction result corresponding to the target field. Otherwise, even without conflict, it is still considered a low-credibility result, requiring manual review or rejection of automatic output to ensure data quality and system reliability. In this embodiment, the method for determining the multi-source credibility score is the same in principle as the method described in the above embodiments, and will not be repeated here.

[0170] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0171] Based on the same inventive concept, this application also provides a document field extraction device for implementing the document field extraction method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more document field extraction device embodiments provided below can be found in the limitations of the document field extraction method described above, and will not be repeated here.

[0172] In one exemplary embodiment, such as Figure 8 As shown, a document field extraction device is provided, including: an acquisition module 801, a verification module 802, a multi-source credibility analysis module 803, and a decision module 804, wherein:

[0173] The acquisition module 801 is used to acquire the multi-source candidate extraction results corresponding to the target field in the document image to be recognized;

[0174] The verification module 802 is used to perform consistency verification on each conflicting extraction result based on the document image to be identified if at least two conflicting extraction results are detected among the multi-source candidate extraction results, so as to obtain the target verification result corresponding to each conflicting extraction result.

[0175] The multi-source credibility analysis module 803 is used to perform multi-source credibility analysis based on each conflict extraction result and its corresponding target verification result, and to obtain the multi-source credibility score corresponding to each conflict extraction result.

[0176] The decision module 804 is used to determine the target multi-source credibility score that meets the preset credibility decision conditions from multiple multi-source credibility scores, and to determine the target extraction result corresponding to the target field based on the target multi-source credibility score and the target conflict extraction result corresponding to the target multi-source credibility score.

[0177] The aforementioned document field extraction device acquires multi-source candidate extraction results of target fields in the document image to be identified. When at least two conflicting extraction results are detected among the multi-source candidate extraction results, a consistency check is performed on each conflicting extraction result based on the document image to be identified, generating a target check result corresponding to each conflicting extraction result, providing an analytical basis for subsequent multi-source credibility analysis. Furthermore, multi-source credibility analysis is performed based on the conflicting extraction results and their corresponding target check results to obtain the corresponding multi-source credibility score. This effectively suppresses the "false high credibility" phenomenon caused by the superposition of original confidence scores or the output of a single high-scoring model due to low-quality extraction results, thereby avoiding blind trust in low-quality sources. Finally, based on preset credibility decision conditions, further decision analysis is performed to select the target conflicting extraction result corresponding to the target credibility score that meets the preset credibility decision conditions, and the target extraction result corresponding to the target field is determined accordingly, improving the accuracy and credibility of document field extraction.

[0178] In one embodiment, the verification module 802 is further configured to:

[0179] The document image to be identified and the conflict extraction results are input into multiple heterogeneous verification models in parallel. Visual consistency verification is performed on each conflict extraction result to obtain the first verification result corresponding to each conflict extraction result.

[0180] Based on the preset rigid verification rules, the rule consistency verification is performed on each conflict extraction result to obtain the second verification result corresponding to each conflict extraction result.

[0181] Based on each first verification result and each second verification result, determine the target verification result corresponding to each conflict extraction result.

[0182] In one embodiment, the verification module 802 is further configured to:

[0183] The document image to be identified and the conflict extraction results are input into multiple heterogeneous verification models in parallel. The visual consistency of each conflict extraction result is verified by the multiple heterogeneous verification models, and multiple visual consistency results corresponding to each conflict extraction result are obtained.

[0184] For each conflict extraction result, based on the credibility weight coefficients of each of the multiple heterogeneous verification models, a weighted average is performed on the multiple visual consistency results corresponding to the conflict extraction results to obtain the first verification result corresponding to the conflict extraction results.

[0185] In one embodiment, the verification module 802 is further configured to:

[0186] Identify document feature information from the image of the document to be identified;

[0187] Based on the document feature information, at least one target rigid verification rule that matches the document feature information is determined from the preset rigid verification rules;

[0188] If the conflict extraction result does not conform to any target rigid verification rule, the second verification result corresponding to the conflict extraction result will be set to the preset value.

[0189] In one embodiment, the conflict extraction result includes conflict candidate field values ​​and at least one initial confidence level corresponding to the conflict candidate field values; the multi-source credibility analysis module 803 is further used for:

[0190] For each initial confidence level corresponding to a conflict candidate field value, the single-source credibility probability of the conflict candidate field value on the associated text extraction model associated with the initial confidence level is calculated based on the initial confidence level and the target verification result.

[0191] Based on the credibility probability of each individual source, joint probability analysis is performed to obtain the multi-source credibility score corresponding to the conflict extraction result.

[0192] In one embodiment, the decision module 804 is further configured to:

[0193] The multi-source credibility score with the highest score among multiple multi-source credibility scores is determined as the candidate multi-source credibility score;

[0194] If the candidate multi-source credibility score is greater than or equal to the preset credibility threshold, and the conflict extraction result corresponding to the candidate multi-source credibility score meets the preset multi-source consensus condition, then the candidate multi-source credibility score is determined as the target multi-source credibility score.

[0195] The target conflict candidate field value in the target conflict extraction result corresponding to the target multi-source credibility score is determined as the target field value corresponding to the target field; the target multi-source credibility score is determined as the target confidence level corresponding to the target field value.

[0196] Each module in the aforementioned document field extraction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0197] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data related to document field extraction. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a document field extraction method.

[0198] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0199] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0200] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0201] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0202] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0203] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0204] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0205] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for extracting document fields, characterized in that, The method includes: Obtain the multi-source candidate extraction results corresponding to the target fields in the image of the document to be identified; If at least two conflicting extraction results are detected among the multi-source candidate extraction results, then based on the document image to be identified, a consistency check is performed on each of the conflicting extraction results to obtain the target check result corresponding to each of the conflicting extraction results. Based on the conflict extraction results and their corresponding target verification results, multi-source credibility analysis is performed to obtain the multi-source credibility score corresponding to each conflict extraction result. From the multiple multi-source credibility scores, a target multi-source credibility score that meets the preset credibility decision conditions is determined. Based on the target multi-source credibility score and the target conflict extraction result corresponding to the target multi-source credibility score, the target extraction result corresponding to the target field is determined.

2. The method according to claim 1, characterized in that, The step of performing consistency verification on each conflict extraction result based on the document image to be identified, and obtaining the target verification result corresponding to each conflict extraction result, includes: The document image to be identified and each of the conflict extraction results are input into multiple heterogeneous verification models in parallel. Visual consistency verification is performed on each of the conflict extraction results to obtain the first verification result corresponding to each of the conflict extraction results. Based on the preset rigid verification rules, the rule consistency verification is performed on each of the conflict extraction results to obtain the second verification result corresponding to each of the conflict extraction results. Based on each of the first verification results and each of the second verification results, the target verification result corresponding to each of the conflict extraction results is determined.

3. The method according to claim 2, characterized in that, The step of inputting the document image to be identified and each of the conflict extraction results into multiple heterogeneous verification models in parallel, performing visual consistency verification on each of the conflict extraction results, and obtaining a first verification result corresponding to each of the conflict extraction results includes: The document image to be identified and each of the conflict extraction results are input into multiple heterogeneous verification models in parallel. The multiple heterogeneous verification models are used to perform visual consistency verification on each of the conflict extraction results to obtain multiple visual consistency results corresponding to each of the conflict extraction results. For each of the conflict extraction results, based on the credibility weight coefficients corresponding to the heterogeneous verification models, a weighted average is performed on the multiple visual consistency results corresponding to the conflict extraction results to obtain the first verification result corresponding to the conflict extraction results.

4. The method according to claim 2, characterized in that, The step of performing rule consistency verification on each conflict extraction result based on preset rigid verification rules to obtain a second verification result corresponding to each conflict extraction result includes: Identify the document feature information of the document image to be identified; Based on the document feature information, at least one target rigid verification rule that matches the document feature information is determined from the preset rigid verification rules; If the conflict extraction result does not conform to any of the target rigid verification rules, then the second verification result corresponding to the conflict extraction result is set to a preset value.

5. The method according to claim 1, characterized in that, The conflict extraction result includes conflict candidate field values ​​and at least one initial confidence level corresponding to the conflict candidate field values; the multi-source credibility analysis based on each conflict extraction result and its corresponding target verification result to obtain the multi-source credibility score corresponding to each conflict extraction result includes: For each initial confidence level corresponding to the conflict candidate field value, the single-source credibility probability of the conflict candidate field value on the associated text extraction model associated with the initial confidence level is calculated based on the initial confidence level and the target verification result. Based on the credibility probabilities of each single source, a joint probability analysis is performed to obtain the multi-source credibility score corresponding to the conflict extraction result.

6. The method according to claim 1, characterized in that, The step of determining the target multi-source credibility score that meets the preset credibility decision conditions from multiple multi-source credibility scores, and determining the target extraction result corresponding to the target field based on the target multi-source credibility score and the target conflict extraction result corresponding to the target multi-source credibility score, includes: The multi-source credibility score with the highest score among the multiple multi-source credibility scores is determined as the candidate multi-source credibility score; If the candidate multi-source credibility score is greater than or equal to a preset credibility threshold, and the conflict extraction result corresponding to the candidate multi-source credibility score satisfies the preset multi-source consensus condition, then the candidate multi-source credibility score is determined as the target multi-source credibility score. The target conflict candidate field value in the target conflict extraction result corresponding to the target multi-source credibility score is determined as the target field value corresponding to the target field; the target multi-source credibility score is determined as the target confidence level corresponding to the target field value.

7. A document field extraction device, characterized in that, The device includes: The acquisition module is used to acquire the multi-source candidate extraction results corresponding to the target fields in the document image to be recognized; The verification module is used to perform consistency verification on each of the conflicting extraction results based on the document image to be identified if at least two conflicting extraction results are detected among the multi-source candidate extraction results, so as to obtain the target verification result corresponding to each of the conflicting extraction results. The multi-source credibility analysis module is used to perform multi-source credibility analysis based on each of the conflict extraction results and their corresponding target verification results, and to obtain the multi-source credibility score corresponding to each of the conflict extraction results. The decision module is used to determine the target multi-source credibility score that meets the preset credibility decision conditions from multiple multi-source credibility scores, and to determine the target extraction result corresponding to the target field based on the target multi-source credibility score and the target conflict extraction result corresponding to the target multi-source credibility score.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images