Verification reasoning error correction method and device based on large model

Abstract

The invention relates to the technical field of intelligent document information processing and business data management, in particular to a verification reasoning error correction method and device based on a large model, and the method comprises the steps: reading a field configuration file containing a field structure, a field mapping relation, an identification model identifier and a cross-field verification rule; analyzing the to-be-processed document file to obtain a document image, and calling the multi-modal large model to output a field key value pair and a field position according to the identification of the identification model; and inputting the field identification result into a rule verification engine to generate an abnormal field list and an error correction prompt, receiving manual correction content to update a field key value pair to obtain a correction field identification result, and generating structured data according to a field mapping relation to be output to a service system. According to the invention, missed extraction and mismatching are reduced, and the manual rechecking range is reduced.

Description

Technical Field

[0001] This invention relates to the field of intelligent document information processing and business data governance technology, and in particular to a verification reasoning error correction method and apparatus based on a large model. Background Technology

[0002] In high-frequency business scenarios such as invoices, contracts, expense reports, and logistics documents, structuring document information is the foundation for automated data entry, risk control audits, and compliance record keeping. Existing solutions largely rely on template rules or traditional optical character recognition and field location. Faced with diverse layouts, fluctuating image quality, similar field semantics, and complex cross-field constraints, these solutions are prone to field omissions, mismatches, and format inconsistencies, leading to data entry failures in business systems, increased manual review costs, and inconsistent data definitions. The industry urgently needs a structured processing mechanism that can be reused across templates, is constrained by rules, and allows for the accumulation of manual error correction experience into continuous improvement capabilities to support large-scale, low-cost, and highly consistent business data flow. Summary of the Invention

[0003] This invention provides a verification, inference, and error correction method and apparatus based on a large model, which is used to at least solve the problem of how to make field extraction results verifiable, error-correctable, and stably mapped and output to business systems in multi-format documents.

[0004] In a first aspect, the present invention provides a verification and inference error correction method based on a large model, comprising the following steps: Read the field configuration file corresponding to the target data. The field configuration file includes the field structure, field mapping relationship, recognition model identifier and cross-field validation rules. The system receives the document file to be processed and parses it to obtain the document image. Based on the recognition model identifier in the field configuration file, it calls the multimodal large model, inputs the document image into the multimodal large model, and outputs the field recognition result based on zero-shot inference. The field recognition result includes field key-value pairs and field positions. The field recognition results are input into the rule validation engine, which generates an abnormal field list and error correction prompts based on cross-field validation rules. It receives manual corrections corresponding to the abnormal field list and updates the field key-value pairs based on the manual corrections to obtain the corrected field recognition results. Based on the field mapping relationship, the corrected field recognition results are converted into structured data and output to the business system.

[0005] In one possible implementation, the field structure includes field names and field data types, and the field mapping relationship includes the correspondence between field names and field identifiers in the business system.

[0006] In one possible implementation, the document file to be processed includes a portable document format file and an image format file. Parsing to obtain the document image includes performing page rendering on the portable document format file to obtain the document image, and performing image decoding on the image format file to obtain the document image.

[0007] In one possible implementation, before inputting the document image into the multimodal large model, image quality correction processing is performed on the document image, including tilt correction, geometric distortion correction, noise reduction, brightness correction, and contrast correction.

[0008] In one possible implementation, inference instruction text is generated based on the field structure in the field configuration file. The inference instruction text is a text instruction used to constrain the set of field names in the field recognition result. The inference instruction text and the document image are input together into the multimodal large model.

[0009] In one possible implementation, the inference instruction text includes output format constraints, which limit the field recognition results to be output in the form of field key-value pairs and limit the field positions to be output in the form of bounding boxes.

[0010] In one possible implementation, the rule validation engine performs format validity checks, value validity checks, and cross-field logical consistency checks on field key-value pairs based on cross-field validation rules, and records fields that fail the validation as an abnormal field list and generates error correction prompts.

[0011] In one possible implementation, the content to be manually corrected is entered through an interactive interface. The interface locates and marks the document image area corresponding to the abnormal field based on the field location, and displays error correction prompts based on the location marks.

[0012] In one possible implementation, manually corrected content, document images, a list of abnormal fields, and the results of corrected field identification are associated and stored as labeled sample data. The labeled sample data is used as training data for supervised training of the multimodal large model, and supervised training of the multimodal large model is performed based on the labeled sample data.

[0013] Secondly, the present invention provides a large-model-based verification and inference error correction device for implementing the large-model-based verification and inference error correction method, the device comprising: The configuration reading module is used to read the field configuration file corresponding to the target data. The field configuration file includes the field structure, field mapping relationship, recognition model identifier and cross-field verification rules. The recognition and inference module is used to receive the document file to be processed and parse it to obtain the document image. According to the recognition model identifier in the field configuration file, it calls the multimodal large model, inputs the document image into the multimodal large model, and outputs the field recognition result based on zero-shot inference. The field recognition result includes field key-value pairs and field position. The verification output module is used to input the field recognition results into the rule verification engine, generate an abnormal field list and error correction prompts based on cross-field verification rules, receive manual correction content corresponding to the abnormal field list, update the field key-value pairs based on the manual correction content to obtain the corrected field recognition results, convert the corrected field recognition results into structured data based on the field mapping relationship, and output them to the business system.

[0014] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows: By constraining the field structure and mapping relationships in the field configuration file, consistency between field output boundaries and business field identifiers is achieved. A model invocation mechanism based on model identifiers enables unified scheduling of field extraction capabilities across different data types. A rule validation engine driven by cross-field validation rules automatically detects consistency in field format, values, and logic. An abnormal field list and error correction prompts guide manual correction, improving the convergence of manual verification scope and error correction efficiency. By correcting the conversion of field recognition results into structured data output, direct integration with business systems and increased data entry success rates are achieved. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the execution flow of the method of the present invention; Figure 2 This is a structural block diagram of the device of the present invention. Detailed Implementation

[0016] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0017] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0018] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0019] Large-scale model validation and inference refers to using the recognition output of a multimodal large-scale model as a constrained and verifiable intermediate result, rather than directly as the final database entry result. Its core lies in two aspects: First, using the multimodal large-scale model to perform zero-shot inference on document images, outputting field key-value pairs and field positions, enabling field extraction to have cross-template and cross-format generalization capabilities; second, introducing business field structures and cross-field constraints as explicit rules to perform consistency verification and anomaly localization on the large-scale model output, and transforming the causes of anomalies into executable error correction prompts, thus forming a closed loop of "recognition—verification—error correction—writeback". Based on this idea, this invention uses field configuration files to uniformly constrain field boundaries and mapping relationships, combined with a rule validation engine and interactive manual correction, to achieve controllable error correction and traceable structured output of the large-scale model.

[0020] like Figure 1 As shown, a verification inference error correction method based on a large model includes the following steps: Read the field configuration file corresponding to the target data. The field configuration file includes the field structure, field mapping relationship, recognition model identifier and cross-field validation rules. In this embodiment, the field configuration files corresponding to the target data are uniformly maintained by the data management service and stored in the configuration library. The field configuration files are retrieved through the data type identifier index and loaded into memory. The field configuration files use a structured format to record the field structure, field mapping relationships, identification model identifiers, and cross-field validation rules. The field structure defines the field name, field data type, required field flags, and display order, and generates a set of field names to constrain the output range of subsequent field identification results. The field mapping relationship defines the one-to-one correspondence between field names and business system field identifiers, and defines field value standardization rules to convert the corrected field identification results into structured data acceptable to the business system. The identification model identifier indicates the multimodal large model version and inference interface address in the model registry, enabling consistent inference capabilities based on data type. Cross-field validation rules are recorded using a set of rule entries. Each rule includes at least the set of participating fields, logical judgment conditions, and error correction prompt text, used by the rule validation engine to generate a list of abnormal fields and error correction prompts, thereby reducing field entry errors and improving output consistency.

[0021] The field structure includes field names and field data types, and the field mapping relationship includes the correspondence between field names and field identifiers in the business system.

[0022] In this embodiment, the field structure in the field configuration file is used to determine the unified data boundary of the target data in structured extraction and verification. The field structure includes at least field names and field data types. Field names use Chinese names consistent with the business meaning and are kept unique within the field configuration file. This is used to generate a set of field names and constrain the output field range of the multimodal large model, thereby avoiding the problem of field omissions or fields with the same name but different meanings when identifying the same data in different templates and batches. Field data types are used to specify the interpretation method and parsing rules for field values. Field data types include at least character, numeric, date, and enumeration types. Character fields are parsed as strings and support the removal of leading and trailing whitespace and uniform processing of full-width and half-width characters. Numeric fields are parsed as decimal numbers and support the extraction of values ​​after removing thousands separators and unit symbols. Date fields are parsed according to a preset date format and output a unified date representation. Enumeration fields are normalized and mapped according to an enumeration dictionary. To ensure feasibility, the rule validation engine performs type consistency checks and format validity checks on field key-value pairs based on the field data type before performing cross-field validation. Fields with type mismatches or parsing failures are written into the exception field list so that they can be displayed in conjunction with error correction prompts.

[0023] The field mapping relationships in the field configuration file are used to establish the correspondence between field names and business system field identifiers. These mapping relationships are stored as a set of mapping entries, each of which includes at least a field name, a business system field identifier, and a field value conversion rule. The business system field identifier indicates the data receiving key or field code for downstream business systems, enabling structured data to directly interface with the business system's inbound or work order interfaces. The field value conversion rules are used to standardize the corrected field identification results during the output stage. These rules include at least character cleaning rules, unit normalization rules, date format unification rules, and enumeration value mapping rules. Specifically, character cleaning rules remove invisible characters and unify delimiters; unit normalization rules convert the identified unit descriptions to the unit representation required by the business system; date format unification rules convert date strings with different writing conventions to the date representation required by the business system; and enumeration value mapping rules map synonyms or near-synonyms output by the multimodal large model to enumeration codes recognized by the business system. By coordinating constraints on field structure and field mapping relationships, it is possible to maintain consistency in field semantics, data type, and field identifiers throughout the chain of "field identification result - manual correction of content - correction of field identification result - structured data output". This reduces the cost of manual review during cross-system integration and provides a stable data foundation for the subsequent location, labeling, and error correction of abnormal fields.

[0024] The system receives the document file to be processed and parses it to obtain the document image. Based on the recognition model identifier in the field configuration file, it calls the multimodal large model, inputs the document image into the multimodal large model, and outputs the field recognition result based on zero-shot inference. The field recognition result includes field key-value pairs and field positions. In this embodiment, after receiving the document file to be processed, the device first determines the file type based on the file header information. The document files to be processed include portable document format files and image format files. For portable document format files, page rendering is performed to generate document images, and each page is assigned a page number for subsequent location. For image format files, image decoding is performed to generate document images. Subsequently, the device reads the recognition model identifier from the field configuration file. The recognition model identifier corresponds to the multimodal large model service address and interface parameters in the model registry. The device establishes an inference request based on the recognition model identifier, sends the document image as image input to the multimodal large model, and carries the set of field names generated by the field structure and the output format constraints in the same request, thereby realizing zero-shot inference field extraction. The multimodal large model returns the field recognition result, which includes field key-value pairs and field positions. The field positions are represented by bounding boxes and stored in association with page numbers for subsequent location annotation when the rule verification engine generates an abnormal field list and for quick input of manually corrected content. Through the unified scheduling of recognition model identifiers and zero-shot inference output, the consistency of field extraction can be maintained under different templates and different document formats, and the cost of template configuration and rule maintenance can be reduced.

[0025] The document files to be processed include portable document format files and image format files. Parsing the document images involves performing page rendering on portable document format files to obtain document images, and performing image decoding on image format files to obtain document images.

[0026] In this embodiment, the document file to be processed is received by the front-end upload interface or the business system push interface. Upon receipt, the file identifier, source information, and reception time are recorded, and the file integrity is verified to avoid parsing failures caused by transmission truncation. The device determines the type of the document file to be processed based on the file header information and extension. The document file to be processed includes portable document format files and image format files. For portable document format files, the device calls the document rendering component to perform page rendering on each page, rasterizing vector text, graphic objects, and embedded bitmaps into a unified document image. During the page rendering process, a page number is generated for each page, and a relationship is established between the page number and the document image so that the subsequent field position can be accurately located in the multi-page document. To improve the stability of field recognition, page size information and resolution information are output synchronously during page rendering, and the page size information and resolution information are stored together with the document image as metadata for consistent presentation of field position write-back and interface positioning annotations. For image format files, the device calls the image decoding component to complete the decoding. The decoded document image retains the original pixel matrix and outputs the image width, image height and color channel information as metadata so that the subsequent multimodal large model input and bounding box localization can use the same coordinate system.

[0027] After parsing the document image, the device performs basic normalization processing, including unifying image orientation, unifying color space, and removing invalid edge blank areas. Unifying image orientation is achieved by detecting the text direction or long side direction; unifying color space is achieved by converting the color image to a standard color representation; and removing invalid edge blank areas is achieved through background color statistics and edge cropping. This basic normalization process reduces input differences between scanned, photographed, and exported documents from different sources, enabling subsequent multimodal large-scale models to perform zero-shot inference under the same input standard, thereby reducing missing field key-value pairs and field position drift. The parsing and normalization results are saved as a document image set, and page number information or single-page identification information is retained in the metadata. This ensures that the field position in the field recognition results accurately points to specific pages and regions, facilitating the rule verification engine's location and annotation when generating an abnormal field list, and enabling manual correction of content for comparison with the original text. By unifying the page rendering of portable document format files and the image decoding of image format files into a single intermediate representation of the document image, this embodiment can employ a consistent subsequent recognition and verification process under different file type inputs, improving the system's versatility and maintainability.

[0028] Before inputting the document image into the multimodal large model, image quality correction processing is performed on the document image, including tilt correction, geometric distortion correction, noise reduction, brightness correction, and contrast correction.

[0029] Before inputting the document image into the multimodal large model, the device performs image quality correction processing to reduce input differences caused by photography, scanning, and transmission compression, ensuring consistency in field key-value pair extraction and field location positioning across documents from different sources. In the tilt correction stage, the device first binarizes and extracts edges from the document image to obtain the direction information of text lines or boundary lines; then, it calculates the rotation angle based on the direction information and performs rotation resampling on the document image to align the main text direction with the horizontal axis of the image, while maintaining consistent updates to the field position coordinate system in the rotated image. In the geometric distortion correction stage, the device detects the page boundary contours in the document image, extracts the four corner points of the page, and establishes a correspondence between the corner points of the original image and the corner points of the target rectangle. Based on this correspondence, perspective correction is performed to normalize the trapezoidal distortion caused by tilted shooting to an approximate rectangular page, thereby reducing the offset of the bounding box in the page edge region. In the denoising stage, the device identifies the noise type of the document image, using median filtering for salt-and-pepper noise and edge-preserving smoothing filtering for compression block effects and texture noise, to suppress noise while preserving the edges of character strokes. In the brightness correction stage, the device estimates the overall brightness shift based on the image's grayscale distribution and performs local brightness compensation on excessively dark or overexposed areas to make the page background more uniform. In the contrast correction stage, the device performs contrast stretching or histogram equalization on the document image to enhance the grayscale difference between the text area and the background area. After completing the above image quality corrections, the corrected document image is used as input to a multimodal large model. This reduces the risk of field omissions and mismatches in zero-shot inference and improves the consistency between field positions and manually corrected positioning annotations, thereby reducing the manual verification cost after the rule validation engine generates a list of abnormal fields.

[0030] Based on the field structure in the field configuration file, inference instruction text is generated. The inference instruction text is a text instruction used to constrain the set of field names in the field recognition result. The inference instruction text and the document image are input together into the multimodal large model.

[0031] In this embodiment, after reading the field configuration file, the device generates inference instruction text based on the field structure in the field configuration file. This text is used to constrain the set of field names and the output organization of the field recognition results when performing zero-shot inference on a multimodal large model. The inference instruction text is constructed by the inference instruction generation unit, which extracts the field names and field data types from the field structure into a field list and assembles them into text instructions according to a preset template. The field list includes a set of field names and a data type description corresponding to each field name. The data type description is used to prompt the multimodal large model to output field values ​​according to the field data type, thereby reducing format drift of date, numerical, and enumeration fields. The inference instruction text also includes an output constraint section, which limits the field recognition results to output only the field names in the field list and gives the field values ​​in the form of field key-value pairs. It also gives the field position for each field, which is represented by a bounding box consistent with the document image coordinate system and associated with the page number, so that the subsequent rule verification engine can locate abnormal fields and complete the location annotation in the interactive interface. To ensure the reusability of inference instruction text across different data types, the inference instruction generation unit maintains the field name set and output constraint segment separately. The field name set is dynamically generated from the field structure, while the output constraint segment is generated from a fixed template, thus maintaining the overall semantic stability of the inference instruction text even when the field structure changes. After generating the inference instruction text, the device inputs the inference instruction text and document image together into the multimodal large model. The input format is that the same inference request includes both image and text input, and the inference request is labeled with the model version corresponding to the recognition model identifier. This allows the multimodal large model to output constrained field recognition results under a unified interface. This approach suppresses issues such as inconsistent field names, missing fields, and field confusion in the multimodal large model's output, and ensures that field key-value pairs and field positions meet the input requirements of subsequent cross-field verification rules, thereby improving the stability and maintainability of the verification and error correction chain.

[0032] The inference instruction text includes output format constraints, which limit the field recognition results to be output in the form of field key-value pairs and limit the field positions to be output in the form of bounding boxes.

[0033] In this embodiment, the inference instruction text includes output format constraints. These constraints limit the output of the multimodal large model to field key-value pairs and the field positions to bounding boxes, ensuring that the field recognition results can be stably parsed and located by the rule validation engine and the interactive interface. The output format constraints are constructed by the inference instruction generation unit based on the set of field names in the field configuration file. These constraints include at least a field output list constraint and a positional output constraint. The field output list constraint limits the output of field recognition results to only field names from the set of field names, requiring each field name to correspond to a unique field value to avoid multiple values ​​for the same field or field name drift. The positional output constraint limits the field positions to be output in bounding box form. The bounding box is represented using the document image pixel coordinate system and stored in association with the page number. For ease of implementation by those skilled in the art, this embodiment defines the bounding box as follows: in, For bounding box, The x-coordinate of the top-left corner of the bounding box. The ordinate of the top left corner of the bounding box. The x-coordinate of the bottom right corner of the bounding box. The coordinates are the bottom right corner ordinates of the bounding box, with the origin at the top left corner of the document image, and the unit is pixels. When an inference request is sent, the device submits the inference instruction text as text input and the document image as image input to the multimodal large model service. After receiving the inference response, the device performs format validation on the field recognition results. The validation includes whether the field name belongs to the field name set, whether the field key-value pairs are complete, whether the bounding box coordinates fall within the document image range, and whether the page number is valid. In response to validation failure, the device adds the corresponding field to the abnormal field list and generates an error correction prompt. In response to validation success, the device uses the bounding box for positioning and annotation in the interactive interface and uses the field key-value pairs as input to the rule validation engine, thereby achieving a consistent process of abnormal field location, manual correction of content entry, and writing back the corrected field recognition results. By uniformly limiting the output format constraints, the output differences of the multimodal large model under different versions can be reduced, improving the parsability and verifiability of the field recognition results.

[0034] The field recognition results are input into the rule validation engine, which generates an abnormal field list and error correction prompts based on cross-field validation rules. It receives manual corrections corresponding to the abnormal field list and updates the field key-value pairs based on the manual corrections to obtain the corrected field recognition results. Based on the field mapping relationship, the corrected field recognition results are converted into structured data and output to the business system.

[0035] In this embodiment, after receiving the field identification results, the rule validation engine performs format validity, value validity, and cross-field consistency checks on each field key-value pair according to the cross-field validation rules, generating an abnormal field list and correction prompts. The abnormal field list includes at least the field name, field location, and reason for the abnormality, while the correction prompts indicate the field values ​​that need to be supplemented, rewritten, or verified. The interactive interface locates and marks the abnormal fields based on their positions and displays the correction prompts, receiving manual corrections corresponding to the abnormal field list. The rule validation engine writes the manual corrections back to the field key-value pairs, forming corrected field identification results, and records the corrected field name and the field values ​​before and after the correction for traceability. Subsequently, based on the field mapping relationship, the corrected field identification results are converted into structured data, which includes business system field identifiers, field values, and validation status, and output to the business system in JSON format via the business interface.

[0036] The rule validation engine performs format validity checks, value validity checks, and cross-field logical consistency checks on field key-value pairs based on cross-field validation rules, and records fields that fail the validation as an abnormal field list and generates error correction prompts.

[0037] In this embodiment, the rule validation engine performs rule-based validation on the field key-value pairs output by the multimodal large model, records fields that fail validation as an abnormal field list, and generates error correction prompts so that subsequent manual corrections can be targeted towards these abnormal fields. The rule validation engine reads cross-field validation rules from the field configuration file. These rules are stored as a set of rule entries, each rule including at least a set of participating field names, a validation type, a judgment condition, and an error correction prompt template. The rule validation engine indexes the field key-value pairs, with the field name as the index key and the field value and field position as the index value, thereby enabling rapid location of participating fields during cross-field logical consistency validation.

[0038] Format validity checks determine whether field values ​​meet the format requirements corresponding to the field's data type. For date fields, the rule validation engine determines whether the field value can be parsed into a date representation and outputs a unified date representation. For numeric fields, the rule validation engine determines whether the field value can be parsed into a decimal number, removing thousands separators and unit symbols during parsing. For enumerated fields, the rule validation engine determines whether the field value belongs to the acceptable value set of the enumeration dictionary. Value validity checks determine whether field values ​​meet business constraints, including non-empty constraints, length constraints, and value range constraints. Non-empty constraints determine whether required fields are empty, length constraints determine whether character fields exceed the preset length range, and value range constraints determine whether numeric fields fall within the preset range. Cross-field logical consistency checks determine whether multiple fields meet preset logical relationships, including time relationships, quantity consistency relationships, and identity consistency relationships. Time relationships determine whether the start date is not later than the end date, quantity consistency relationships determine the logical consistency between summary and detail fields, and identity consistency relationships determine the consistency of the identifier for the same entity in different fields.

[0039] When generating the list of abnormal fields and error correction prompts, the rule validation engine records fields that fail validation as abnormal field entries. Each abnormal field entry includes at least the field name, field location, abnormal type, and abnormal reason. It also generates error correction prompt text based on the error correction prompt template. The error correction prompt text is bound to the field location, allowing the interactive interface to locate and annotate the document image region corresponding to the abnormal field. By combining format validity validation, value validity validation, and cross-field logical consistency validation, it can promptly detect missing fields, inconsistent field formats, and cross-field contradictions after zero-shot inference output from a multimodal large model. The validation results are output as an abnormal field list and error correction prompts, thereby reducing the scope of manual review and improving the reliability of the corrected field recognition results.

[0040] Manual corrections are entered through an interactive interface. The interface locates and marks the document image area corresponding to the abnormal field based on the field location, and displays error correction prompts based on the location marks.

[0041] In this embodiment, manually corrected content is entered through an interactive interface. This interface links the list of abnormal fields output by the rule validation engine, error correction prompts, and the document image, allowing for targeted corrections based on the abnormal fields. After receiving the list of abnormal fields, the interface sorts and displays them by name, showing the field name, current value, cause of the error, and error correction prompt for each field. The interface positions and annotates the corresponding document image area based on the field's location. This annotation includes drawing a bounding box outline on the document image and displaying the field name label near the bounding box. When an abnormal field spans multiple pages, the interface automatically switches to the corresponding page based on the page number and scrolls to the bounding box area, reducing manual search time. To ensure consistent coordinates for the positioning annotations, the interface uses the same document image pixel coordinate system as the field location and performs coordinate mapping synchronously when the document image is scaled, ensuring that the bounding box accurately covers the abnormal field text area at different scaling ratios.

[0042] The interactive interface uses location markers to display error correction prompts. This includes a pop-up error correction prompt panel when the boundary of an abnormal field is selected, displaying the reason for the anomaly, suggested correction methods, and the target field value that can be entered. Manual corrections are entered at the field level. The interface provides input controls for each abnormal field, selecting the control type based on the field's data type: date fields provide date input controls, enumeration fields provide dropdown selection controls, and numeric fields provide numeric input controls with a prompt for unit normalization rules. After manual corrections are submitted, the interface binds the corrections to the field name and sends the corrections, field name, and field location to the rule validation engine. The rule validation engine updates the field key-value pairs and generates the corrected field identification result. By linking abnormal field location markers with error correction prompts, the scope of manual corrections is narrowed, the probability of erroneous changes is reduced, and the consistency between manually corrected content and subsequent structured data output in terms of field names and locations is ensured.

[0043] Manually corrected content, document images, lists of abnormal fields, and the results of corrected field identification are associated and stored as labeled sample data. The labeled sample data is used as training data for supervised training of the multimodal large model. Supervised training of the multimodal large model is performed based on the labeled sample data.

[0044] In this embodiment, after generating the abnormal field list and inputting the manually corrected content, the device associates and stores the manually corrected content, document image, abnormal field list, and corrected field recognition results to form labeled sample data for subsequent supervised training of the multimodal large model. The associated storage uses the file identifier as the primary key and the page number and field name as association keys, ensuring a one-to-one correspondence between abnormal field entries, error correction prompts, field positions, and manually corrected content within the same document image. The labeled sample data includes at least: document image and page number, inference instruction text, abnormal field list, error correction prompts, manually corrected content, and corrected field recognition results. The abnormal field list records the field name, field position, and reason for the abnormality; the manually corrected content records the field name and corrected field value; and the corrected field recognition results record the corrected field key-value pairs and field position, thus providing both field extraction supervision information and location supervision information in the labeled sample data.

[0045] When using labeled sample data for supervised training, the device uses document images and inference instruction text as training input, and the corrected field recognition results as training labels. The training labels provide field values ​​in key-value pair format, retaining the bounding boxes corresponding to the field positions. This allows the multimodal large model to learn to output a consistent set of field names and field positions under the same field structure constraints during training. To reduce the impact of noisy samples on training, the device only selects labeled sample data that meets consistency conditions into the training set. These consistency conditions include at least: the manually corrected content and the corrected field recognition results are consistent at the field name level; the field position falls within the document image range; and the field data type passes validation. The device performs deduplication and version management on the labeled sample data. When the same file identifier is corrected multiple times, the latest corrected version is retained, and the correction time and the corrector's identifier are recorded for traceability.

[0046] Supervised training is performed offline, and the training process includes data partitioning, model training, and model validation: Labeled sample data is divided into training and validation sets; supervised training of the multimodal model is performed on the training set, enabling the multimodal model to output field key-value pairs and field positions given document images and inference instruction text; on the validation set, metrics such as field name matching, field value consistency, and field position overlap are calculated to determine whether the trained multimodal model meets the deployment conditions. After training, the device registers the trained multimodal model as a new model version and updates the model version pointed to by the recognition model identifier, ensuring that the updated multimodal model is prioritized for zero-shot inference of similar target data. By accumulating manually corrected content into labeled sample data and conducting closed-loop training, the multimodal model can gradually adapt to specific document formats and business field rules, reducing the size of the abnormal field list and the workload of manual correction.

[0047] like Figure 2 As shown, a large-model-based verification and inference error correction device is used to implement the large-model-based verification and inference error correction method. The device includes: The configuration reading module is used to read the field configuration file corresponding to the target data. The field configuration file includes the field structure, field mapping relationship, recognition model identifier, and cross-field verification rules. The configuration reading module includes at least a processor, non-volatile memory, main memory, and a communication interface. The non-volatile memory is used to store the field configuration file or a cached copy thereof. The field configuration file stores the field structure, field mapping relationship, recognition model identifier, and cross-field verification rules in a structured format. When the field configuration file is distributed by an external configuration center, the communication interface is used to establish a network connection with the configuration center and receive the configuration content. The processor builds a configuration index table in the main memory, compiles the field name, field data type, business system field identifier, and cross-field verification rules into a fast-accessible data structure, and parses the recognition model identifier into model call parameters and service address information for reuse by the subsequent recognition inference module and verification output module. This module can set the configuration version number and check code to be stored in the non-volatile memory for configuration rollback and consistency verification under restart or network outage conditions.

[0048] The recognition and inference module receives the document file to be processed and parses it to obtain the document image. Based on the recognition model identifier in the field configuration file, it calls the multimodal large model, inputs the document image into the multimodal large model, and outputs the field recognition result based on zero-shot inference. The field recognition result includes field key-value pairs and field positions. The recognition and inference module includes at least a processor, main memory, a file input interface, an image processing acceleration unit, and a network interface. The file input interface receives the document file to be processed and writes it to the buffer. The processor calls the decoding component to perform page rendering for portable document format files and image decoding for image format files, generating a document image and storing it in main memory. The image processing acceleration unit performs image quality correction and format conversion on the document image, including tilt correction, geometric distortion correction, noise reduction, brightness correction, and contrast correction, thereby reducing input differences. The network interface is used to establish a connection with the multimodal large model service based on the recognition model identifier and send inference requests. The inference request carries the document image and inference instruction text. The processor receives the inference response and parses it to obtain the field recognition result. The field recognition result is written to the main memory in the form of field key-value pairs and field positions. The field position is represented by a bounding box in the document image coordinate system and is associated with the page number and file identifier for subsequent abnormal field location and annotation.

[0049] The verification output module is used to input the field recognition results into the rule verification engine, generate an abnormal field list and error correction prompts based on cross-field verification rules, receive manual corrections corresponding to the abnormal field list, update the field key-value pairs based on the manual corrections to obtain the corrected field recognition results, and convert the corrected field recognition results into structured data based on the field mapping relationship and output them to the business system. The verification output module includes at least a processor, main memory, rule execution unit, interaction interface, and business system interface. The rule execution unit is loaded into the main memory by the processor and executed. For field key-value pairs, it sequentially performs format validity verification, value validity verification, and cross-field logical consistency verification, generates an abnormal field list and error correction prompts, and writes them to the main memory. The interaction interface is used to communicate with the manual correction terminal, send the abnormal field list, field position, and error correction prompts to the interaction interface, and receive manual corrections. The processor updates the field key-value pairs based on the manual corrections to obtain the corrected field recognition results, and records correction logs for traceability. The business system interface is used to convert the corrected field recognition results into structured data based on the field mapping relationship, and output the structured data to the business system in the data format agreed upon by the business system. To ensure availability, this module can store the output queue and failure retry records in non-volatile memory for reliable delivery when there is network jitter or the business system is unreachable.

[0050] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects.

[0051] The above are merely embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A verification and inference error correction method based on a large model, characterized in that, Includes the following steps: Read the field configuration file corresponding to the target data. The field configuration file includes the field structure, field mapping relationship, recognition model identifier and cross-field validation rules. The system receives the document file to be processed and parses it to obtain the document image. Based on the recognition model identifier in the field configuration file, it calls the multimodal large model, inputs the document image into the multimodal large model, and outputs the field recognition result based on zero-shot inference. The field recognition result includes field key-value pairs and field positions. The field recognition results are input into the rule validation engine, which generates an abnormal field list and error correction prompts based on cross-field validation rules. The engine receives manual corrections corresponding to the abnormal field list and updates the field key-value pairs based on the manual corrections to obtain the corrected field recognition results. Based on the field mapping relationship, the corrected field recognition results are converted into structured data and output to the business system.

2. The method according to claim 1, characterized in that, The field structure includes field names and field data types, and the field mapping relationship includes the correspondence between field names and field identifiers in the business system.

3. The method according to claim 1, characterized in that, The document files to be processed include portable document format files and image format files. The parsed document images are obtained by performing page rendering on portable document format files and image decoding on image format files.

4. The method according to claim 3, characterized in that, Before inputting the document image into the multimodal large model, image quality correction processing is performed on the document image, including tilt correction, geometric distortion correction, noise reduction, brightness correction, and contrast correction.

5. The method according to claim 1, characterized in that, Based on the field structure in the field configuration file, inference instruction text is generated. The inference instruction text is a text instruction used to constrain the set of field names in the field recognition result. The inference instruction text and the document image are input together into the multimodal large model.

6. The method according to claim 5, characterized in that, The inference instruction text includes output format constraints, which limit the field recognition results to be output in the form of field key-value pairs and limit the field positions to be output in the form of bounding boxes.

7. The method according to claim 1, characterized in that, The rule validation engine performs format validity checks, value validity checks, and cross-field logical consistency checks on field key-value pairs based on cross-field validation rules, and records fields that fail the validation as an abnormal field list and generates error correction prompts.

8. The method according to claim 7, characterized in that, Manual corrections are entered through an interactive interface. The interface locates and marks the document image area corresponding to the abnormal field based on the field location, and displays error correction prompts based on the location marks.

9. The method according to claim 8, characterized in that, Manually corrected content, document images, lists of abnormal fields, and the results of corrected field identification are associated and stored as labeled sample data. The labeled sample data is used as training data for supervised training of the multimodal large model. Supervised training of the multimodal large model is performed based on the labeled sample data.

10. A large-model-based verification and inference error correction device, used to implement the large-model-based verification and inference error correction method according to any one of claims 1-9, characterized in that, The device includes: The configuration reading module is used to read the field configuration file corresponding to the target data. The field configuration file includes the field structure, field mapping relationship, recognition model identifier and cross-field verification rules. The recognition and inference module is used to receive the document file to be processed and parse it to obtain the document image. According to the recognition model identifier in the field configuration file, it calls the multimodal large model, inputs the document image into the multimodal large model, and outputs the field recognition result based on zero-shot inference. The field recognition result includes field key-value pairs and field position. The verification output module is used to input the field recognition results into the rule verification engine, generate an abnormal field list and error correction prompts based on cross-field verification rules, receive manual correction content corresponding to the abnormal field list, update the field key-value pairs based on the manual correction content to obtain the corrected field recognition results, convert the corrected field recognition results into structured data based on the field mapping relationship, and output them to the business system.

Images