Table correction method and device based on spatial position relation and multi-modal verification

Abstract

The embodiment of the invention provides a table correction method and device based on a spatial position relation and multi-modal verification, and the method fundamentally changes a traditional normal form which depends on a fixed rule or plain text matching by constructing an intelligent correction process which takes the spatial position relation as a dominant and fuses the multi-modal verification. According to the method, by automatically extracting the reference object and analyzing the spatial topological relation between the fields, universal understanding and self-adaptive matching of various table structures are achieved, and the accuracy and robustness of field dislocation correction are remarkably improved. And an introduced multi-dimensional confidence evaluation system and a hierarchical decision-making mechanism ensure that each correction operation has a reliable basis, so that the safety and controllability of automatic processing are realized, and the risk of'more correction and more mistakes' is effectively avoided. Finally, according to the method, data integrity repair is integrated while error correction is completed, high-quality structured data is output, and the end-to-end automation level and result reliability of a processing link after table recognition are greatly improved.

Description

A table correction method and apparatus based on spatial location relationship and multimodal verification Technical Field

[0001] The embodiments in this specification relate to the field of data processing technology, and in particular to a table correction method based on spatial location relationships and multimodal verification. Background Technology

[0002] Current table recognition systems based on optical character recognition or visual language models commonly suffer from misalignment between field names and corresponding field values ​​when outputting structured data, severely impacting the direct usability of the data. Existing solutions primarily rely on predefined rules or templates for specific table formats, but these methods suffer from poor versatility, high deployment and maintenance costs, and cannot adapt to diverse table layouts. Other methods attempt to match by calculating text content similarity, but completely ignore the inherent two-dimensional spatial layout characteristics of tables, resulting in low accuracy and a high susceptibility to errors when content is similar. Furthermore, existing technologies lack an effective mechanism for evaluating the reliability of the correction results; the correction process is often arbitrary, risking the introduction of new errors and hindering the achievement of safe and reliable automated post-processing.

[0003] Therefore, a better solution is urgently needed. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a table correction method based on spatial location relationships and multimodal verification. One or more embodiments of this specification also relate to a table correction apparatus based on spatial location relationships and multimodal verification, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a table correction method based on spatial positional relationships and multimodal verification is provided, comprising: acquiring preliminary recognition results from a table recognition system, the preliminary recognition results including the text content of multiple fields identified from a table image and the position coordinates of the multiple fields in the table image; extracting at least one reference field from the preliminary recognition results based on preset rules; analyzing the spatial positional relationships between all fields in the table image based on the position coordinates of the reference field to determine the logical column structure of the table and the row correspondence between fields; performing multi-strategy matching between field names and candidate field values ​​based on the logical column structure and row correspondence to obtain correction suggestions; calculating a comprehensive confidence score for each correction suggestion; and implementing or rejecting the corresponding correction suggestion using different strategies according to the level of the comprehensive confidence score, and performing integrity repair processing on the corrected data.

[0006] In one possible implementation, the reference field includes at least one of the following types: a unique identifier field conforming to a preset encoding rule, a feature data field conforming to a preset format feature, and a composite identifier field composed of letters and numbers.

[0007] In one possible implementation, analyzing the spatial relationships between all fields in a table image includes: based on the coordinates of a reference field, using an adaptive tolerance clustering method to group fields with similar horizontal positions into the same logical column, and verifying the alignment of field name labels with candidate field values ​​in the vertical direction; simultaneously, applying semantic constraints related to the table type to verify the rationality of the field values.

[0008] In one possible implementation, multi-strategy matching includes at least one of an exact matching strategy, an inclusion relation matching strategy, and a format pattern matching strategy; the exact matching strategy is used when the content of the candidate field value is completely consistent with the expected value inferred based on the spatial location relationship; the inclusion relation matching strategy is used when the content of the candidate field value is part of the correct content; and the format pattern matching strategy matches the data format of the field value with the semantic expectation of the field name.

[0009] In one possible implementation, the overall confidence score is obtained by weighted synthesis of spatial distance sub-score, format matching sub-score, and semantic constraint sub-score, calculated as the product of the base confidence score and a dynamic adjustment factor, where the base confidence score is:

[0010] in, For spatial distance sub-scores, For format matching sub-scores, Score the semantic constraint sub-score. , , These are the weighting coefficients.

[0011] In one possible implementation, different strategies are employed based on the overall confidence score, including: when the overall confidence score is higher than a first threshold, the correction suggestion is executed directly; when the overall confidence score is between the first and second thresholds, cross-validation based on an additional reference field is initiated, and the correction suggestion is executed only after the validation passes; when the overall confidence score is lower than the second threshold, the automatic execution of the correction suggestion is rejected and the score is marked as pending review.

[0012] In one possible implementation, the integrity repair process for the corrected data includes at least one of the following operations: completing missing field values ​​based on spatial location relationships, removing duplicate records based on content similarity and spatial proximity, and performing format standardization on all field values.

[0013] According to a second aspect of the embodiments of this specification, a table correction device based on spatial positional relationship and multimodal verification is provided, comprising: a position determination module configured to acquire preliminary recognition results from a table recognition system, the preliminary recognition results including text content of multiple fields identified from a table image and position coordinates of the multiple fields in the table image; a reference determination module configured to extract at least one reference field from the preliminary recognition results based on preset rules; a correspondence module configured to analyze the spatial positional relationship between all fields in the table image based on the position coordinates of the reference field to determine the logical column structure of the table and the row correspondence between fields; a correction suggestion module configured to perform multi-strategy matching between field names and candidate field values ​​based on the logical column structure and row correspondence to obtain correction suggestions; and a repair processing module configured to calculate a comprehensive confidence score for each correction suggestion; and to execute or reject the corresponding correction suggestion using different strategies according to the level of the comprehensive confidence score, and to perform integrity repair processing on the corrected data.

[0014] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, wherein when the computer-executable instructions are executed by the processor, the steps of the table correction method based on spatial positional relationship and multimodal verification described above are implemented.

[0015] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the table correction method based on spatial positional relationships and multimodal verification described above.

[0016] According to a fifth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is instructed to perform the steps of the table correction method based on spatial positional relationship and multimodal verification described above.

[0017] This specification provides a table correction method and apparatus based on spatial location relationships and multimodal verification. The method fundamentally changes the traditional paradigm that relies on fixed rules or pure text matching by constructing an intelligent correction process that is primarily driven by spatial location relationships and integrates multimodal verification. By automatically extracting reference points and analyzing the spatial topological relationships between fields, this method achieves a generalized understanding and adaptive matching of various table structures, significantly improving the accuracy and robustness of field misalignment correction. The introduced multi-dimensional confidence assessment system and hierarchical decision-making mechanism ensure that each correction operation has a reliable basis, achieving safe and controllable automated processing and effectively eliminating the risk of "more correction, more errors." Finally, this method integrates data integrity restoration while completing error correction, outputting high-quality structured data, greatly improving the end-to-end automation level and result reliability of the table recognition post-processing stage. Attached Figure Description

[0018] Figure 1 is a flowchart of a table correction method based on spatial position relationship and multimodal verification according to an embodiment of this specification; Figure 2 is a structural schematic diagram of a table correction device based on spatial position relationship and multimodal verification according to an embodiment of this specification; Figure 3 is a structural block diagram of a computing device according to an embodiment of this specification. Detailed Implementation

[0019] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0020] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0021] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0022] This specification provides a table correction method based on spatial location relationships and multimodal verification. This specification also relates to a table correction device based on spatial location relationships and multimodal verification, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0023] Referring to Figure 1, Figure 1 shows a flowchart of a table correction method based on spatial positional relationship and multimodal verification according to an embodiment of this specification, which specifically includes the following steps.

[0024] Step 101: Obtain preliminary recognition results from the table recognition system. The preliminary recognition results include the text content of multiple fields identified from the table image and the position coordinates of multiple fields in the table image. Step 102: Extract at least one reference field from the preliminary recognition results based on preset rules. The reference field must be unique or have strong features in its content. Step 103: Based on the position coordinates of the reference field, analyze the spatial relationship between all fields in the table image to determine the logical column structure of the table and the row correspondence between fields. Step 104: Based on the logical column structure and row correspondence, perform multi-strategy matching between field names and candidate field values ​​to obtain correction suggestions. Step 105: Calculate a comprehensive confidence score for each correction suggestion. Based on the comprehensive confidence score, implement or reject the corresponding correction suggestion using different strategies, and perform integrity restoration processing on the corrected data.

[0025] In this context, "table recognition system" refers to any system capable of recognizing text content from an image and outputting its coordinates, such as optical character recognition systems, visual language models, or hybrid systems. "Preliminary recognition results" refers to the structured or semi-structured data output by the system, typically containing the text content of each recognized text block and its bounding box coordinates in the original image. "Reference field" refers to a field selected from all recognized fields that possesses clear formatting characteristics or unique content; its position within the table is relatively stable and can serve as a benchmark for spatial analysis. "Logical column structure" refers to the abstract structure reflecting the actual column division of the table, derived through cluster analysis of the field's horizontal coordinates, distinct from the visual column lines. "Row correspondence" refers to the vertical association between field name labels and their corresponding data values ​​within the same logical column. "Correction suggestion" refers to a system-proposed scheme for re-establishing the association between a field name and a field value. "Comprehensive confidence score" is a quantitative value used to evaluate the reliability of the correction suggestion. "Integrity restoration processing" refers to operations such as data completion, deduplication, and standardization to improve overall data quality.

[0026] The following detailed embodiment further illustrates this method. Taking the image processing of a financial statement as an example, the table recognition system (such as a hybrid system of OCR and VLM) first processes the image and outputs preliminary recognition results. This result is a list, where each entry contains the recognized text (such as "Assets", "1001", "1,234.56") and the coordinates of the upper left and lower right corners of the rectangular area where the text is located.

[0027] Next, the aforementioned steps are performed. The system automatically filters reference fields from all identified "field values" based on its built-in rule base. For example, it identifies the text "1001" as conforming to the format characteristic of "4-digit pure numeric encoding," and the text "1,234.56" as conforming to the format of "numerical value with thousands place and decimal point." Since these values ​​are unique or easily distinguishable within the table, they are selected as reference fields.

[0028] Subsequently, using the coordinates of the aforementioned reference field as the center, the spatial relationship of the entire table is analyzed. The system calculates the horizontal center point of the bounding boxes of all fields and, with the x-coordinate of the reference object as a reference, sets a dynamic tolerance range to group fields with similar horizontal positions together, thereby determining that the table has several logical columns such as "Subject Code," "Subject Name," and "Amount." At the same time, it checks whether the labels (such as "Ending Balance") in each logical column are roughly located above their possible data values, establishing a preliminary row correspondence.

[0029] Based on the established logical column and row correspondence, the system begins matching. For example, it finds a field with the value "1,234.56", which, based on its coordinates, is classified into the "Amount" logical column. However, the initial identification incorrectly associates it with the "Account Name" label. The system then initiates multi-strategy matching, first attempting an exact match to search for other more matching labels within the "Amount" column. Then, considering the format pattern (which is in numeric format), it confirms that the data should belong to the "Amount" category, thus generating a correction suggestion to "re-associate the value '1,234.56' with the 'Ending Balance' label."

[0030] The system calculates a comprehensive confidence score for the proposed correction. It assesses the spatial distance of the value to the center of its correct column (closer distance results in a higher score), the degree to which its numerical format matches the expected value in the "Amount" column (a perfect format match results in a high score), and verifies that the value conforms to the business semantics that "the amount should be positive" (if it does, the score is high). These sub-scores are weighted and summed, and the consistency of multiple pieces of evidence is considered to arrive at a final high confidence score.

[0031] Based on this high confidence score, the system directly adopts and executes the correction suggestion, updating the field correspondence. Finally, the system performs integrity repair, such as checking and completing missing currency symbols due to incomplete recognition, standardizing all inconsistent date formats to "YYYY-MM-DD" format, and deleting identical record rows caused by duplicate recognition.

[0032] Through this series of coherent steps from beginning to end, the method achieves automatic and accurate correction of field misalignment and improves overall data quality.

[0033] The beneficial effect of the method defined in this claim lies in providing a systematic solution for correcting misaligned table fields by constructing a complete logical chain from data input, reference extraction, spatial analysis, intelligent matching, confidence assessment to decision-making and repair. This method does not rely on fixed rule templates but rather performs reasoning based on the spatial features of the image itself, thus possessing good versatility. Simultaneously, the introduction of confidence assessment and hierarchical decision-making mechanisms ensures the safety and controllability of the correction process, effectively avoiding the risks associated with blind correction.

[0034] In one possible implementation, the reference field includes at least one of the following types: a unique identifier field conforming to a preset encoding rule, a feature data field conforming to a preset format feature, and a composite identifier field composed of letters and numbers.

[0035] Among these, unique identifier fields can refer to codes or numbers that have a unique identification function in a business context, such as order numbers, ID card numbers, and student ID numbers. Preset encoding rules can refer to patterns used to define the format of these fields, such as regular expressions. Feature data fields can refer to data with distinct, programmable detection format characteristics, such as amounts with specific symbols (¥, $), strings conforming to date and time patterns, and physical quantities with units (kg, m). Composite identifier fields can refer to identifiers with relatively fixed structures, composed of letters and numbers combined according to certain rules, commonly found in logistics, warehousing, and other fields, such as container numbers, cargo location codes, and SKU codes.

[0036] Following the aforementioned steps, when extracting reference fields from the initial identification results, the system identifies different types in the following ways: For unique identifier fields, the system calls a pre-stored regular expression library for matching. For example, using the pattern `"ID[::]?\\s` `(\\w{8,20})` is used to match identifiers that may follow the "ID" tag; or the pattern `"\\d{4}-\\d{2}-\\d{4}"` is used to match serial number formats like "2023-01-0001". Once a match is found, it is also verified whether the match is unique in the table to enhance its reliability as a reference.

[0037] For feature data fields, the system has built-in format detectors. The amount detector looks for text containing currency symbols or conforming to the writing conventions for numerical amounts (such as commas separating thousands places and periods as decimal points). The date detector attempts to match text with various common formats such as "YYYY / MM / DD" and "DD-MMM-YY". Phone numbers, email addresses, and other data are also identified using corresponding patterns.

[0038] For composite identifier fields, the system applies industry-specific rules. For example, in logistics documents, it identifies container codes that conform to the structure of "3 letters + 7 numbers + 1 check letter"; in warehouse management tables, it identifies codes like "A-01-02" that represent aisle-shelf-level.

[0039] The system scores all identified candidate reference fields based on criteria including the inherent stability weight of their type (e.g., unique identifiers typically have the highest weight), the uniformity of their distribution in the table, and their semantic importance. Ultimately, the fields with the highest overall scores are selected as the core references for spatial positioning. This multi-type extraction strategy ensures that sufficiently stable and reliable positioning benchmarks can be found in tables of different industries and formats.

[0040] The beneficial effect of the reference type classification and extraction strategy defined in this claim is that by covering multiple common and stable field types such as business identifiers, feature data, and composite codes, the robustness and applicability of the reference identification module are greatly enhanced. This allows the method of the present invention to adapt to forms in various fields such as finance, logistics, education, and healthcare, without the need for separate configuration for each form, significantly improving the versatility and automation level of the method.

[0041] In one possible implementation, analyzing the spatial relationships between all fields in a table image includes: based on the coordinates of a reference field, using an adaptive tolerance clustering method to group fields with similar horizontal positions into the same logical column, and verifying the alignment of field name labels with candidate field values ​​in the vertical direction; simultaneously, applying semantic constraints related to the table type to verify the rationality of the field values.

[0042] Among these, adaptive tolerance clustering methods refer to clustering algorithms where the tolerance parameter is not a fixed value but dynamically adjusted based on the actual situation of the table (such as the distribution density of reference objects, image width). Vertical alignment refers to the fact that the bottom of the field name label and the top of its corresponding field value are close to each other on the Y-axis, and their horizontal center lines are basically aligned. Semantic constraints related to table type refer to data validity rules set according to the business domain to which the table belongs; for example, in financial statements, asset amounts should not be negative; in personnel information tables, age should be an integer within a reasonable range.

[0043] Following the steps described above, after obtaining the reference object field, the system performs a detailed analysis of spatial location relationships.

[0044] First, column alignment detection is performed. Instead of using a fixed pixel tolerance, the system employs an adaptive tolerance algorithm. The algorithm's base tolerance is a certain percentage (e.g., 5%) of the image width, and then it is fine-tuned based on the horizontal distribution density of the reference fields in the current table: if the references are sparsely distributed horizontally, the tolerance is increased to accommodate possible deviations; if they are densely distributed, the tolerance is decreased to improve segmentation accuracy. Using the X-coordinate of each reference as the center, the system calculates the distance from the horizontal center point of all fields to these centers, initially classifying fields whose distance is less than the dynamic tolerance into the column group containing that reference. Through iteration and weighted averaging, the precise centerline position of each logical column is finally determined.

[0045] Next, row-level relationship verification is performed. For fields initially grouped into the same logical column, the system analyzes their vertical coordinates. It assumes that the field name label is located above its corresponding data value. The system calculates the vertical distance and horizontal offset between each field that could be a label and other fields that could be values ​​within the column. By setting a reasonable vertical distance threshold (such as 10% of the image height), label-value pairs with small vertical distances and small horizontal offsets are filtered out, thereby confirming or establishing row-level correspondences.

[0046] Throughout the spatial analysis process, the system synchronously applies semantic constraints for plausibility verification. For example, when analyzing an invoice, the system invokes semantic constraint rules for the "invoice" type to check candidate values ​​matching the "tax rate" field, determining whether they represent common tax rate percentages (such as 6%, 9%, or 13%). For candidate values ​​matching the "total amount," it attempts to verify whether they are approximately equal to the sum of other detailed amounts. Any potential match that severely violates semantic constraints, even if the spatial relationship is acceptable, will be flagged as suspicious or rejected early on, thus guiding subsequent steps to make more accurate judgments.

[0047] The beneficial effect of the spatial relationship analysis method defined in this claim is that, by combining adaptive column clustering, refined row validation, and pre-emptive semantic filtering, it achieves a deep and accurate understanding of the two-dimensional structure of a table. This not only provides a solid and reliable spatial basis for field matching, but also effectively reduces the complexity and error rate of subsequent matching steps by introducing business logic early on, thereby improving the efficiency and accuracy of the entire correction process.

[0048] In one possible implementation, multi-strategy matching includes at least one of an exact matching strategy, an inclusion relation matching strategy, and a format pattern matching strategy; the exact matching strategy is used when the content of the candidate field value is completely consistent with the expected value inferred based on the spatial location relationship; the inclusion relation matching strategy is used when the content of the candidate field value is part of the correct content; and the format pattern matching strategy matches the data format of the field value with the semantic expectation of the field name.

[0049] Among these strategies, precise matching can refer to a system predicting a specific value should appear at a certain position based on a reference and column structure; if a candidate field value is exactly the same as the predicted text, a match is directly established. Containment matching can handle the situation where a fragment of text is recognized versus the complete text; for example, OCR might only recognize a portion of a long encoded string. Format pattern matching can refer to associating data based on its structural characteristics (such as date or number formats) and the data type implied by the field names, without relying on the specific text content.

[0050] Following the steps described above, after completing the spatial relationship analysis and determining the logical column structure, the system applies multi-strategy matching to field names and values ​​that are suspected of being misaligned.

[0051] When the system can infer the theoretical value of another field within the same row based on the value of a reference and its row, it will employ an exact match strategy. For example, in a consecutively numbered table, the reference row number is "001," and its "Name" is "Product A." In the adjacent row, the row number is identified as "002," but the "Name" field is misaligned. Based on the reference "002," the system searches within the space of the "Name" column for a field whose text content is "Product B" (expected value). If found, an exact match is established directly.

[0052] Inclusion matching strategies are often used to compensate for differences between different recognition systems. For example, VLM might recognize a field value as "123456", while OCR might recognize "Order-123456-END" in the same spatial region. The system confirms the inclusion relationship by calculating the text similarity between "123456" and "Order-123456-END" (e.g., determining whether the former is a substring of the latter). Subsequently, the system can choose to use the complete text from the OCR to correct the incomplete result from VLM and establish a correct match.

[0053] Format pattern matching plays a crucial role when content matching fails. For example, in a logical column where the field name is identified as "Date of Birth," there are two candidate values: "Zhang San" and "1990-01-01." Although "Zhang San" may be spatially closer, the system initiates format pattern matching and detects that "1990-01-01" perfectly matches the date format, while "Zhang San" does not. Combining the semantic expectation of "Date of Birth," the system will prioritize matching the candidate values ​​in the date format with the "Date of Birth" label and assign them a higher confidence level.

[0054] These strategies are not isolated; the system dynamically selects and combines them based on the context to find the most reasonable matching scheme.

[0055] The beneficial effect of the multi-strategy matching scheme defined in this claim is that by integrating content-based precise matching, fragment-based inclusion matching, and structure-based format matching, a multi-level, complementary matching mechanism is formed. This greatly enhances the system's ability to handle various complex misalignment scenarios. Whether it is an identification error, incomplete information, or format confusion, there are corresponding strategies to deal with it, thereby significantly improving the matching success rate and the robustness of correction.

[0056] In one possible implementation, the overall confidence score is obtained by weighted synthesis of spatial distance sub-score, format matching sub-score, and semantic constraint sub-score, calculated as the product of the base confidence score and a dynamic adjustment factor, where the base confidence score is:

[0057] in, For spatial distance sub-scores, For format matching sub-scores, Score the semantic constraint sub-score. , , These are the weighting coefficients.

[0058] Specifically, the overall confidence score is obtained by weighted synthesis of spatial distance sub-score, format matching sub-score, and semantic constraint sub-score. The calculation method is the product of the basic confidence score and the dynamic adjustment factor. The basic confidence score is the weighted sum of the spatial distance sub-score, format matching sub-score, and semantic constraint sub-score. The spatial distance sub-score, format matching sub-score, and semantic constraint sub-score are the aforementioned sub-scores. The weight coefficients are configurable parameters. The dynamic adjustment factor is greater than 1 when the evidence is consistent across multiple dimensions and less than 1 when the evidence is contradictory.

[0059] The spatial distance sub-score is a value inversely proportional to the horizontal distance from the candidate field value to its correct logical column center or relevant reference point; the closer the distance, the higher the score. The format matching sub-score reflects the degree to which the data format of the candidate field value conforms to the expected format of the current column label. The semantic constraint sub-score reflects the degree to which the candidate field value satisfies relevant business rules. The weighting coefficient is a parameter pre-set according to the table type to adjust the importance of the three sub-scores in the final base confidence score. The dynamic adjustment factor is a multiplicative coefficient used for the final calibration of the base confidence score to reflect the consistency between multi-dimensional evidence.

[0060] Following the previous steps, when calculating the overall confidence score for each correction suggestion, the system performs the following operations: First, it calculates the sub-scores for each of the three dimensions. The spatial distance sub-score is calculated based on the difference between the horizontal center coordinate of the candidate field value and the center line coordinate of its logical column. The smaller the difference, the closer the score is to full marks. The format matching sub-score is evaluated based on the results of the matching strategy: exact matching usually receives full marks, inclusion matching is scored based on the coverage of the fragment and the complete text, and format pattern matching is scored based on the precision of the format conformance (e.g., a perfect conformance to the ISO date format scores higher than a fuzzy conformance). The semantic constraint sub-score is calculated by checking whether the candidate value passes all applicable business rule validations; a partial score is accumulated for each rule passed.

[0061] Then, the system loads preset weighting coefficients based on the type of table being processed (such as "financial statement" or "logistics waybill"). For example, for financial statements with strict formatting, the weighting coefficient for the format matching sub-score will be set higher; for logistics documents whose layout may be slightly messy, the weighting coefficient for the spatial distance sub-score is more important. The system uses these weighting coefficients to perform a weighted sum of the three sub-scores to obtain the basic confidence level.

[0062] Finally, the system applies a dynamic adjustment factor. The system evaluates the relationship between the three sub-scores: if the spatial, formative, and semantic scores are all high, it indicates a high degree of consistency in the evidence, and the dynamic adjustment factor is greater than 1 (e.g., 1.1), resulting in a bonus to the base confidence score. If a situation arises where the spatial score is high but the semantic score is extremely low (e.g., the numerical position is correct but it seriously violates business rules), the evidence is deemed contradictory, and the dynamic adjustment factor is less than 1 (e.g., 0.7), resulting in a penalty to the base confidence score. Multiplying the weighted base confidence score by this dynamic adjustment factor yields the final comprehensive confidence score used for decision-making.

[0063] The beneficial effect of the confidence calculation model defined in this claim is that by establishing three quantifiable evaluation dimensions—space, format, and semantics—and introducing weight configuration and dynamic consistency calibration, the reliability assessment of each correction operation becomes scientific, objective, and interpretable. This provides an accurate and reliable basis for subsequent hierarchical decisions and is the core mechanism to ensure the safety and controllability of the entire correction method.

[0064] In one possible implementation, different strategies are employed based on the overall confidence score, including: when the overall confidence score is higher than a first threshold, the correction suggestion is executed directly; when the overall confidence score is between the first and second thresholds, cross-validation based on an additional reference field is initiated, and the correction suggestion is executed only after the validation passes; when the overall confidence score is lower than the second threshold, the automatic execution of the correction suggestion is rejected and the score is marked as pending review.

[0065] The first threshold can refer to a high confidence level; exceeding this threshold indicates the revised suggestion is highly reliable. The second threshold can refer to a low confidence level; falling below this threshold indicates the revised suggestion is too risky. Cross-validation based on additional reference fields refers to the process of re-validating the spatial relationships involved in the current revised suggestion from different perspectives using other reliable fields that were not initially selected as core references. Marking it as pending review means placing the record and its detailed information (such as original content, revised suggestion, confidence score, contradictions, etc.) into a queue, awaiting manual intervention and processing.

[0066] Following the steps described above, after obtaining the overall confidence score for each suggested correction, the system enters the adaptive decision-making phase.

[0067] For correction suggestions with a score higher than 0.8 (the first threshold), the system classifies them as high-confidence corrections. These corrections typically stem from perfect spatial alignment, exact format matching, and no semantic conflicts. The system directly applies these corrections, automatically updates the field correspondences in the data structure, and triggers subsequent format standardization processes, achieving fully automated and efficient processing.

[0068] For correction suggestions with scores between 0.5 and 0.8 (between the first and second thresholds), the system classifies them as medium-confidence corrections. These corrections may have uncertainties in some aspect, such as slightly greater spatial distance or incomplete format matching. In this case, the system does not immediately adopt the suggestion but initiates a cross-validation mechanism. The system selects 2-3 other reliable references from the candidate reference pool that are different from the initial references used for localization and are not on the same row as the currently corrected field. Using these new references as benchmarks, the system re-evaluates whether the spatial relationships of the fields involved in the current correction suggestion are still reasonable. Only when the consistency score of the cross-validation results based on multiple independent references also exceeds an internal threshold (e.g., 0.7) is the system finally adopted the correction suggestion. This is equivalent to a "secondary review," significantly improving the reliability of medium-risk corrections.

[0069] For correction suggestions with a score below 0.5 (the second threshold), the system classifies them as low confidence. This typically means there are significant doubts or contradictions across multiple dimensions, including spatial, format, and semantic aspects, making automatic correction extremely risky. The system will refuse to perform any automatic modifications, maintaining the original recognition result. Simultaneously, the system will generate a detailed pending review record, clearly indicating the suspected misalignment, the system's suggested correction, and the reasons for the low confidence score (such as "excessive spatial deviation," "format not meeting expectations," "violation of business rules: negative amount," etc.). These records are then compiled into a manual review interface for final decision-making by professionals, completely eliminating the possibility of "making things worse with each correction."

[0070] The beneficial effect of the hierarchical correction decision-making strategy defined in this claim is that it achieves an optimal balance between risk and efficiency. In high-confidence scenarios, it pursues fully automated high efficiency; in medium-confidence scenarios, it improves reliability through cross-validation, expanding the scope of automation under controllable risk; and in low-confidence scenarios, it decisively entrusts the task to humans to maintain a minimum quality standard. This strategy ensures that the entire correction process is both proactive and robust, maximizing the end-to-end automation rate while minimizing the risk of error correction.

[0071] In one possible implementation, the integrity repair process for the corrected data includes at least one of the following operations: completing missing field values ​​based on spatial location relationships, removing duplicate records based on content similarity and spatial proximity, and performing format standardization on all field values.

[0072] Specifically, filling in missing field values ​​based on spatial location relationships refers to filling in missing values ​​when the system infers the value a cell should have based on column structure and row correspondence, but the recognition result shows that the cell is empty or the recognition fails. This is done by analyzing other text information (such as the original result from OCR) within that coordinate area of ​​the original image. Removing duplicate records based on content similarity and spatial proximity refers to identifying and merging fields with highly similar text content and very close spatial locations, which are likely the result of the same content being recognized multiple times. Format standardization refers to uniformly converting the same type of data into a preset standard representation, such as standardizing all dates to "YYYY-MM-DD" and all amounts to a numeric format with two decimal places and no currency symbol.

[0073] Following the steps described above, after correcting the field correspondence, the system further performs data integrity repair to output high-quality data that can be used directly.

[0074] For missing field completion, the system utilizes the established precise table structure grid. For example, in the corrected data structure, if the "Serial Number" column contains consecutive numbers, but the "Unit Price" field in row 5 is empty, the system will use the precise coordinate range of the "Unit Price" cell in row 5 to perform a reverse lookup of all text blocks provided by the OCR system that were not initially recognized. It will then search for text whose coordinates highly overlap with the target cell; for example, if it finds "￥25.00", it will be added as the missing value. This fully utilizes the redundant recognition information of the OCR.

[0075] To remove duplicate records, the system primarily addresses the issue of the same content being recorded multiple times due to bounding box splitting. The system compares all field values. If two values ​​have a text content similarity exceeding a high threshold (e.g., 0.95) and their bounding box coordinates are very close in the horizontal or vertical direction (e.g., less than 20 pixels), it is considered a duplicate. The system retains the value with the higher confidence or more complete bounding box, deletes the other, and updates the related field relationships.

[0076] For format standardization, the system cleanses all fields according to the configured standard format. For example, dates in various formats such as "January 1, 2023", "01 / 01 / 2023", and "2023-01-01" are all converted to the internally unified format "2023-01-01". Amounts such as "1,234.56", "￥1234.56", and "1234.5600" are uniformly converted to the pure numeric format "1234.56". This ensures the consistency and standardization of output data, facilitating direct data entry or analysis by downstream systems.

[0077] The beneficial effect of the integrity repair operation defined in this claim is that it elevates the endpoint of the method from simple "error correction" to comprehensive "data quality improvement". Through a series of operations such as completion, deduplication, and standardization, the output structured data not only has correct field relationships, but also possesses integrity, consistency, and standardization. This truly realizes end-to-end automated conversion from raw images to high-quality usable data, greatly enhancing the added value of the entire data processing workflow.

[0078] Corresponding to the above method embodiments, this specification also provides an embodiment of a table correction device based on spatial position relationship and multimodal verification. Figure 2 shows a schematic diagram of the structure of a table correction device based on spatial position relationship and multimodal verification provided in one embodiment of this specification. As shown in Figure 2, the device includes: a position determination module 201, configured to acquire preliminary recognition results from a table recognition system, the preliminary recognition results including the text content of multiple fields identified from the table image and the position coordinates of the multiple fields in the table image; a reference determination module 202, configured to extract at least one reference field from the preliminary recognition results based on preset rules; a correspondence module 203, configured to analyze the spatial positional relationship between all fields in the table image based on the position coordinates of the reference field, so as to determine the logical column structure of the table and the row correspondence between fields; a correction suggestion module 204, configured to perform multi-strategy matching between field names and candidate field values ​​based on the logical column structure and row correspondence to obtain correction suggestions; and a repair processing module 205, configured to calculate a comprehensive confidence score for each correction suggestion; and to implement or reject the corresponding correction suggestion using different strategies according to the level of the comprehensive confidence score, and to perform integrity repair processing on the corrected data.

[0079] In one possible implementation, the reference field includes at least one of the following types: a unique identifier field conforming to a preset encoding rule, a feature data field conforming to a preset format feature, and a composite identifier field composed of letters and numbers.

[0080] In one possible implementation, analyzing the spatial relationships between all fields in a table image includes: based on the coordinates of a reference field, using an adaptive tolerance clustering method to group fields with similar horizontal positions into the same logical column, and verifying the alignment of field name labels with candidate field values ​​in the vertical direction; simultaneously, applying semantic constraints related to the table type to verify the rationality of the field values.

[0081] In one possible implementation, multi-strategy matching includes at least one of an exact matching strategy, an inclusion relation matching strategy, and a format pattern matching strategy; the exact matching strategy is used when the content of the candidate field value is completely consistent with the expected value inferred based on the spatial location relationship; the inclusion relation matching strategy is used when the content of the candidate field value is part of the correct content; and the format pattern matching strategy matches the data format of the field value with the semantic expectation of the field name.

[0082] In one possible implementation, the overall confidence score is obtained by weighted synthesis of spatial distance sub-score, format matching sub-score, and semantic constraint sub-score, calculated as the product of the base confidence score and a dynamic adjustment factor, where the base confidence score is:

[0083] in, For spatial distance sub-scores, For format matching sub-scores, Score the semantic constraint sub-score. , , These are the weighting coefficients.

[0084] In one possible implementation, different strategies are employed based on the overall confidence score, including: when the overall confidence score is higher than a first threshold, the correction suggestion is executed directly; when the overall confidence score is between the first and second thresholds, cross-validation based on an additional reference field is initiated, and the correction suggestion is executed only after the validation passes; when the overall confidence score is lower than the second threshold, the automatic execution of the correction suggestion is rejected and the score is marked as pending review.

[0085] In one possible implementation, the integrity repair process for the corrected data includes at least one of the following operations: completing missing field values ​​based on spatial location relationships, removing duplicate records based on content similarity and spatial proximity, and performing format standardization on all field values.

[0086] The above is a schematic scheme of a table correction device based on spatial position relationship and multimodal verification according to this embodiment. It should be noted that the technical solution of this table correction device based on spatial position relationship and multimodal verification belongs to the same concept as the technical solution of the table correction method based on spatial position relationship and multimodal verification described above. For details not described in detail in the technical solution of the table correction device based on spatial position relationship and multimodal verification, please refer to the description of the technical solution of the table correction method based on spatial position relationship and multimodal verification described above.

[0087] Figure 3 shows a structural block diagram of a computing device 300 according to one embodiment of this specification. The components of the computing device 300 include, but are not limited to, a memory 310 and a processor 320. The processor 320 is connected to the memory 310 via a bus 330, and a database 350 is used to store data.

[0088] The computing device 300 also includes an access device 340, which enables the computing device 300 to communicate via one or more networks 360. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 340 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0089] In one embodiment of this specification, the aforementioned components of the computing device 300, as well as other components not shown in FIG. 3, may be connected to each other, for example, via a bus. It should be understood that the block diagram of the computing device shown in FIG. 3 is merely for illustrative purposes and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0090] The computing device 300 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 300 can also be a mobile or stationary server.

[0091] The processor 320 executes computer-executable instructions, which, when executed by the processor, implement the steps of the table correction method based on spatial location relationships and multimodal verification described above. The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the table correction method based on spatial location relationships and multimodal verification described above belong to the same concept. Details not described in detail in the technical solution of the computing device can be found in the description of the technical solution of the table correction method based on spatial location relationships and multimodal verification described above.

[0092] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the table correction method based on spatial positional relationships and multimodal verification described above.

[0093] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the table correction method based on spatial position relationship and multimodal verification described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the table correction method based on spatial position relationship and multimodal verification described above.

[0094] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the table correction method based on spatial positional relationship and multimodal verification described above.

[0095] The above is an illustrative example of a computer program according to this embodiment. It should be noted that the technical solution of this computer program belongs to the same concept as the aforementioned table correction method based on spatial positional relationship and multimodal verification. Details not described in detail in the computer program's technical solution can be found in the description of the aforementioned table correction method based on spatial positional relationship and multimodal verification.

[0096] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0097] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0098] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0099] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0100] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A table correction method based on spatial location relationship and multimodal verification, characterized in that, The method includes: acquiring preliminary recognition results from a table recognition system, the preliminary recognition results including the text content of multiple fields identified from a table image and the position coordinates of the multiple fields in the table image; extracting at least one reference field from the preliminary recognition results based on preset rules; analyzing the spatial positional relationship between all fields in the table image based on the position coordinates of the reference field to determine the logical column structure of the table and the row correspondence between fields; performing multi-strategy matching between field names and candidate field values ​​based on the logical column structure and the row correspondence to obtain correction suggestions; calculating a comprehensive confidence score for each correction suggestion; and implementing or rejecting the corresponding correction suggestion using different strategies according to the level of the comprehensive confidence score, and performing integrity repair processing on the corrected data.

2. The method according to claim 1, characterized in that, The reference field includes at least one of the following types: a unique identifier field that conforms to a preset encoding rule, a feature data field that conforms to a preset format feature, and a composite identifier field composed of letters and numbers.

3. The method according to claim 1, characterized in that, The analysis of the spatial relationship between all fields in the table image includes: based on the coordinates of the reference field, using an adaptive tolerance clustering method to group fields with similar horizontal positions into the same logical column, and verifying the alignment of field name labels and candidate field values ​​in the vertical direction; at the same time, applying semantic constraints related to the table type to verify the rationality of the field values.

4. The method according to claim 1, characterized in that, The multi-strategy matching includes at least one of the following: exact matching strategy, inclusion relationship matching strategy, and format pattern matching strategy; the exact matching strategy is used when the content of the candidate field value is completely consistent with the expected value inferred based on the spatial location relationship; the inclusion relationship matching strategy is used when the content of the candidate field value is part of the correct content; the format pattern matching strategy matches the data format of the field value with the semantic expectation of the field name.

5. The method according to claim 1, characterized in that, The overall confidence score is obtained by weighted synthesis of spatial distance sub-score, format matching sub-score, and semantic constraint sub-score, and is calculated as the product of the base confidence score and a dynamic adjustment factor, wherein the base confidence score is: in, The spatial distance sub-score is given. The score is the sub-score for the specified format. Score the semantic constraint sub-score. , , These are the weighting coefficients.

6. The method according to claim 1 or 5, characterized in that, Different strategies are employed based on the overall confidence score, including: when the overall confidence score is higher than a first threshold, the correction suggestion is executed directly; when the overall confidence score is between the first and second thresholds, cross-validation based on an additional reference field is initiated, and the correction suggestion is executed only after the validation passes; when the overall confidence score is lower than the second threshold, the automatic execution of the correction suggestion is rejected and the score is marked as pending review.

7. The method according to claim 1, characterized in that, The integrity restoration process for the corrected data includes at least one of the following operations: completing missing field values ​​based on spatial location relationships, removing duplicate records based on content similarity and spatial proximity, and performing format standardization on all field values.

8. A table correction device based on spatial positional relationship and multimodal verification, characterized in that, include: The location determination module is configured to obtain preliminary recognition results from the table recognition system, the preliminary recognition results including the text content of multiple fields identified from the table image and the position coordinates of the multiple fields in the table image; The reference determination module is configured to extract at least one reference field from the preliminary identification results based on preset rules; The correspondence module is configured to analyze the spatial positional relationship between all fields in the table image based on the position coordinates of the reference field, so as to determine the logical column structure of the table and the row correspondence between fields; The correction suggestion module is configured to perform multi-strategy matching between field names and candidate field values ​​based on the logical column structure and the row correspondence to obtain correction suggestions. The repair processing module is configured to calculate a comprehensive confidence score for each of the aforementioned repair suggestions; Based on the overall confidence score, different strategies are adopted to implement or reject the corresponding correction suggestions, and the corrected data is subjected to integrity restoration processing.

9. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the table correction method based on spatial position relationship and multimodal verification as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the table correction method based on spatial positional relationship and multimodal verification as described in any one of claims 1 to 7.

Images