A structure-aware semantic positioning and traceable data extraction method, device and equipment for heterogeneous electronic spreadsheets and a medium

Abstract

The application discloses a structure perception semantic positioning and traceable data extraction method and device for heterogeneous electronic spreadsheets, equipment and medium, comprising: carrying out partition processing on tab data according to data row continuity, blank interval threshold and area capacity limit to obtain a plurality of data areas; screening data areas to be processed according to area abstract information and field description information; performing field semantic matching, legality verification and matching verification result summarization on the corresponding selectable cells based on the candidate identifier record; obtaining corresponding field values and position information from the corresponding selectable cell mapping according to the final field candidate identifier of each field; performing coordinate positioning and summarization processing on the extraction results of each extraction mode, and outputting the field extraction result, wherein the field extraction result at least contains field values and coordinate information. The application belongs to the field of data extraction. The application can improve the authenticity of the extraction result.

Description

Technical Field

[0001] This invention relates to the field of data extraction, and in particular to a method, apparatus, device, and medium for structure-aware semantic localization and traceable data extraction for heterogeneous spreadsheets. Background Technology

[0002] Existing spreadsheet data extraction technologies typically employ preset templates, rule matching, or convert the worksheet to text and then use natural language processing models for field identification and data extraction. Among these, template-based methods require pre-defining page header names, header levels, field positions, and data arrangement. When different workbooks from different sources differ in page header naming, header structure, cell merging methods, or data arrangement direction, template mismatch issues can easily occur.

[0003] To address the aforementioned issues, cell content in a worksheet can be converted into linear text or document content for analysis, achieving a certain degree of semantic recognition. However, this conversion process can easily weaken or even lose the original cell's row and column relationships, merging relationships, and spatial context, making it difficult to accurately map the extracted results to their actual location within the original worksheet. Furthermore, when the same field value or text appears repeatedly in multiple places within the worksheet, relying solely on the field value for reverse lookup makes it even more difficult to determine the actual location of the field, easily leading to misidentification of the same data in different areas such as the design area, measurement area, and summary area as the target data. Therefore, accurately locating fields in complex tables is a pressing problem that needs to be solved. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and medium for structure-aware semantic localization and traceable data extraction in heterogeneous spreadsheets, solving the technical problem of how to accurately locate fields in complex spreadsheets and improving the accuracy of field localization in complex spreadsheets.

[0005] In a first aspect, the present invention provides a method for structure-aware semantic localization and traceable data extraction for heterogeneous spreadsheets, including: Obtain the tab data that meets the tab selection criteria in the target workbook, and partition the tab data according to the data row continuity, blank interval threshold and area capacity limit to obtain several data areas; The data regions to be processed are filtered based on the region summary information and field description information, and candidate identifier records are generated for the selectable cells in the data regions to be processed. The candidate identifier records include location information, content information and context information. The data regions to be processed include several selectable cells. Based on the candidate identifier records, perform field semantic matching and legality verification on the corresponding selectable cells, summarize the matching and verification results, and determine the final field candidate identifiers for each field. Based on the final field candidate identifier of each field, the corresponding field value and position information are obtained from the corresponding selectable cell mapping, and the extraction mode of each field is determined. The extraction mode includes direct extraction mode or block extraction mode. The extraction results of each extraction mode are located and summarized by coordinates, and the field extraction results are output. The field extraction results include at least the field values ​​and coordinate information.

[0006] Furthermore, the tab data that meets the tab selection criteria in the target workbook is obtained, and the tab data is partitioned according to data row continuity, blank interval threshold, and area capacity limit to obtain several data areas, including: Receive the workbook identifier, tab selection criteria, and field description information, and load the corresponding tab data according to the tab selection criteria; Retrieve the content of non-empty cells, cell row and column positions, and anchor point relationships of merged cells corresponding to the tab data; When partitioning tab data, an initial data region is formed based on consecutive data rows; The initial data region is segmented when consecutive blank rows exceed the blank interval threshold. When the area capacity exceeds the area capacity limit, the data is further divided to obtain several data areas; Configure overlapping rows and inherited header contexts for adjacent data areas.

[0007] Furthermore, the data region to be processed is filtered based on the region summary information and field description information, and candidate identifier records are generated for the selectable cells in the data region to be processed, including: Determine the regional boundaries and distributed sampling content of each data region based on the regional summary information; When the number of regions exceeds the preset number, filter out the data regions to be processed from the data regions based on the field description information; Generate candidate identifier records for selectable cells in the data range to be processed.

[0008] Furthermore, based on the candidate identifier records, the corresponding selectable cells are subjected to field semantic matching, validity verification, and the matching verification results are summarized to determine the final field candidate identifiers for each field, including: Based on the field keys and candidate identifiers in the candidate identifier records, perform field semantic matching on the corresponding selectable cells; The validity of the matching results is verified based on the validity of the candidate identifier, the type of the field key, and the format of the candidate content. Summarize the matching and verification results for each data region to be processed; Based on the uniqueness and validity of the candidate identifiers corresponding to each field, the final candidate identifiers for each field are determined.

[0009] Furthermore, based on the uniqueness and validity of the candidate identifiers corresponding to each field, the final candidate identifiers for each field are determined, including: When a field corresponds to a unique and valid candidate identifier, that candidate identifier is taken as the final candidate identifier for that field. When a field corresponds to multiple valid candidate identifiers, a review set is constructed, and the final candidate identifier of the field is re-determined based on the constraints.

[0010] Furthermore, determine the extraction mode for each field, including: When all fields have candidate identifiers for the final field, the direct extraction mode is used; When at least one field does not correspond to the final field candidate identifier, the uncorresponding field is extracted in a block extraction mode, while the remaining fields are extracted directly.

[0011] Furthermore, the extraction results from each extraction mode are processed for coordinate positioning and summarization, and the field extraction results are output. The field extraction results include at least the field values ​​and coordinate information, including: When a unique data value is matched, the corresponding coordinate information is determined; When multiple identical data values ​​are matched, coordinate information is determined based on row-level relationships, column-level relationships, proximity context information, and field description information. The location results corresponding to each extraction mode are summarized and processed, and the field extraction results are output.

[0012] Secondly, the present invention provides a structure-aware semantic localization and traceable data extraction device for heterogeneous spreadsheets, comprising: The acquisition module is used to acquire tab data that meets the tab selection criteria in the target workbook, and to partition the tab data according to the data row continuity, blank interval threshold and area capacity limit to obtain several data areas. The cell module is used to filter the data region to be processed based on the region summary information and field description information, and generate candidate identifier records for the selectable cells in the data region to be processed. The candidate identifier records include location information, content information and context information. The data region to be processed includes several selectable cells. The identifier determination module is used to perform field semantic matching, legality verification, and matching verification result summary on the corresponding selectable cells based on the candidate identifier records, and to determine the final field candidate identifier for each field. The extraction module is used to obtain the corresponding field value and position information from the corresponding selectable cell mapping based on the final field candidate identifier of each field, and to determine the extraction mode of each field. The extraction mode includes direct extraction mode or block extraction mode. The summary output module is used to perform coordinate positioning and summary processing on the extraction results of each extraction mode, and output the field extraction results. The field extraction results must include at least the field values ​​and coordinate information.

[0013] Thirdly, the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to execute a structure-aware semantic localization and traceable data extraction method for heterogeneous spreadsheets, as provided in the first aspect.

[0014] Fourthly, the present invention provides a non-transitory computer-readable storage medium that, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to execute a structure-aware semantic localization and traceable data extraction method for heterogeneous spreadsheets as provided in the first aspect.

[0015] One or more technical solutions provided in this invention have at least the following technical effects or advantages: This invention avoids the model's freely generated content from entering the final result by having a semantic model select candidate identifiers and a server read field values ​​and coordinates from the original cell mapping. This effectively suppresses model fictitiousness and ensures that the source of the extracted results is real and traceable.

[0016] This invention addresses multiple identical field values ​​by combining field descriptions, parent headings, row and column relationships, and adjacent context for candidate selection, thereby improving the ability to resolve ambiguities in complex tables. Through regional pre-routing, distributed sampling, full-area rescanning for missed values, and cross-batch verification, this invention reduces the amount of irrelevant cells processed while lowering the risk of missed detections and minimizing the impact of processing order on the final result.

[0017] For complex structures such as hierarchical headers, repeated headers, group titles, merged record bands, and adjacent heterogeneous table blocks, this invention can perform differentiated processing to reduce cross-regional erroneous extraction.

[0018] This invention supports reverting to block extraction when only some fields are matched, and ensures stable operation of the processing chain through deterministic coordinate positioning. It can also be extended to multi-tab scenarios, maintaining the hierarchical correspondence between requests, tabs, fields, and coordinates. This invention boasts high generalization, accuracy, traceability, and processing efficiency. It eliminates the need to pre-create complete templates for different workbooks, utilizing cell coordinates, merge relationships, blank spaces, area boundaries, and title context to adapt to heterogeneous table layouts. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a structure-aware semantic localization and traceable data extraction method for heterogeneous spreadsheets provided by the present invention. Detailed Implementation

[0021] This invention provides a structure-aware semantic localization and traceable data extraction method for heterogeneous spreadsheets, solving the technical problem of how to accurately locate fields in complex spreadsheets in the prior art.

[0022] The technical solution of this invention is to solve the above-mentioned technical problems, and the overall idea is as follows: A method for structure-aware semantic localization and traceable data extraction in heterogeneous spreadsheets includes: acquiring tab data that meets the tab selection criteria in a target workbook, and partitioning the tab data according to data row continuity, blank interval threshold, and area capacity limit to obtain several data areas; filtering the data areas to be processed based on area summary information and field description information, and generating candidate identifier records for selectable cells in the data areas to be processed, the candidate identifier records including location information, content information, and context information, the data areas to be processed including several selectable cells; performing field semantic matching, legality verification, and summarizing the matching verification results for the corresponding selectable cells based on the candidate identifier records, and determining the final field candidate identifier for each field; obtaining the corresponding field value and location information from the corresponding selectable cell mapping based on the final field candidate identifier for each field, and determining the extraction mode for each field, the extraction mode including direct extraction mode or block extraction mode; performing coordinate positioning and summarizing processing on the extraction results of each extraction mode, and outputting the field extraction results, the field extraction results including at least field value and coordinate information.

[0023] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0024] First, it should be clarified that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0025] The various thresholds in this invention can be determined based on actual conditions or experiments, and are not limited in this invention.

[0026] This invention provides, for example Figure 1 The method for structure-aware semantic localization and traceable data extraction for heterogeneous spreadsheets, shown below, includes steps S11-S15: Step S11: Obtain the tab data in the target workbook that meets the tab selection criteria, and partition the tab data according to the data row continuity, blank interval threshold and area capacity limit to obtain several data areas.

[0027] This includes: receiving workbook identifiers, tab selection criteria, and field description information, and loading the corresponding tab data based on the tab selection criteria; obtaining the content of non-empty cells, cell row and column position information, and merged cell anchor point relationships corresponding to the tab data; forming an initial data region based on consecutive data rows when partitioning the tab data; splitting the initial data region when consecutive blank rows exceed the blank interval threshold; continuing to split when the region capacity exceeds the region capacity limit to obtain several data regions; and configuring overlapping rows and inherited title context for adjacent data regions.

[0028] Specifically: First, the system receives a workbook identifier, tab selection criteria, and one or more field descriptions. The workbook identifier is used to determine the target workbook to be processed, the tab selection criteria are used to determine one or more tabs to be processed from the target workbook, and the field descriptions are used to characterize the target fields and their semantic features that need to be located or extracted later.

[0029] After the tabs are selected, the tab data that meets the tab selection criteria is loaded, and the cells in the tabs are parsed to obtain at least the content of non-empty cells, the row and column position information of each non-empty cell, and the anchor point relationship of merged cells.

[0030] Among them, the content of non-empty cells is used to reflect the actual data in the tab and structural information such as titles and headers; the row and column position information is used to maintain the spatial position of each cell in the original tab; and the anchor point relationship of merged cells is used to determine the actual starting cell corresponding to the merged area and its associated range.

[0031] Based on this, rows in the tabs are used as the basic partitioning unit. Continuous data rows are identified based on whether there are valid non-empty cells in each row, and an initial data area is formed according to the continuous data rows. In other words, when multiple adjacent rows continuously contain valid data, they are treated as the same initial data area.

[0032] After forming the initial data area according to the continuous data rows, the blank interval between adjacent data rows is determined along the row direction of the tab data; when the number of consecutive blank rows exceeds the preset blank interval threshold, the consecutive blank rows are regarded as structural separations between different data blocks, and the initial data area is segmented at the corresponding position.

[0033] For data regions that are still large after being segmented with blank intervals, the number of cells in the region is further counted and compared with the preset region capacity limit. When the number of cells in a region exceeds the region capacity limit, the original row-level structure is not changed. Instead, the region is segmented according to complete data rows so that each segmented data region is composed of complete rows. This avoids directly truncating a row according to the number of cells, which would destroy the correspondence between fields in the same row.

[0034] After completing the above segmentation, configure overlapping rows for adjacent data areas so that even if adjacent areas retain a preset number of common data rows at the boundary position, the field values, table headers or context information near the area boundary will not be completely cut off due to the area segmentation. At the same time, the titles, headers, and other title contexts that can characterize the region structure that have been identified in the previous region are inherited to the adjacent subsequent data regions, so that even if the subsequent regions do not contain complete titles, they can still obtain the field semantics and structural context corresponding to the data blocks to which they belong.

[0035] Step S12: Filter the data region to be processed based on the region summary information and field description information, and generate candidate identifier records for the selectable cells in the data region to be processed. The candidate identifier records include location information, content information and context information. The data region to be processed includes several selectable cells.

[0036] This includes: determining the regional boundaries and distributed sampling content of each data region based on the regional summary information; when the number of regions exceeds the preset number, filtering out the data regions to be processed from the data regions based on the field description information; and generating candidate identifier records for selectable cells in the data regions to be processed.

[0037] Specifically: For the obtained data regions, the region boundaries are determined according to the actual data range contained in each data region. The region boundaries are used to characterize the starting row, ending row and corresponding column range of the data region in the original tab. Meanwhile, within each data region, the contents of non-empty cells are selected as distributed sampling content according to a preset distributed sampling method, so that the region summary information not only includes the spatial range of the data region, but also reflects the actual data characteristics at different locations in the data region.

[0038] The resulting regional summary information is used to characterize the overall structure and content distribution of each data region.

[0039] When the number of data regions exceeds the preset number, the field description information is matched with the region summary information of each data region to determine one or more data regions that are highly correlated with the field to be located, and the matched data regions are determined as the data regions to be processed. The field description information may include the field name, field semantics, field characteristics, or data content characteristics used to describe the target field.

[0040] For fields where relevant data regions cannot be pre-selected based on field description information, or where region pre-selection has been performed but no valid results have been obtained in the pre-selected regions, the region pre-selection results will not be regarded as the final limitation. Instead, the qualification to perform full-region supplementary scanning for subsequent processing of all data regions will be reserved. This will avoid the omission of target fields due to insufficient sampling of region summaries, differences in expression between field descriptions and region content, or the target field being located in an unpre-selected region.

[0041] For each selectable cell in the data region to be processed, a stable candidate identifier is generated, and a candidate identifier record corresponding to the candidate identifier is established. The candidate identifier record includes a field key, candidate identifier, position information, content information, and context information. The field key identifies the field to be located corresponding to the candidate cell, the candidate identifier uniquely points to the corresponding candidate cell in the subsequent selection process, the location information includes at least the A1 coordinates and row and column array of the cell in the original workbook, and the content information includes the original value of the cell; the region identifier is used to determine the data region to which the candidate cell belongs, and the context information is used to describe the structural information such as the row, column, adjacent cells and related titles around the candidate cell, so that the candidate cells can be distinguished according to the field semantics and table structure in the future.

[0042] It should be noted that the title context inherited from adjacent data areas can be used as context information for candidate cells in subsequent judgments, but the title-inherited cells themselves are not selectable cells, so no selectable candidate identifiers are generated for such cells; only cells in the data area to be processed that actually meet the selection criteria are generated as candidate identifier records.

[0043] Step S13: Based on the candidate identifier records, perform field semantic matching, legality verification, and summarize the matching verification results for the corresponding selectable cells, and determine the final field candidate identifiers for each field.

[0044] This includes: performing semantic field matching on the corresponding selectable cells based on the field keys and candidate identifiers in the candidate identifier records; validating the matching results based on the validity of the candidate identifiers, the type of the field keys, and the format of the candidate content; summarizing the matching verification results for each data area to be processed; and determining the final field candidate identifier for each field based on the uniqueness and validity of the candidate identifiers corresponding to each field, including: when a field corresponds to a unique and valid candidate identifier, using that candidate identifier as the final field candidate identifier for that field; when a field corresponds to multiple valid candidate identifiers, constructing a review set and redetermining the final field candidate identifier for that field based on constraints.

[0045] Specifically: The field key in the candidate identifier record is used as the identifier of the field to be matched, and the candidate identifier is used as the unique reference of the corresponding selectable cell. Instead of directly requiring the semantic model to return the cell content, A1 coordinates or other structured results, the semantic model is provided with the candidate identifier record corresponding to each field and its necessary context information. This allows the semantic model to determine whether the candidate cell semantically matches the field based on the field description, field key, original value of the selectable cell, and information such as the same row, same column, adjacent cells and inherited header context.

[0046] The semantic model returns only a hit / miss status and a candidate identifier for each field. "Hit / miss status" indicates whether the current candidate meets the field's semantic requirements, and the "candidate identifier" points to the selected cell that was hit. The server performs validity checks on the results returned by the semantic model, rather than directly trusting the model's output. First, check if the returned candidate identifier exists in the candidate identifier record corresponding to the current field. If a non-existent candidate identifier is returned, the result is judged as an illegal result and rejected. Second, check if the data type of the field key meets the preset requirements. If the field key type is abnormal, the corresponding result is rejected. The format of the candidate content is checked again. If the model returns the candidate content as a list instead of the required string, or if the returned data structure does not conform to the preset format, it is also judged as an illegal result.

[0047] For header cells formed by header inheritance relationships and used only to provide context, even if the semantic model incorrectly returns them as the target field hit object, they are validated and rejected by the type and selectable attributes of the cell to which the candidate identifier belongs, thus ensuring that only the legal selectable cells in the data area to be processed are ultimately involved in the field candidate determination.

[0048] Furthermore, since data regions can be pre-selected based on region summary information and field description information, and the eligibility for full-region rescanning is still retained for fields that are not pre-selected or for which pre-selection yields no results, Therefore, the same field may produce matching results in multiple data regions or multiple region batches; the server merges the valid candidate identifiers returned by each region batch according to the field key and removes the results that failed the aforementioned validity check.

[0049] When a field corresponds to only one unique and valid candidate identifier after aggregation, no additional semantic review is performed, and the candidate identifier is directly determined as the final candidate identifier for that field. When a field corresponds to two or more valid candidate identifiers, the valid candidate identifiers and their candidate identifier records are constructed into a review set for that field, and candidate selection is performed again under server constraints. The review process is required to select candidate identifiers that already exist and have passed the validity check from the review set, and it is not allowed to regenerate candidate identifiers or return identifiers outside the set.

[0050] During the review, the location information, content information, and context information in the candidate identifier record are used to distinguish multiple legitimate candidates in combination with the field semantics; if the review yields a unique and constrained candidate identifier, it is determined as the final field candidate identifier for that field.

[0051] Step S14: Based on the final field candidate identifier of each field, obtain the corresponding field value and position information from the corresponding selectable cell mapping, and determine the extraction mode of each field. The extraction mode includes direct extraction mode or block extraction mode.

[0052] This includes: when all fields have a final field candidate identifier, the direct extraction mode is used; when at least one field does not correspond to a final field candidate identifier, the uncorresponding field is extracted in a block extraction mode, and the remaining fields are extracted directly.

[0053] Specifically: Optional cell mapping is used to establish a deterministic correspondence between candidate identifiers and original worksheet cells, wherein at least the original cell value corresponding to the candidate identifier and the original cell location information are stored.

[0054] Therefore, after determining the final candidate identifier of a certain field, the server uses the final candidate identifier as an index to directly read the original field value and position information corresponding to the candidate identifier from the corresponding selectable cell mapping. The position information includes at least the A1 coordinates in the original worksheet.

[0055] The server does not use field values ​​or coordinates generated or supplemented by the semantic model as the final extraction result. Even if the model returns a string that is different from the original cell content or generates coordinates by itself, it only uses the real cell values ​​and position information that already exist in the selectable cell mapping as the basis for the final output.

[0056] Therefore, the field values ​​and location information of each field can be traced back to the actual cells in the original worksheet, avoiding inconsistencies between the returned values ​​and the actual data in the worksheet caused by the model freely generating content.

[0057] After obtaining the field values ​​and position information corresponding to each field, the extraction mode is determined based on whether there is a final field candidate identifier for each field.

[0058] When all fields have candidate identifiers for the final field, and each candidate identifier for the final field passes the aforementioned validity check, the direct extraction mode is adopted. In this case, all fields can be directly read from the corresponding selectable cell mapping without the need for block extraction, thereby reducing the overhead of repeated data scanning and model processing.

[0059] When at least one field does not correspond to a final field candidate identifier, the extraction is not repeated for fields that have already obtained a valid final field candidate identifier. Instead, the field that does not correspond to a final field candidate identifier is identified as the field that needs to be extracted in block mode. The remaining fields that have obtained a final field candidate identifier continue to be extracted in direct mode, thus forming a processing method in which direct extraction mode and block extraction mode are applied in parallel.

[0060] For uncorresponding fields that enter the block extraction mode, the server divides the original tab data into blocks by row according to the preset context window budget, ensuring that each block can provide the necessary field context while controlling the data scale of a single processing. The number of extraction tasks that can be executed simultaneously is further limited according to the tab-level concurrency limit and the block-level concurrency limit. The waiting time is controlled through the task queue and the queue timeout mechanism. When a block task enters the queue and has not obtained processing resources after the preset queue timeout time, the task is terminated or marked according to the preset timeout rules.

[0061] Step S15: Perform coordinate positioning and summary processing on the extraction results of each extraction mode, and output the field extraction results. The field extraction results shall at least include field values ​​and coordinate information.

[0062] This includes: determining the corresponding coordinate information when a unique data value is matched; determining the coordinate information based on row-to-row, column-to-row, adjacent context information, and field description information when multiple identical data values ​​are matched; summarizing the positioning results corresponding to each extraction mode and outputting the field extraction results.

[0063] Specifically, it includes: For a certain field value returned by the block extraction mode, perform data matching in the original tab data; When the field value matches only one data value within the corresponding valid data range, the cell containing the data value is directly determined as the corresponding coordinate information, and the A1 coordinate of the cell and the corresponding basic row and column array are obtained.

[0064] A unique data value refers to a situation where there are no other identical matching values ​​within the valid matching range of the current field. Therefore, a deterministic approach can be used to directly complete the coordinate positioning.

[0065] When the same field value matches multiple identical data values ​​in the original tab, the position of any identical data value is not directly used as the final coordinate. Instead, the true coordinate candidate is constructed based on the row relationship, column relationship, adjacent context information and field description information between the candidate data values. Among them, the row relationship is used to determine the degree of association between the candidate data value and the row or related header of the field; the column relationship is used to determine whether the candidate data value is in the column structure corresponding to the target field; the adjacent context information is used to distinguish the candidate cell by using the title, header and related data around the candidate cell; and the field description information is used to determine the degree of matching between the semantics of different candidate positions and the target field.

[0066] When selecting from multiple real coordinate candidates using a semantic model, the semantic model is only allowed to return candidate identifiers that have already been generated, and is not allowed to generate coordinates freely; the server performs validity checks on the returned candidate identifiers and retains deterministic fallback coordinates in abnormal situations, thereby ensuring that the coordinate information always corresponds to the real cells in the original worksheet.

[0067] After completing the coordinate positioning for both the direct extraction mode and the block extraction mode, the positioning results for each extraction mode are further summarized according to the field key. The direct extraction results, block extraction results, and necessary coordinate backtracking results for the same field are uniformly merged. For the direct extraction mode, since the actual field value and position information have already been read from the selectable cell mapping based on the final field candidate identifier, its coordinates can be directly used as the positioning result for that field. For the block extraction mode, coordinates are obtained by directly locating unique values ​​or by using candidate locations based on row / column relationships, adjacent context information, and field description information for duplicate values. During summarization, the field key is used as the association basis to determine the final field value and its corresponding coordinates for the same field, while retaining error information or abnormal states generated during the location process. For abnormal cases where the semantic model cannot complete the selection, deterministic backtracking coordinates are used to avoid losing the already determined true location due to abnormal model output. The final output field extraction results include at least field values ​​and coordinate information, where the coordinate information can at least include the A1 coordinate and its corresponding basic row and column array, ensuring that each output field value can be traced back to a specific cell in the original worksheet. Furthermore, by uniformly summarizing the results of each extraction mode by field key, duplicates or inconsistent results for the same field caused by separate outputs from the direct extraction mode and the block extraction mode are avoided.

[0068] The following are some of the parameter settings provided by this invention: Blank interval threshold; 40–200, e.g., 80; controls fine-grained candidate context; can be appropriately reduced for complex merge tables; Maximum number of cells in a single batch; 100-600, e.g., 300; controls the size of a single model request. Number of overlapping rows in a region; 0 to 5, e.g., 2; to prevent labels and values ​​from being cut into different regions; Inherit the line number of the heading context; 0 to 6, for example, 3; Preserve the semantics of the parent heading or group heading; The number of consecutive blank rows allowed within a region; 0 to 3, for example, 1; distinguish between intervals within the same table block and independent table blocks; Hierarchical header search depth; 2~12, e.g. 8; Limit the collection range of parent paths; Maximum number of candidate coordinates per field; 5 to 50, e.g., 20; to prevent excessive duplicate values ​​from causing context bloat. Number of context cells per axis; 4–30, e.g., 12; provides row, column, and proximity semantics; Tab concurrency limit; 1 to 16, e.g., 4; balance throughput with external model capacity; Block concurrency limit; 1 to 32, for example 8; controls the amount of parallel processing for large tabs; Queue waiting timeout; 5-120 seconds, e.g., 30 seconds; to avoid infinite waiting when overloaded.

[0069] In summary, this invention, by having a semantic model select candidate identifiers and a server read field values ​​and coordinates from the original cell mapping, avoids the model freely generating content in the final result, effectively suppressing model fictitiousness and ensuring the authenticity and traceability of the source of the extracted results.

[0070] This invention addresses multiple identical field values ​​by combining field descriptions, parent headings, row and column relationships, and adjacent context for candidate selection, thereby improving the ability to resolve ambiguities in complex tables. Through regional pre-routing, distributed sampling, full-area rescanning for missed values, and cross-batch verification, this invention reduces the amount of irrelevant cells processed while lowering the risk of missed detections and minimizing the impact of processing order on the final result.

[0071] For complex structures such as hierarchical headers, repeated headers, group titles, merged record bands, and adjacent heterogeneous table blocks, this invention can perform differentiated processing to reduce cross-regional erroneous extraction.

[0072] This invention supports reverting to block extraction when only some fields are matched, and ensures stable operation of the processing chain through deterministic coordinate positioning. It can also be extended to multi-tab scenarios, maintaining the hierarchical correspondence between requests, tabs, fields, and coordinates. This invention boasts high generalization, accuracy, traceability, and processing efficiency. It eliminates the need to pre-create complete templates for different workbooks, utilizing cell coordinates, merge relationships, blank spaces, area boundaries, and title context to adapt to heterogeneous table layouts.

[0073] Based on the same inventive concept, this invention provides a structure-aware semantic localization and traceable data extraction device for heterogeneous spreadsheets, comprising: The acquisition module is used to acquire tab data that meets the tab selection criteria in the target workbook, and to partition the tab data according to the data row continuity, blank interval threshold and area capacity limit to obtain several data areas. The cell module is used to filter the data region to be processed based on the region summary information and field description information, and generate candidate identifier records for the selectable cells in the data region to be processed. The candidate identifier records include location information, content information and context information. The data region to be processed includes several selectable cells. The identifier determination module is used to perform field semantic matching, legality verification, and matching verification result summary on the corresponding selectable cells based on the candidate identifier records, and to determine the final field candidate identifier for each field. The extraction module is used to obtain the corresponding field value and position information from the corresponding selectable cell mapping based on the final field candidate identifier of each field, and to determine the extraction mode of each field. The extraction mode includes direct extraction mode or block extraction mode. The summary output module is used to perform coordinate positioning and summary processing on the extraction results of each extraction mode, and output the field extraction results. The field extraction results must include at least the field values ​​and coordinate information.

[0074] Based on the same inventive concept, the present invention also provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to execute a structure-aware semantic localization and traceable data extraction method for heterogeneous spreadsheets, as described above.

[0075] Based on the same inventive concept, the present invention also provides a non-transitory computer-readable storage medium, which, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to execute a structure-aware semantic localization and traceable data extraction method for heterogeneous spreadsheets as described above.

[0076] Based on the same inventive concept, this invention also provides a structure-aware semantic localization and traceable data extraction system for heterogeneous spreadsheets: The spreadsheet loading module reads available tabs from the target workbook, converts the tab content into two-dimensional data that retains row and column indices, and reads the anchor point relationships of merged cells. Tab selection can use exact names, keywords, or explicit regular expression rules; when multiple tabs are matched by explicit regular expressions, they can be processed separately while maintaining the correspondence between the request and the results of each tab.

[0077] The coordinate and merge relationship preservation module generates stable candidate identifiers, A1 coordinates, and a base row and column array for each non-empty cell, while establishing a mapping from candidate identifiers to original values, coordinates, and merge ranges. The merged cell coverage area is preserved as text metadata, but the final value still returns to its true anchor point, avoiding the model inferring coordinates from the displayed text.

[0078] The structured region building module divides the worksheet into multiple regions based on consecutive non-empty rows, the allowed number of consecutive blank rows to span, and the maximum number of cells in a single region. When a region is too large, it continues to split along complete rows, carrying a preset number of overlapping rows between adjacent regions; simultaneously, it inherits the nearest text header or grouping row from above the region as an unselectable context cell. This controls the scale of semantic processing at a time and reduces the probability of labels and values ​​separating at split boundaries.

[0079] The field region pre-routing module is used for multi-region worksheets. It first selects potentially relevant region identifiers using the boundaries, sampling summaries, and field descriptions of each region, then performs cell-by-cell candidate selection only on the selected regions. The region summary simultaneously extracts information from the beginning, middle, and end of the region, avoiding missing later features by only sampling the prefix. If no field candidate is found in the pre-routing region, the remaining regions are automatically scanned.

[0080] The constrained candidate selection module provides the semantic model with field keys, field descriptions, candidate cell identifiers, coordinates, values, and context within the same region. However, the model is required to return only candidate identifiers and is not allowed to directly generate final values ​​or free coordinates. The server performs type validation, field whitelist validation, candidate identifier existence validation, and label value exclusion validation on the returned structure. Only when all validations pass will the original values ​​and coordinates be read from the local mapping table. Results that clearly select the field label itself rather than the field value are rejected.

[0081] The cross-batch candidate verification module is used when multiple candidates for the same field are obtained in different regional batches. It generates cross-batch candidate identifiers for each candidate, combines the candidate value, coordinates, and adjacent context into a smaller verification set, and then requests the semantic model to select one of the identifiers. The final selection still needs to be verified by the server's candidate whitelist; if the verification fails, the model's free text is not used to replace the real candidate.

[0082] The hierarchical header and structural boundary recognition module is used to calculate precise, regular, fuzzy, and hierarchical path matching information for candidate headers. The hierarchical path collects parent headers of a limited depth along the row or column direction from the leaf-level headers, requiring the expected parent headers to appear in sequence, thereby distinguishing leaf-level headers with the same name under different business groups. When extracting continuous data, it combines signals such as multiple label bands, merged cell channel patterns, record band types, and target columns or rows being taken over by other structures to identify boundaries; repeated headers and group titles are treated as skippable structures and are not directly treated as data or termination boundaries.

[0083] The result merging and rollback module is used to directly return values ​​and coordinates when the region selection fully matches all requested fields, skipping the full-text block call. When only a partial match is made, confirmed fields are retained, and block extraction, result merging, and coordinate positioning are performed on the missing fields. If semantic coordinate disambiguation fails, returns an invalid value, or the model service malfunctions, rollback is performed using unique value coordinates or existing deterministic coordinate results. Finally, merging is done on a field key basis, prioritizing valid candidates for intelligent paths, and unconfirmed fields do not overwrite existing non-empty results.

[0084] Since the electronic device described in this embodiment is an electronic device used to implement the information processing method in the embodiments of the present invention, those skilled in the art can understand the specific implementation methods and various variations of the electronic device in this embodiment based on the information processing method described in the embodiments of the present invention. Therefore, how the electronic device implements the method in the embodiments of the present invention will not be described in detail here. Any electronic device used by those skilled in the art to implement the information processing method in the embodiments of the present invention falls within the scope of protection of the present invention.

[0085] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0086] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0087] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0088] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0089] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0090] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for structure-aware semantic localization and traceable data extraction for heterogeneous spreadsheets, characterized in that, include: Obtain the tab data that meets the tab selection criteria in the target workbook, and partition the tab data according to the data row continuity, blank interval threshold and area capacity limit to obtain several data areas; The data regions to be processed are filtered based on the region summary information and field description information, and candidate identifier records are generated for the selectable cells in the data regions to be processed. The candidate identifier records include location information, content information and context information. The data regions to be processed include several selectable cells. Based on the candidate identifier records, perform field semantic matching and legality verification on the corresponding selectable cells, summarize the matching and verification results, and determine the final field candidate identifiers for each field. Based on the final field candidate identifier of each field, the corresponding field value and position information are obtained from the corresponding selectable cell mapping, and the extraction mode of each field is determined. The extraction mode includes direct extraction mode or block extraction mode. The extraction results of each extraction mode are located and summarized by coordinates, and the field extraction results are output. The field extraction results include at least the field values ​​and coordinate information.

2. The method for structure-aware semantic localization and traceable data extraction for heterogeneous spreadsheets as described in claim 1, characterized in that, Retrieve tab data from the target workbook that meets the tab selection criteria, and partition the tab data according to data row continuity, blank interval threshold, and area capacity limit to obtain several data areas, including: Receive the workbook identifier, tab selection criteria, and field description information, and load the corresponding tab data according to the tab selection criteria; Retrieve the content of non-empty cells, cell row and column positions, and anchor point relationships of merged cells corresponding to the tab data; When partitioning the tab data, an initial data region is formed based on consecutive data rows; The initial data region is segmented when consecutive blank rows exceed the blank interval threshold. When the area capacity exceeds the area capacity limit, the data is further divided to obtain several data areas; Configure overlapping rows and inherited header contexts for adjacent data areas.

3. The method for structure-aware semantic localization and traceable data extraction for heterogeneous spreadsheets as described in claim 1, characterized in that, The data regions to be processed are filtered based on the region summary information and field description information, and candidate identifier records are generated for the selectable cells in the data regions to be processed, including: The regional boundaries and distributed sampling content of each data region are determined based on the regional summary information. When the number of regions exceeds the preset number, filter out the data regions to be processed from the data regions based on the field description information; Generate candidate identifier records for selectable cells in the data range to be processed.

4. The method for structure-aware semantic localization and traceable data extraction for heterogeneous spreadsheets as described in claim 1, characterized in that, Based on the candidate identifier records, the corresponding selectable cells are subjected to field semantic matching, legality verification, and the matching verification results are summarized to determine the final field candidate identifiers for each field, including: Based on the field keys and candidate identifiers in the candidate identifier records, perform field semantic matching on the corresponding selectable cells; The validity of the matching results is verified based on the validity of the candidate identifier, the type of the field key, and the format of the candidate content. Summarize the matching and verification results for each data region to be processed; Based on the uniqueness and validity of the candidate identifiers corresponding to each field, the final candidate identifiers for each field are determined.

5. The method for structure-aware semantic localization and traceable data extraction for heterogeneous spreadsheets as described in claim 4, characterized in that, Based on the uniqueness and validity of the candidate identifiers corresponding to each field, the final candidate identifiers for each field are determined, including: When a field corresponds to a unique and valid candidate identifier, that candidate identifier is taken as the final candidate identifier for that field. When a field corresponds to multiple valid candidate identifiers, a review set is constructed, and the final candidate identifier of the field is re-determined based on the constraints.

6. The method for structure-aware semantic localization and traceable data extraction for heterogeneous spreadsheets as described in claim 1, characterized in that, Determine the extraction mode for each field, including: When all fields have candidate identifiers for the final field, the direct extraction mode is used; When at least one field does not correspond to the final field candidate identifier, the uncorresponding field is extracted in a block extraction mode, while the remaining fields are extracted directly.

7. The method for structure-aware semantic localization and traceable data extraction for heterogeneous spreadsheets as described in claim 1, characterized in that, The extraction results from each extraction mode are processed for coordinate positioning and summarization, and the field extraction results are output. The field extraction results should at least include field values ​​and coordinate information, including: When a unique data value is matched, the corresponding coordinate information is determined; When multiple identical data values ​​are matched, coordinate information is determined based on row-level relationships, column-level relationships, proximity context information, and field description information. The location results corresponding to each extraction mode are summarized and processed, and the field extraction results are output.

8. A structure-aware semantic localization and traceable data extraction device for heterogeneous spreadsheets, characterized in that, include: The acquisition module is used to acquire tab data that meets the tab selection criteria in the target workbook, and to partition the tab data according to the data row continuity, blank interval threshold and area capacity limit to obtain several data areas. The cell module is used to filter the data region to be processed based on the region summary information and field description information, and generate candidate identifier records for the selectable cells in the data region to be processed. The candidate identifier records include location information, content information and context information. The data region to be processed includes several selectable cells. The identifier determination module is used to perform field semantic matching, legality verification, and matching verification result summary on the corresponding selectable cells based on the candidate identifier records, and to determine the final field candidate identifier for each field. The extraction module is used to obtain the corresponding field value and position information from the corresponding selectable cell mapping based on the final field candidate identifier of each field, and to determine the extraction mode of each field. The extraction mode includes direct extraction mode or block extraction mode. The summary output module is used to perform coordinate positioning and summary processing on the extraction results of each extraction mode, and output the field extraction results. The field extraction results must include at least the field value and coordinate information.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute a structure-aware semantic localization and traceable data extraction method for heterogeneous spreadsheets as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to execute a structure-aware semantic localization and traceable data extraction method for heterogeneous spreadsheets as described in any one of claims 1 to 7.

Images