Table analysis method and related device

Abstract

Embodiments of the present application disclose a table analysis method and related equipment, which are used for identifying table boundaries in an electronic table and splitting and independently analyzing a worksheet with a complex or mixed layout, so as to efficiently and accurately convert table data into structured data that can be calculated and stored. The method comprises: traversing cells of each worksheet in a target electronic table file to encode the cells according to content types of the cells, and generating a feature matrix used for representing a layout of the worksheet; determining, according to the feature matrix, a plurality of candidate split points on the worksheet in the target electronic table file; dividing the worksheet into a plurality of table blocks based on the candidate split points; and respectively analyzing cells in the plurality of table blocks to generate structured data corresponding to each table block.

Description

Technical Field

[0001] This application relates to the field of data processing, and more particularly to table parsing methods and related equipment. Background Technology

[0002] In daily business operations and data analysis, it is often necessary to extract data from complex and irregularly formatted spreadsheets and transform it into calculable and storable structured information. Therefore, automating the identification and extraction of such spreadsheet data with high quality is key to improving work efficiency and achieving business digitization. Currently, one solution is to leverage artificial intelligence technologies, such as the understanding capabilities of multimodal large models, to process these unstructured spreadsheet files. The multimodal large model first converts the target spreadsheet file into a visually recognizable format such as an image or PDF. Then, by designing specific prompts, it guides the multimodal large model to visually recognize and understand the content within the image, ultimately outputting structured data directly from the multimodal large model.

[0003] However, when the aforementioned existing technologies understand the content of tables, they need to process a large amount of visual and textual information simultaneously within a limited context. When the data size of a single worksheet of a table is large or the data density is high, the model is prone to improper information selection, omission of local content, or inconsistency in the parsing results during the processing. This leads to instability in the table data parsing process and makes it difficult to continuously output accurate and reliable structured data for spreadsheets with many rows and dense data content. Summary of the Invention

[0004] Based on the above problems, this application provides a table parsing method and related equipment, aiming to solve the problems of unstable parsing, incomplete or inaccurate data extraction caused by context window limitations and illusion problems when existing visual recognition-based multimodal models parse complex or large-scale spreadsheets.

[0005] In a first aspect, embodiments of this application provide a table parsing method, including:

[0006] Traverse the cells of each worksheet in the target spreadsheet file to encode the cells according to their content type, and generate a feature matrix to characterize the worksheet layout;

[0007] Based on the feature matrix, multiple candidate splitting points on the worksheets in the target spreadsheet file are determined;

[0008] Based on the candidate splitting points, the worksheet is divided into multiple table blocks;

[0009] The cells within each of the multiple table blocks are parsed to generate structured data corresponding to each table block.

[0010] Secondly, embodiments of this application also provide a table parsing apparatus, including:

[0011] The encoding unit is used to traverse the cells of each worksheet in the target spreadsheet file to encode the cells according to the content type of the cells, and generate a feature matrix to characterize the worksheet layout.

[0012] The determining unit is used to determine multiple candidate splitting points on the worksheets of the target spreadsheet file based on the feature matrix.

[0013] A partitioning unit is used to divide the worksheet into multiple table blocks based on the candidate partitioning points;

[0014] The parsing unit is used to parse the cells in the multiple table blocks respectively, and generate structured data corresponding to each table block.

[0015] Thirdly, embodiments of this application also provide a computer device, including:

[0016] Central processing unit, memory, input / output interfaces;

[0017] The memory is either a short-term storage memory or a persistent storage memory;

[0018] The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the table parsing method described in the first aspect of the embodiments of this application or any specific implementation thereof.

[0019] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is used to implement the table parsing method described in the first aspect or any specific implementation of the first aspect of the embodiments of this application.

[0020] Fifthly, embodiments of this application also provide a computer program product having a computer program / instruction stored thereon, which, when executed by a processor, is used to implement the table parsing method described in the first aspect or any specific implementation of the first aspect of the embodiments of this application.

[0021] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0022] This application embodiment generates a feature matrix by traversing worksheet cells and encoding them based on the content type of the cells. Furthermore, multiple candidate splitting points are determined based on the feature matrix, enabling the rapid location of potential table boundaries within the worksheet and thus identifying multiple table regions mixed within the same worksheet. Since the determination of candidate splitting points primarily depends on the content type of the cells in the worksheet, the parsing process does not require overall semantic inference of the content within a large number of cells. Therefore, it can still execute stably even with large or dense worksheet data, avoiding processing failures or inconsistent results caused by excessive input information in existing technologies. Furthermore, dividing the worksheet into multiple table blocks based on candidate splitting points reduces the number of cells and structural complexity required for each parsing iteration, making the input scale of each parsing more controllable and reducing the probability of partial omissions or inconsistencies when dealing with large amounts of data. On the other hand, even if a table block has poor parsing results due to abnormal formatting, it will not affect the parsing results of other table blocks, making the fault tolerance during table parsing stronger. Finally, the multiple table blocks are parsed separately to generate corresponding structured data, so that the output results can maintain a one-to-one correspondence with different table areas in the original worksheet, thereby improving the accuracy and reliability of the structured data and effectively alleviating the problems of information omission, inconsistent results and unstable parsing that are prone to occur when the existing multimodal large model scheme processes large-scale worksheets. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0024] Figure 1 A schematic diagram of a system architecture provided for an embodiment of this application;

[0025] Figure 2 This is a schematic flowchart of a table parsing method provided in an embodiment of this application;

[0026] Figure 3 A timing diagram for sampling and parsing path determination of table block structure features provided in this application embodiment;

[0027] Figure 4 This is a schematic diagram of a table parsing device provided in an embodiment of this application;

[0028] Figure 5 This is a schematic diagram of a computer device structure provided in an embodiment of this application. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] The method provided in this application embodiment can be applied to, for example, Figure 1 In the system architecture shown, terminal 102 communicates with server 101 via a network. Data storage system 100 is used to store the spreadsheet files that server 101 needs to process during the table parsing process, the intermediate results generated during the parsing process, and the structured data generated during parsing. Data storage system 100 can be integrated and deployed on server 101, or it can be deployed in the cloud or on other network servers to meet data processing scenarios with different scales and performance requirements.

[0031] Terminal 102 can act as an uploader of spreadsheet files or a receiver of parsing results. It submits the target spreadsheet file to be parsed to server 101 via a client application and receives the parsing results returned by server 101. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices, including smartwatches, smart bracelets, or head-mounted devices. Here, the client application can be a file management interface, a data analysis platform, a business system front-end page, or other applications that support spreadsheet uploading and result display.

[0032] Server 101 is deployed with a server-side application for executing the table parsing method provided in this application embodiment. Server 101 can be implemented as a standalone server or as a server cluster composed of multiple servers to support high concurrency or large-scale table processing needs. After receiving the target spreadsheet file uploaded by terminal 102, server 101 parses the spreadsheet file, including but not limited to traversing cells of the worksheet, constructing a feature matrix, identifying candidate splitting points, splitting table blocks, determining the structure type of table blocks, and generating corresponding structured data. Server 101 can also call a preset language model during the parsing process to perform semantic verification or structure type determination on candidate splitting points or table block structures, thereby improving the accuracy and stability of the parsing results.

[0033] Data storage system 100 is used to store the target spreadsheet file, feature information generated during parsing, candidate splitting point information, table block boundary information, and the table data files and metadata corresponding to the finally generated structured data or complex table blocks. Data storage system 100 can be a standalone physical storage server, a distributed storage system composed of multiple storage nodes, or a cloud storage system providing object storage, relational databases, cache databases, cloud databases, or cloud storage services. For example, data storage system 100 can be implemented using a distributed file system, NoSQL database, relational database, or cloud storage service. Its specific implementation can be flexibly configured according to the actual deployment environment and business needs, but is not limited to the examples mentioned above.

[0034] In one optional implementation, after completing the parsing of the spreadsheet file, the server 101 can write the generated structured data into the data storage system 100 and return the access address or feature information of the parsing result to the terminal 102 so that the terminal 102 can display, download, or further analyze the parsing result. In another implementation, the server 101 can also directly send the parsed structured data or spreadsheet data file to the terminal 102 via the network for local storage or subsequent processing by the terminal 102.

[0035] It should be noted that the method provided in this application embodiment can be executed independently on the server 101 side, independently on the terminal 102 side, or jointly by the terminal 102 and the server 101. For example, in scenarios with limited computing resources or high real-time requirements, the terminal 102 can perform partial cell traversal or feature information extraction operations and send the intermediate results to the server 101; when processing large-scale, structurally complex spreadsheet files, the server 101 can centrally complete the entire parsing process to improve overall processing efficiency and stability. Figure 1 The system architecture shown is only used to illustrate the technical principles of the embodiments of this application and does not constitute a limitation on the scope of protection of this application. Equivalent changes or substitutions made to the system architecture by those skilled in the art without departing from the technical concept of this application shall fall within the scope of protection of this application.

[0036] In daily business operations and data analysis, it is often necessary to extract data from complex and irregularly formatted spreadsheets and transform it into calculable and storable structured information. Therefore, automating the identification and extraction of such spreadsheet data with high quality is key to improving work efficiency and achieving business digitization. Currently, one solution is to leverage artificial intelligence technologies, such as the understanding capabilities of multimodal large models, to process these unstructured spreadsheet files. The multimodal large model first converts the target spreadsheet file into a visually recognizable format such as an image or PDF. Then, by designing specific prompts, it guides the multimodal large model to visually recognize and understand the content within the image, ultimately outputting structured data directly from the multimodal large model.

[0037] However, when understanding the content of a table, the aforementioned existing technologies need to process a large amount of visual and textual information simultaneously within a limited context. When the data size or density of a single worksheet is large, the inherent limitation of the multimodal large model context window length often makes it impossible to process all the content completely. This can lead to improper information selection, omission of local content, or inconsistencies in the parsing results during the processing, ultimately resulting in an unstable table data parsing process and difficulty in continuously outputting accurate and reliable structured data for spreadsheets with many rows and dense data content.

[0038] Based on the above problems, this application provides a table parsing method, such as... Figure 2 As shown, the method includes the following steps S201-S204.

[0039] S201: Traverse the cells of each worksheet in the target spreadsheet file to encode the cells according to their content type, generating a feature matrix that characterizes the worksheet layout.

[0040] In this embodiment, each worksheet in the target spreadsheet file is traversed sequentially to determine the actual row and column range used by each worksheet. Within this range, cell attribute information is read cell by cell. These attributes include, but are not limited to, cell content value, cell content type, whether the cell is empty, whether the cell is in date format, whether the cell is in numeric format, whether the cell is a formula, whether the cell is text, and whether the cell is a merged cell. Based on a pre-defined cell content type mapping rule, different content types are mapped to different discrete code values. The same content type corresponds to the same code value, and different content types correspond to different code values. In practice, considering processing performance, only type information related to the table structure can be retained, while the specific data content itself can be ignored to reduce subsequent processing complexity. Subsequently, according to the cell's row and column position in the worksheet, the corresponding code value is filled into a two-dimensional array that corresponds one-to-one with the worksheet's row and column structure, forming a feature matrix that characterizes the overall layout of the worksheet. Each element in the feature matrix corresponds spatially to a cell in the worksheet.

[0041] S202: Based on the feature matrix, determine multiple candidate split points on the worksheets in the target spreadsheet file.

[0042] In this embodiment, the feature matrix can first be statistically analyzed along the row direction to calculate the proportion of non-empty type codes or the number of non-zero elements in each row, thereby reflecting whether the data row is a data-dense row. When the proportion of non-empty types in several consecutive rows continuously decreases or approaches zero, the corresponding row position can be marked as a candidate split point in the row direction. Similarly, the feature matrix can be statistically analyzed along the column direction to calculate the proportion of non-empty type codes or the number of non-zero elements in each column. When the proportion of non-empty types in several consecutive columns continuously decreases or approaches zero, the corresponding column position can be marked as a candidate split point in the column direction. In another feasible implementation, the connectivity relationship between adjacent cells in the feature matrix can also be combined to determine whether there is an interruption in the structurally continuous region in the row or column direction, thereby further assisting in determining the position of the candidate split point. Then, the row index and column index that meet the preset judgment conditions are respectively used as candidate horizontal split points and candidate vertical split points to obtain multiple candidate split points for subsequent worksheet partitioning.

[0043] S203: Divide the worksheet into multiple table blocks based on candidate split points.

[0044] In this embodiment, candidate horizontal split points are first sorted by row index in ascending order, and candidate vertical split points are sorted by column index in ascending order. Then, the entire worksheet is divided into several rectangular regions using the row intervals between adjacent candidate horizontal split points and the column intervals between adjacent candidate vertical split points as boundaries. For each rectangular region, it can be further determined whether there is at least one cell with a non-empty type code inside the rectangular region. If so, the rectangular region is determined as a table block, and the boundary information of the table block, such as the row start position, row end position, column start position, and column end position, is recorded. If the rectangular region is entirely composed of empty type codes, the rectangular region can be considered an invalid region and will not be processed as a table block.

[0045] S204: Parse the cells in multiple table blocks separately to generate structured data for each table block.

[0046] In this embodiment, the parsing process includes at least header area identification, data area location, and cell content extraction. For each table block, the original data content and position of the cell are first read within the boundary of the table block. Combining the style characteristics of each row and column within the table block, the row area where the header is located and the row area where the data is located are determined. The name or identifier of each field is determined based on the text content in the header row, and the data type of the corresponding field is determined based on the content type of each column in the data row. The parsed header fields are aligned with the data row content, and each data row is mapped to a data record with the field as the key and the cell content as the value, thus forming structured data. For typeable content such as numerical values ​​and date / time, type conversion and cleaning can be performed according to preset data normalization rules to unify similar data under different representations. For example, numerical representations containing thousands separators are converted to standard numerical representations in the data normalization rules, different date formats are unified into the same date representation method, and unparseable outliers are recorded with preset null value identifiers or outlier identifiers. In the embodiments of this application, structured data can be stored in the form of key-value pair sets, record sets, or tables to meet different application requirements, so as to facilitate subsequent writing to the database, conversion to other preset formats that are easy to parse, or access by other business modules.

[0047] In summary, the table parsing method provided in this application first constructs a feature matrix representing the worksheet layout using cell content type encoding, then automatically identifies candidate splitting points based on the feature matrix and divides the worksheet into multiple table blocks, and finally performs structured parsing on each table block. This ultimately achieves automatic identification and separation of multiple table regions from complex worksheets and generates corresponding structured data according to table blocks. Thus, when dealing with complex spreadsheet files containing multiple tables, this application can improve the accuracy of table region identification and the stability of structured parsing, reduce manual intervention and erroneous parsing, and improve the processing efficiency and quality of parsed data.

[0048] In one embodiment, determining multiple candidate splitting points on a worksheet in a target spreadsheet file based on a feature matrix includes: determining empty rows and columns with a non-empty cell percentage lower than a first threshold based on the percentage of non-empty cells in each row and column of the feature matrix; and determining candidate splitting points on the worksheet based on the position of the empty rows and columns in the feature matrix.

[0049] Statistical calculations are performed on the row and column directions of the feature matrix obtained in the previous steps to obtain the percentage of non-empty cells in each row and column. It can be understood that each matrix element in the feature matrix corresponds one-to-one with a cell in the worksheet, and the value of the matrix element represents the content type of the corresponding cell; where the code value representing an empty cell is a preset null value, and the code value representing a non-empty cell is another code value besides the preset null value. Therefore, for any row, the percentage of non-empty cells in that row can be obtained by counting the number of matrix elements in that row whose code value is not equal to the preset null value and dividing that number by the total number of matrix elements in that row; similarly, for any column, the percentage of non-empty cells in that column can be obtained by counting the number of matrix elements in that column whose code value is not equal to the preset null value and dividing that number by the total number of matrix elements in that column.

[0050] Furthermore, the percentage of non-empty cells is compared with a first threshold. The first threshold is a pre-set percentage threshold used to characterize whether a row or column can be structurally considered a blank dividing line. When the percentage of non-empty cells in a row is lower than the first threshold, the row is determined to be an empty row. When the percentage of non-empty cells in a column is lower than the first threshold, the column is determined to be an empty column, thus obtaining a set of empty rows and a set of empty columns. For example, when the first threshold is set to 5%, if a row has only 2 non-empty cells out of 100 cells, the percentage of non-empty cells in that row is 2%, which is less than the 5% set by the first threshold; therefore, that row can be determined to be an empty row. Similarly, if the percentage of non-empty cells in a column is lower than the first threshold, that column is determined to be an empty column.

[0051] Since the feature matrix and the worksheet maintain consistency in row and column dimensions, the row index of an empty row in the feature matrix can be directly mapped to the corresponding row position in the worksheet, and the column index of an empty column in the feature matrix can be directly mapped to the corresponding column position in the worksheet. Based on this, the row positions corresponding to empty rows are used as candidate split points in the horizontal direction, and the column positions corresponding to empty columns are used as candidate split points in the vertical direction, thus forming candidate split points in the worksheet to indicate table boundary divisions. It should be noted that candidate split points only indicate locations where table boundaries may exist in the structural distribution and are not directly equivalent to the final table segmentation boundaries. In some other feasible implementations, additional verification steps can be used to confirm or withdraw candidate split points to improve the accuracy of the splitting results.

[0052] This application embodiment analyzes the proportion of non-empty cells in the feature matrix, which can quickly identify empty rows and columns in the worksheet that may be used to separate different table areas without relying on complex semantic understanding, and further determine candidate splitting points accordingly, which helps to reduce the risk of incorrectly splitting the worksheet.

[0053] In one embodiment, before dividing the worksheet into multiple table blocks based on candidate splitting points, the method further includes: for each candidate splitting point, extracting the cell content of a first region corresponding to the candidate splitting point; wherein, the first region is a local region formed by extending a preset number of rows or columns to the adjacent sides of the candidate splitting point with the candidate splitting point as the center row or column; inputting the cell content of the first region into a preset language model for semantic verification, and obtaining the verification result output by the language model; in response to the verification result of the language model indicating that the cell content of the first region belongs to the same table, the candidate splitting point is withdrawn.

[0054] Before splitting the worksheet based on candidate splitting points, this embodiment extracts the cell content of a first region corresponding to each candidate splitting point. Specifically, the candidate splitting point can be a row-oriented splitting point or a column-oriented splitting point. When the candidate splitting point is in the row direction, the row containing the candidate splitting point is determined as the center row; when the candidate splitting point is in the column direction, the column containing the candidate splitting point is determined as the center column. The first region is a local area formed by extending a preset number of rows or columns to the adjacent sides based on the center row or center column. Here, the preset number of extensions is used to define the range of the local area. For example, when the candidate splitting point is the 10th row and the preset number is 2 rows, the first region may include the cell content of rows 8 to 12; when the candidate splitting point is the 6th column and the preset number is 3 columns, the first region may include the cell content of columns 3 to 9. Furthermore, to ensure that the extracted content is consistent with the actual boundaries of the worksheet, when the extension near the center row or column exceeds the effective range of the worksheet, the first region can be truncated to the effective row and column range of the worksheet.

[0055] The cell content of the first region is then input into a pre-defined language model for semantic verification. This pre-defined language model can be a predetermined natural language processing model used to understand the input text and the relative positional relationships between texts, and output the verification results. To enable the language model to accurately understand the structural relationships of the first region within a limited input scale, the cell content in the first region can be serialized according to its original row and column order. During serialization, structural information related to the rationality of the segmentation is preserved. This structural information may include, but is not limited to, cell row numbers, column numbers, relative positions, whether they are empty, and the coverage of merged cells. This allows the language model to identify whether there are semantic and structural connections between the content on both sides of the candidate segmentation point in terms of header continuity, field consistency, and data row continuity. The obtained verification results are used to characterize whether the candidate segmentation point incorrectly segments the same table, and the verification results may include binary judgment information and corresponding confidence information. The confidence information characterizes the credibility of the binary judgment information.

[0056] In response to the validation results output by the language model, when the validation result indicates that the cell content of the first region belongs to the same table, it means that the content on both sides of the candidate split point has a semantic continuity or subordinate relationship. The candidate split point is more likely to correspond to the layout whitespace within the table, the separation between the table header and data area, or the structural line break within the same table, rather than the true boundary used to separate two independent tables. Therefore, the candidate split point should be revoked. Here, the revocation process can be manifested as deleting the candidate split point from the candidate split point set, or marking the candidate split point as an unsplitable point.

[0057] Before splitting the worksheet, this embodiment first constructs a local first region around the candidate split point and performs semantic verification using a preset language model. It uses the continuity of the content on both sides of the candidate split point to identify and remove candidate split points that may lead to incorrect worksheet splitting, thereby reducing the probability of incorrectly splitting the same table.

[0058] In one embodiment, after dividing the worksheet into multiple table blocks based on candidate splitting points, the method further includes: identifying free text cells located outside the outer border of the table blocks, using the outer frame of the table blocks as the outer boundary of the table blocks; calculating the distance between the free text cells and each table block based on the row and column position relationship between the free text cells and each table block in the worksheet; and associating the free text cells with the nearest table block to treat the free text cells as attached text information of the table blocks.

[0059] It is understood that each table block corresponds to a defined range of rows and columns within the worksheet. This range can be determined by the minimum row number, maximum row number, minimum column number, and maximum column number of the cells within the table block, forming the outer border surrounding the table block. Here, the outer border serves as the outer boundary of the table block. Based on the outer boundary of the worksheet, the cells in the worksheet are traversed, and cells located outside the outer boundary of any table block with text content are selected as free text cells. It should be noted that free text cells do not belong to the main data area of ​​any table block. However, in actual business documents, free text cells often appear as titles, notes, unit descriptions, or author information. Therefore, to avoid misidentifying the header text or data text inside the table block as free text cells, this embodiment primarily filters within the range outside the outer boundary, thereby ensuring that the identified free text cells are spatially separated from the main area of ​​the table block.

[0060] After locating the free text cell, first determine its row and column numbers within the worksheet. Then, determine the row start, row end, column start, and column end positions corresponding to the outer boundary of each table block. Next, calculate the row and column distances from the free text cell to the outer boundary of the table block based on the row and column coordinates. Specifically, if the row number of the free text cell is between the row start and row end positions of the outer boundary of the table block, the row distance is 0; if the row number is less than the row start position, the row distance is the difference between the row start position and the row number; if the row number is greater than the row end position, the row distance is the difference between the row number and the row end position. Similarly, the column distance is calculated in the same way as the row distance. Furthermore, the row distance and column distance are combined according to a preset distance composition rule to obtain the distance between the free text cell and the table block. The distance composition rule can be to sum the row distance and column distance or take the larger value of the two. In this way, the distance between the free text cell and each table block can be obtained using only the row and column coordinate relationship of the worksheet without relying on pixel-level coordinates.

[0061] For each free text cell, based on the obtained distance results, the target table block closest to the free text cell is determined among multiple table blocks, and an association is established between the free text cell and the target table block. This association characterizes that the free text cell semantically belongs to the target table block, and this association can be implemented by recording mapping relationships, such as recording the coordinate information of the free text cell, its text content, and the identification information of the target table block in metadata. This allows the free text cell to be output or stored as supplementary text information of the target table block. Furthermore, if the free text cell is located above the target table block and is closest to its outer boundary, the free text cell corresponds to the title information of that table block; similarly, if the free text cell is located below the table block and is closest to its outer boundary, it often corresponds to the notes or tabulation information of that table block. Through the above association processing, this information can be retained along with the main data of the table block.

[0062] After the table block is split, this application embodiment identifies the free text cells outside the table block and performs distance calculation and association based on the row and column position relationship. This allows the contextual information such as titles, notes, and units that were originally scattered outside the table block to be attributed to the corresponding table block. This completes the contextual semantic information of the table without changing the main data of the table block, reducing the problems of ambiguity in subsequent structured parsing or incomplete business meaning caused by missing context.

[0063] In one embodiment, the cells within multiple table blocks are parsed separately to generate structured data corresponding to each table block. This includes: extracting feature information to characterize the structural features of each table block; the structural features include at least the row distribution features, column distribution features, and style features of the table block; sampling the data in the table block to obtain sampled data; inputting the feature information and sampled data into a preset language model to obtain the structure type determination result of the table block; and parsing the data in the table block according to the structure type determination result of the table block to generate the corresponding structured data.

[0064] In this embodiment, a table block is a rectangular area defined by its outer boundary within a worksheet. When reading cells within this rectangular area, not only is the original content of the cells obtained, but also attribute information characterizing the structure and style of the table block, thus forming feature information. Specifically, row distribution features characterize the non-empty distribution and content type distribution of the table block along the row direction; column distribution features characterize the non-empty distribution and content type distribution of the table block along the column direction; and style features characterize the style distribution and hierarchy of cells within the table block. These can include style features such as bolding ratio, indentation hierarchy, border distribution, alignment distribution, and the occurrence of merged cells. To ensure that feature information is comparable across different table blocks, it can be generated in a unified manner. For example, using a preset set of content types as the classification dimension, the proportion of different content types in each row or column is statistically analyzed, and the resulting row distribution features, column distribution features, and style statistics are combined to form feature information. This allows the feature information to objectively reflect the structural regularity and complexity of the table block without relying on specific text semantics.

[0065] Data in a table block is sampled, and a portion of row and column data is selected as sampled data using a preset sampling strategy. The preset sampling strategy constrains the sampling range and quantity to avoid extracting all cells when the number of rows or columns in the table block is large, which would result in excessive data volume and affect the table parsing process. In this embodiment, the sampled data may include row-based sampled data and column-based sampled data. The cell content in the row-based sampled data and the cell content in the column-based sampled data retain their relative row and column positions within the table block during extraction, facilitating model recognition of table structures such as headers, dimension fields, and measure fields. In other words, for a table block containing a large number of detailed data rows, several rows can be selected downwards from the starting row containing the header as row-based sampled data, and several columns can be selected to the right from the starting column as column-based sampled data.

[0066] After obtaining the sampled data and feature information, the feature information and sampled data are input into a preset language model to obtain the structure type determination result of the table block. Specifically, the preset language model is a pre-determined language model that can combine and analyze the input feature information and sampled data content and output the structure type determination result. During the model input stage, the feature information can be expressed in a structured summary form, while the sampled data can be expressed in a serialized form that preserves the row and column order, enabling the language model to understand the relative layout relationship between sample cells. In this embodiment, the structure type determination result is used to characterize the preset structure type to which the table block belongs. The preset structure types include at least a first type table and a second type table. The first type table can correspond to a data table with a regular structure that is suitable for structured storage; the second type table can correspond to a complex table with multi-level headers, a perspective layout, many merged cells, or significant changes in row and column patterns. The structure type determination result output by the model can include a type identifier and confidence information corresponding to the type identifier, so that the parsing step can select a matching parsing method based on the structure type determination result, and, when necessary, use confidence information to improve the controllability and stability of the determination process.

[0067] Finally, based on the structure type determination result of the table block, the data in the table block is parsed and corresponding structured data is generated. Specifically, when the structure type determination result indicates that the table block belongs to the first type of table, the parsing method may include determining the header area, extracting field names, determining field data types, parsing data rows one by one, cleaning and type-converting cell content, and organizing the parsed data into multiple records according to the preset field order, thereby forming structured data that can be written to a database or output in a preset format; when the structure type determination result indicates that the table block belongs to the second type of table, the parsing method may include maintaining the original layout boundaries of the table block and extracting its data area, while generating structured data to describe the structure of the table block.

[0068] Before parsing each table block, this embodiment first extracts feature information that can characterize the complexity of the structure, and uses a preset language model to determine the structure type based on the feature information and sampled data, thereby avoiding the instability and high cost of semantic understanding of large-scale table blocks.

[0069] In one embodiment, sampling data in a table block to obtain sampled data includes: starting from the first row of the table block, selecting multiple rows sequentially as candidate sampled rows, and determining whether to continue sampling downwards based on the similarity of the row distribution features corresponding to any two adjacent candidate sampled rows; stopping downward sampling when the similarity of several consecutive pairs of candidate sampled rows is greater than a second threshold, thus obtaining row sampled data in units of rows; starting from the first column of the table block, selecting multiple columns sequentially as candidate sampled columns, and determining whether to continue sampling to the right based on the similarity of the column distribution features corresponding to any two adjacent candidate sampled columns, and stopping sampling to the right when the similarity of several consecutive pairs of candidate sampled columns is greater than a third threshold, thus obtaining column sampled data in units of columns; and limiting the amount of data in both the row sampled data and the column sampled data so that the number of sampled rows in the row sampled data does not exceed a fourth threshold and the number of sampled columns in the column sampled data does not exceed a fifth threshold.

[0070] During data sampling, starting from the first row of a table block, since the table block is a rectangular area with defined boundaries, the first row usually contains header information or data start information and is generally structurally representative; therefore, the first row is used as the initial candidate sampling row. Based on this, subsequent rows are selected sequentially downwards according to their row numbers as new candidate sampling rows. For any two adjacent candidate sampling rows, their corresponding row distribution features are extracted. These row distribution features characterize the distribution of cells with different content types, the proportion of non-empty cells, and the style distribution within the candidate sampling row, among other structural information. Subsequently, according to a preset similarity calculation rule, the similarity of the row distribution features of adjacent rows is calculated. Specifically, this can be done, for example, using cosine similarity. For instance, if the feature information of two candidate sampling rows or columns is represented as vectors A and B, the cosine similarity between the two candidate sampling rows or columns is obtained by calculating the dot product of vectors A and B and dividing by the product of the magnitudes of vectors A and B.

[0071] When the similarity between adjacent candidate sample rows is lower than the second threshold, it indicates that the current row may still be in the header area or a structural change area, and further sampling is needed. When the similarity of the row distribution characteristics of several consecutive pairs of candidate sample rows is greater than the second threshold, further sampling stops to obtain row-based sampled data. Specifically, the second threshold is a pre-set similarity threshold, such as 0.9, used to determine whether adjacent rows have a high correlation in structural distribution. During the downward sampling process, when a predetermined number of consecutive adjacent candidate sample rows meet the condition that their similarity is greater than the second threshold, it can be considered that the structure of the table block in the row direction is stable, and continuing to sample more rows has little gain for structure determination, so the sampling process in the row direction is terminated. Finally, all candidate sample rows selected from the first row to the stop sampling position are used as row sampled data, thus ensuring the representativeness of the sampled data while avoiding redundant sampling of a large number of data rows with repetitive structures in the table block.

[0072] Similarly, starting from the first column of the table block, multiple columns are selected sequentially as candidate sampling columns, and the similarity of column distribution features between adjacent candidate sampling columns determines whether to continue sampling to the right. Specifically, the first column usually contains row identifiers, dimension fields, or other key descriptive fields, reflecting the initial structure of the table block in the column direction, and is therefore used as the initial candidate sampling column. Based on this, subsequent columns are selected to the right in ascending order of column number as new candidate sampling columns. For any two adjacent candidate sampling columns, their corresponding column distribution features are extracted. These column distribution features characterize structural information such as the distribution of cells with different content types, the proportion of non-empty cells, and the style distribution within the column. Subsequently, according to a preset similarity calculation rule, the similarity of the column distribution features of adjacent columns is calculated to determine whether the column structure has changed, thereby determining whether further sampling to the right is necessary.

[0073] When the similarity of the column distribution features of several consecutive pairs of candidate sampling columns is greater than the third threshold, sampling to the right stops to obtain column-based sampling data. The third threshold is a pre-set column direction similarity threshold, such as 0.9, used to determine the consistency of adjacent columns in structural distribution. During the sampling process to the right, when a predetermined number of consecutive adjacent candidate sampling columns meet the condition that their similarity is greater than the third threshold, it can be considered that the structure of the table block in the column direction has become stable. Continuing to sample more columns has limited contribution to the determination of the table structure type, so the sampling process in the column direction is terminated. Finally, all candidate sampling columns selected from the first column to the stop sampling position are used as column sampling data.

[0074] After obtaining the row and column sampled data, data volume limits are applied to both. For row sampled data, the number of sampled rows is limited to no more than a fourth threshold; for column sampled data, the number of sampled columns is limited to no more than a fifth threshold. Both the fourth and fifth thresholds are pre-set maximum quantity thresholds, such as 7. When the number of row sampled data obtained through the aforementioned similarity determination exceeds the fourth threshold or the number of column sampled data exceeds the fifth threshold, truncation can be performed according to preset rules. For example, only the first few rows or columns starting from the first row or column can be retained as the final sampling result, or truncation can be performed based on the corresponding threshold.

[0075] The sampling method based on row and column distribution feature similarity in this application allows the sampling process to stop according to the changes in the table block's own structure. This reduces the amount of data collected while ensuring that the sampled data can represent the table's structural features to a certain extent. This effectively controls the complexity and resource consumption of subsequent language model processing and improves the stability and efficiency of the table parsing process when dealing with large-scale, high-density table blocks.

[0076] Please refer to Figure 3 , Figure 3 This illustrates the process of determining the structure type of a single table block object and selecting the appropriate data processing path accordingly. The following section combines... Figure 3 The specific steps of this embodiment are described below. First, the table block object sends a request to the feature information generator to generate feature information characterizing the row or column structure features within the table block. The feature information may include content type distribution features along the row and column directions, non-empty cell distribution features, and style distribution features, etc. After receiving the request, the feature information generator traverses and analyzes the table block and returns the generated complete feature information to the sampling controller.

[0077] After obtaining complete feature information, the sampling controller enters a sampling control loop to perform sampling operations on the data in the table block. During this loop, the sampling controller calculates the similarity of corresponding feature information between adjacent rows or columns based on the feature information to determine whether the structure of the table block in the row or column direction has become stable. When it is detected that the similarity between adjacent feature values ​​remains at a high level and the similarity is greater than a preset similarity threshold (e.g., 0.9) or the number of samples has reached a preset upper limit rule (e.g., 7 rows), the sampling controller terminates the sampling loop, thereby avoiding redundant sampling of a large number of data rows or columns with repetitive structures.

[0078] After the sampling process is complete, the sampling controller sends the feature information, along with the final row and column sampled data, to the preset language model. Based on the input feature information and sampled data, the preset language model understands and analyzes the overall structure of the current table block and outputs the corresponding structure type determination result. The structure type determination result characterizes whether the table block is suitable for regularized structure parsing, such as whether it belongs to a well-structured detail table or a complex perspective or summary table.

[0079] Subsequently, the preset language model returns the structure type determination result to the decision routing module. Upon receiving the structure type determination result, the decision routing module selects processing path branches for the table block according to a pre-defined processing strategy. When the determination result indicates that the current table block belongs to a structure type suitable for rule-based processing, the decision routing module marks the table block as a table block that can be extracted, transformed, and loaded (Extract, Transform, Load) and performs structured data import processing. When the determination result indicates that the current table block belongs to a table type with a complex structure that is not suitable for direct import, the decision routing module marks the table block as a comma-separated values ​​(CSV) export processing type, used to perform data export and metadata generation operations while maintaining the original structure.

[0080] Specifically, in one embodiment, the data in the table block is parsed according to the structure type determination result of the table block to generate corresponding structured data, including: in response to the determination result indicating that the table block belongs to a first type of table, the table block is determined as a regular table block, and the data in the regular table block is converted according to the field information and data type information corresponding to the regular table block to generate structured data corresponding to the regular table block; in response to the determination result indicating that the table block belongs to a second type of table, the table block is determined as a complex table block, the data of the complex table block is extracted and metadata corresponding to the complex table block is generated.

[0081] When the structure type determination result indicates that the table block belongs to the first type of table, the table block is identified as a regular table block, and rule-based parsing processing is performed based on the structural characteristics of the regular table block. Specifically, in this embodiment, the first type of table is a table with a relatively regular structure, typically having a clear and unambiguous header row, a stable column structure, and a relatively uniform data type distribution. After determining that the table block is a regular table block, the header area is first identified from the table block, and the field information is determined based on the text content in the header area. The field information is used to represent the business meaning of each column. At the same time, combined with the content type distribution of each column cell in the data row of the table block, the data type information corresponding to each field is determined, such as numeric type, date type, or text type. Then, based on the field information and data type information corresponding to the regular table block, the data in the regular table block is subjected to type conversion processing to generate the structured data corresponding to the regular table block. Specifically, each row of data in the rule table block is parsed row by row. The cell content in each row is mapped to the corresponding fields according to the field order, and type conversion operations are performed according to the data type of the corresponding field. For example, numeric content represented in string form is converted to numeric type, text content in date format is converted to a unified date encoding format, or text content containing unit symbols is normalized. Through the above type conversion, the heterogeneous data in the original table is uniformly converted into structured data that conforms to the preset data type constraints, which can be used for subsequent database storage, data analysis, or business calculations.

[0082] When the structure type determination result indicates that the table block belongs to the second type of table, the table block is identified as a complex table block. In this embodiment, the second type of table typically has characteristics such as multi-level headers, pivot structure, many merged cells, or frequent changes in row and column layout. Therefore, the structure of the second type of table is difficult to directly convert into a regularized field structure without losing business semantics. Therefore, after identifying the table block as a complex table block, this embodiment does not perform forced field decomposition and data type regularization on it. Instead, it extracts the complete data area based on the physical boundary of the table block, thereby maintaining the original layout relationship and data integrity of the complex table block. Based on the completion of the data extraction of the complex table block, metadata corresponding to the complex table block is generated to describe the structure and semantics of the complex table block. Specifically, the metadata is used to characterize the structural features and business meaning of the complex table block, including but not limited to the meaning of row dimensions, column dimensions, business interpretation of each metric field, and aggregation or calculation relationships between rows and columns. Metadata can be generated in the form of structured text or tokenized documents and stored together with the data corresponding to complex table blocks. In this way, when accessing complex table block data later, the structural semantics can be understood by combining the metadata, without having to perform unstable parsing of complex table blocks, which would result in inaccurate parsing results.

[0083] In one embodiment, the extraction of data from complex table blocks and the generation of metadata corresponding to the complex table blocks described in the foregoing embodiments includes: extracting data from complex table blocks and exporting it as a table data file; calling a preset language model to analyze the structure of the complex table blocks, generating metadata for at least describing the row dimensions, column dimensions, and measures of the table blocks, and storing the metadata in association with the table data file.

[0084] Since complex table blocks also have row start positions, row end positions, column start positions, and column end positions, the data area of ​​the complex table block can be located in the original worksheet based on the above boundary information, and the cell content and its row and column positions within that area can be read cell by cell. To avoid data loss during parsing due to merged cells, multi-level headers, or pivot structures in complex table blocks, this application does not decompose the data area into fields. Instead, it writes the cell content into the table data file according to the original row and column order of the data area, so that the exported table data file can reflect the original layout of the complex table block. The table data file can be a file in a preset file format, such as CSV or other parsable table file formats. During the export process, when there are merged cells, the coverage area of ​​the merged cells can be recorded according to preset export rules or retained as placeholders.

[0085] Then, a preset language model is invoked to analyze the structure of the complex table block, generating metadata that at least describes the row dimensions, column dimensions, and measures of the complex table block. This metadata is then associated and stored with the table data file, resulting in an integrated output of the data file and structural description. To ensure the preset language model can accurately identify the perspective relationships and multi-level header meanings of the complex table block, the header area of ​​the complex table block, key row and column data samples, and structural information related to merged cells can be used as model input. The structural information at least reflects the header hierarchy, the organization of row and column labels, and the location of the measures. The metadata describes the structural semantics of the complex table block. Row dimensions explain the meaning of categorical or grouped fields expanding along the row direction, column dimensions explain the meaning of categorical or grouped fields expanding along the column direction, and measures explain the summarized, statistical, or calculated indicator fields in the table and their measurement meanings. The metadata can be generated in structured text or markup document format and includes location information matching the table data file, allowing for subsequent comparison and analysis of the corresponding areas in the table data file based on the metadata. The correspondence between associated storage metadata and table data files can be achieved by setting the same table block identifier, file index, or reference address for both, enabling the corresponding metadata to be retrieved synchronously when accessing the table data file. In summary, when dealing with complex table blocks that are structurally complex and difficult to directly regularize into the database, this embodiment first exports the data of the complex table block in the form of a table data file without loss of quality, preserving its original layout relationship. Then, it uses a preset language model to generate metadata with structural semantics such as row dimensions, column dimensions, and metrics, and stores it in association with the data file. This avoids semantic loss or parsing errors caused by forcibly adjusting complex tables, and allows subsequent reading of the table to understand the structural meaning of complex table blocks through metadata, improving the understandability, reusability, and stability of complex spreadsheet parsing results.

[0086] To implement the table parsing method of this application embodiment, this application embodiment also provides a table parsing apparatus, such as... Figure 4 As shown, the device includes:

[0087] Encoding unit 401 is used to traverse the cells of each worksheet in the target spreadsheet file to encode the cells according to the content type of the cells and generate a feature matrix for characterizing the worksheet layout.

[0088] The determining unit 402 is used to determine multiple candidate splitting points on the worksheets of the target spreadsheet file based on the feature matrix.

[0089] The partitioning unit 403 is used to divide the worksheet into multiple table blocks based on the candidate splitting points;

[0090] The parsing unit 404 is used to parse the cells in the multiple table blocks respectively to generate structured data corresponding to each table block.

[0091] In one embodiment, the determining unit 402 is specifically used to: determine empty rows and columns with a non-empty cell percentage lower than a first threshold based on the proportion of non-empty cells in each row and the proportion of non-empty cells in each column of the feature matrix; and determine candidate splitting points on the worksheet according to the positions of the empty rows and columns in the feature matrix.

[0092] In one embodiment, the determining unit 402 is specifically configured to: extract the cell content of a first region corresponding to each candidate segmentation point; wherein, the first region is a local region formed by extending a preset number of rows or columns to the adjacent sides of the candidate segmentation point with the candidate segmentation point as the center row or column; input the cell content of the first region into a preset language model for semantic verification, and obtain the verification result output by the language model; in response to the verification result of the language model indicating that the cell content of the first region belongs to the same table, the candidate segmentation point is withdrawn.

[0093] In one embodiment, the device further includes: a free text cell detection unit; the free text cell detection unit is specifically configured to: identify free text cells located outside the outer boundary of the table block, using the outer frame of the table block as the outer boundary of the table block; calculate the distance between the free text cell and each table block in the worksheet based on the row and column position relationship between the free text cell and each table block; and associate the free text cell with the nearest table block to regard the free text cell as attached text information of the table block.

[0094] In one embodiment, the parsing unit 404 is specifically configured to: extract feature information for each table block to characterize the structural features of the table block; the structural features include at least the row distribution features, column distribution features, and style features of the table block; sample the data in the table block to obtain sampled data; input the feature information and the sampled data into a preset language model to obtain a structural type determination result for the table block; and parse the data in the table block according to the structural type determination result to generate corresponding structured data.

[0095] In one embodiment, the apparatus further includes: a data sampling unit; the data sampling unit is specifically configured to: starting from the first row of the table block, sequentially select multiple rows as candidate sampling rows, and for any two adjacent candidate sampling rows, determine whether to continue sampling downwards based on the similarity of the row distribution features corresponding to the two candidate sampling rows; when the similarity of several consecutive pairs of candidate sampling rows is greater than a second threshold, stop sampling downwards to obtain row sampling data in units of rows; starting from the first column of the table block, sequentially select multiple columns as candidate sampling columns, and for any two adjacent candidate sampling columns, determine whether to continue sampling to the right based on the similarity of the column distribution features corresponding to the two candidate sampling columns; when the similarity of several consecutive pairs of candidate sampling columns is greater than a third threshold, stop sampling to the right to obtain column sampling data in units of columns; and respectively limit the data volume of the row sampling data and the column sampling data, such that the number of sampling rows in the row sampling data does not exceed a fourth threshold, and the number of sampling columns in the column sampling data does not exceed a fifth threshold.

[0096] In one embodiment, the parsing unit 404 is specifically configured to: in response to determining that the structure type determination result indicates that the table block belongs to a first type of table, determine the table block as the rule table block, and perform type conversion on the data in the rule table block according to the field information and data type information corresponding to the rule table block to generate structured data corresponding to the rule table block; in response to determining that the structure type determination result indicates that the table block belongs to a second type of table, determine the table block as a complex table block, extract the data of the complex table block and generate metadata corresponding to the complex table block.

[0097] In one embodiment, the parsing unit 404 is specifically used to: extract the data of the complex table block and export it as a table data file; call a preset language model to analyze the structure of the complex table block, generate metadata for at least describing the row dimension, column dimension and metric value of the table block, and associate and store the metadata with the table data file.

[0098] It should be noted that the table parsing device provided in the above embodiments is only illustrated by the division of the above-described program modules. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the table parsing device and the table parsing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0099] Based on the hardware implementation of the above program modules, and in order to implement the table parsing method provided in this application embodiment, this application embodiment also provides a computer device, such as... Figure 5 As shown, computer device 500 includes:

[0100] Central processing unit 501, memory 502, and input / output interface 503;

[0101] The memory 502 is a short-term storage memory or a persistent storage memory;

[0102] The central processing unit 501 is configured to communicate with the memory 502 and execute instructions in the memory 502 to perform any of the above table parsing methods.

[0103] Of course, in practical applications, the various components in the computer device 500 are coupled together through a bus system 504. It is understood that the bus system 504 is used to realize communication between these components. In addition to a data bus, the bus system 504 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 5 The general designated all buses as Bus System 504.

[0104] The memory 502 in this embodiment is used to store various types of data to support the operation of the computer device 500. Examples of such data include any computer program used to operate on the computer device 500.

[0105] It is understood that when the processor in the computer device described above executes the computer program, it can also realize the functions of each unit in the corresponding device embodiments described above, which will not be repeated here. Exemplarily, the computer program can be divided into one or more modules / units, one or more modules / units are stored in memory and executed by the processor to complete the various embodiments of this application. One or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the computer device. For example, the computer program can be divided into units in the aforementioned computer device, and each unit can implement the specific functions described in the corresponding computer device above.

[0106] Computer equipment can be desktop computers, laptops, handheld computers, and cloud servers, among other computing devices. Computer equipment may include, but is not limited to, processors and memory. Those skilled in the art will understand that processors and memory are merely examples of computer equipment and do not constitute a limitation on the computer equipment. It may include more or fewer components, or combinations of certain components, or different components. For example, computer equipment may also include input / output devices, network access devices, buses, etc.

[0107] A processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of a computer device, connecting all parts of the computer device through various interfaces and lines.

[0108] Memory can be used to store computer programs and / or modules. The processor performs various functions of the computer device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can primarily include a program storage area and a data storage area. The program storage area can store the operating system, at least one application program required for a function, etc.; the data storage area can store data created based on terminal usage, etc. Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital cards (SD), flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0109] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, performs any of the table parsing methods described above.

[0110] This application also provides a computer program product that stores a computer program / instruction thereon. When the computer program / instruction is executed by a processor, it is used to implement the table parsing method described in the first aspect or any specific implementation of the first aspect of this application.

[0111] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0112] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0113] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0114] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0115] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A table parsing method, characterized in that, include: Traverse the cells of each worksheet in the target spreadsheet file to encode the cells according to their content type, and generate a feature matrix to characterize the worksheet layout; Based on the feature matrix, multiple candidate splitting points on the worksheets in the target spreadsheet file are determined; Based on the candidate splitting points, the worksheet is divided into multiple table blocks; The cells within each of the multiple table blocks are parsed to generate structured data corresponding to each table block.

2. The method according to claim 1, characterized in that, The step of determining multiple candidate splitting points on the worksheets of the target spreadsheet file based on the feature matrix includes: Based on the proportion of non-empty cells in each row and the proportion of non-empty cells in each column of the feature matrix, empty rows and empty columns with a proportion of non-empty cells lower than a first threshold are determined. Candidate split points on the worksheet are determined based on the positions of the empty rows and columns in the feature matrix.

3. The method according to claim 1, characterized in that, Before dividing the worksheet into multiple table blocks based on the candidate splitting points, the method further includes: For each candidate segmentation point, extract the cell content of the first region corresponding to the candidate segmentation point; wherein, the first region is a local region formed by extending a preset number of rows or columns to the adjacent sides of the candidate segmentation point with the candidate segmentation point as the center row or column; The cell content of the first region is input into a preset language model for semantic verification, and the verification result output by the language model is obtained. If the verification result of the language model indicates that the cell content of the first region belongs to the same table, then the candidate split point is withdrawn.

4. The method according to claim 1, characterized in that, After dividing the worksheet into multiple table blocks based on the candidate splitting points, the method further includes: Using the outer frame of the table block as the outer boundary of the table block, identify free text cells located outside the outer boundary of the table block; Based on the row and column position relationship between the free text cell and each of the table blocks in the worksheet, calculate the distance between the free text cell and each of the table blocks; The detached text cell is associated with the nearest table block to make the detached text cell an attached text information of the table block.

5. The method according to claim 1, characterized in that, The step of parsing the cells within the multiple table blocks to generate structured data corresponding to each table block includes: For each of the table blocks, feature information is extracted to characterize the structural features of the table block; the structural features include at least the row distribution features, column distribution features, and style features of the table block; The data in the table block is sampled to obtain sampled data; The feature information and the sampled data are input into a preset language model to obtain the structure type determination result of the table block; Based on the structure type determination result of the table block, the data in the table block is parsed to generate corresponding structured data.

6. The method according to claim 5, characterized in that, The step of sampling the data in the table block to obtain sampled data includes: Starting from the first row of the table block, multiple rows are selected sequentially as candidate sampling rows. For any two adjacent candidate sampling rows, it is determined whether to continue sampling downwards based on the similarity of the row distribution features corresponding to the two candidate sampling rows. When the similarity of several consecutive pairs of candidate sampling rows is greater than the second threshold, sampling downwards is stopped, and row sampling data is obtained in rows. Starting from the first column of the table block, multiple columns are selected sequentially as candidate sampling columns. For any two adjacent candidate sampling columns, it is determined whether to continue sampling to the right based on the similarity of the column distribution characteristics corresponding to the two candidate sampling columns. When the similarity of several consecutive pairs of candidate sampling columns is greater than the third threshold, sampling to the right is stopped, and column sampling data is obtained in units of columns. The data volume of the row sampled data and the column sampled data are respectively limited so that the sampled rows in the row sampled data do not exceed a fourth threshold and the sampled columns in the column sampled data do not exceed a fifth threshold.

7. The method according to claim 5, characterized in that, The step of parsing the data in the table block based on the structure type determination result of the table block to generate corresponding structured data includes: In response to the determination that the structure type determination result indicates that the table block belongs to the first type of table, the table block is determined as the rule table block, and the data in the rule table block is converted according to the field information and data type information corresponding to the rule table block to generate the structured data corresponding to the rule table block; In response to the determination that the structure type determination result indicates that the table block belongs to the second type of table, the table block is identified as a complex table block, the data of the complex table block is extracted, and metadata corresponding to the complex table block is generated.

8. The method according to claim 7, characterized in that, The step of extracting data from the complex table block and generating metadata corresponding to the complex table block includes: Extract the data from the complex table blocks and export it as a table data file; The structure of the complex table block is analyzed by calling a preset language model, and metadata is generated to at least describe the row dimension, column dimension and metric value of the table block. The metadata is then associated with and stored in the table data file.

9. A computer device, characterized in that, include: Central processing unit, memory, and input / output interfaces; The memory is either a short-term storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it is used to implement the method as described in any one of claims 1 to 8.

11. A computer program product having a computer program / instructions stored thereon, characterized in that, When executed by a processor, the computer program / instructions are used to implement the method as described in any one of claims 1 to 8.

Images