A method and apparatus for generating a structured data template

By defining data map templates and parsing cell attributes, structured data templates are generated, solving the problem of rapid conversion of unstructured table files and enabling efficient data retrieval and analysis. This is applicable to structured databases and file formats.

CN114936194BActive Publication Date: 2025-11-21CHENGDU SPACEON IND CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202210452267.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-27
Publication Date
2025-11-21
Estimated Expiration
2042-04-27

AI Technical Summary

Technical Problem

Existing technologies struggle to quickly convert unstructured tabular files into structured data, resulting in low efficiency in retrieval, calculation, and analysis, and failing to meet the needs of multidimensional data association and collision analysis.

Method used

By defining a data map template, extracting the text content feature types of table files, parsing cell attributes, constructing information item structures, and generating structured data templates, it supports storage in database, JSON, and XML file formats.

Benefits of technology

It enables rapid structuring of unstructured tabular files, supports efficient retrieval, calculation, and analysis, is applicable to structured databases and file formats, and facilitates subsequent standardized data development.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114936194B_ABST
    Figure CN114936194B_ABST
Patent Text Reader

Abstract

The application discloses a method for generating a structured data template, comprising the following steps: defining data map template information relationship, wherein the information relationship comprises a text content feature type of a data map template and a corresponding template data structure; obtaining a table file, and summarizing a text content feature type of the table file; defining the corresponding template data structure according to the text content feature type of the table file; parsing the table file according to the template data structure, extracting cell attribute information to fill information item content of the template data structure, and constructing a table content data map; and selecting and processing information items of the data map to generate a structured data template. According to the above technical scheme, the unstructured table file can be extracted with a standard specification to generate a structured data template for storage, thereby preparing for later standardized data development.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application relates to the field of file data processing, in particular to a method and device for generating a structured data template. BACKGROUND

[0002] Although informationization construction is developing rapidly, there are still many occasions using unstructured table files. In the management involving personnel, a large number of case files and other documents written by grassroots staff before the establishment of personnel management system and related information system have a unified format, but belong to unstructured text table, for example, there are merged combinations and multiple values in multiple cells. Unstructured table text is not easy to retrieve, summarize, calculate and analyze systematically. In order to realize multi-dimensional data association and collision analysis and support upper-level business, a large number of standard documents need to be structured and extracted, corresponding structures are extracted for different types, templates are generated, and structured data is extracted from the filled case files for subsequent intelligence analysis business. SUMMARY

[0003] The main purpose of the present application is to disclose a scheme for generating a structured data template, which is applied to data analysis of standard unstructured table files, extracts the structure of such table files, summarizes them into standardized data templates, supports the existence of structured databases and structured files, matches the template with the unstructured table files of the same type, extracts the data structure in a standard and normative manner, integrates it with the structure in the template to generate a structured data template for storage, and prepares for the development of standardized data in the later stage.

[0004] In the first aspect, in order to achieve the above-mentioned purpose, the present application provides a method for generating a structured data template, comprising the following steps:

[0005] Defining data map template information relationship, which includes text content feature type of data map template and corresponding template data structure; the text content feature and data structure are agreed through information item warehouse; wherein the template data structure is a tree structure composed of information items, and the information item is the smallest data unit;

[0006] Obtaining a table file and summarizing the text content feature type of the table file;

[0007] Defining the corresponding template data structure according to the text content feature type of the table file;

[0008] Parsing the table file according to the template data structure, extracting cell attribute information to fill the information item content of the template data structure, and constructing a table content data map;

[0009] Select and process the information items of the data map to generate a structured data template.

[0010] Further, an information item includes a plurality of key-value pairs, and each key-value pair is composed of an information name and an information value.

[0011] Further, when the information item includes a key-value pair with an information name of information item type, the corresponding information value includes a node and a text; when the information item includes a key-value pair with an information value of text, the information item further includes a key-value pair with an information name of value position; when the information item includes a key-value pair with an information value of node, the information item further includes a key-value pair with an information name of sub-information item, and the corresponding information value of the sub-information item includes one or more information items.

[0012] Further, the text content feature type of the table file is summarized, which includes identifying the title of the table file, obtaining the text information of the title, including string information, font name, and font size number; combining these information to generate feature item information, and generating the text content feature type by MD5 encryption of the feature item information.

[0013] Further, the table file is parsed according to the template data structure, which includes obtaining the attributes of each cell in the table, including content attribute, position attribute, type attribute, and layout attribute.

[0014] Further, the content attribute is the text content of a specified cell extracted from the table file; the type attribute is the type of the content extracted from the cell, including a title cell and a value cell; and the position attribute is the specific position extracted from the cell, including table number, column number, and row number.

[0015] Further, the layout attribute of a specified cell can be determined by the attributes of the specified cell and the attributes of the corresponding adjacent cells, including aggregation combination, single-value combination, and multi-value combination.

[0016] Further, the information item content of the template data structure is filled by extracting the attribute content of all cells in the table file and corresponding information item structure, including the correspondence between the information item title and the content attribute, the correspondence between the title position and the position attribute, the correspondence between the value position and the position attribute, and the correspondence between the information item type and the layout attribute.

[0017] Further, the construction of the table content data map includes realizing the content of the information item and synthesizing the structured data file corresponding to the table file; and the realization mode of the structured data file supports a database table structure, a custom protocol file format, a JSON file format, and an XML file format.

[0018] Further, the information items of the data map are selected and processed to generate a structured data template, including: loading a structured data file, performing content cleaning on the structured data file, including extracting key-value pairs in the information items, cleaning information values, and retaining information names; and saving the content-cleaned structured data file as a structured data template.

[0019] In another aspect, the application provides a device for generating a structured data template, including:

[0020] A data map template structure definition module is configured to define information relationships of a data map template, the information relationships including text content feature types of the data map template and corresponding template data structures; wherein the template data structures are tree structures formed by information items, and the information items are minimum data units; one information item includes multiple key-value pairs, and the key-value pairs are formed by information names and information values;

[0021] A table file attribute extraction module is configured to obtain a table file, summarize text content feature types of the table file, define corresponding template data structures, parse the table file according to the template data structures, and extract cell attribute information;

[0022] A data map construction module is configured to fill the information item content of the template data structures with the cell attribute information, and construct a table content data map;

[0023] A structured data template generation module is configured to select and process information items of the data map to generate a structured data template.

[0024] Further, the table file attribute extraction module includes:

[0025] A text content feature type generation unit is configured to identify a title of a table file, obtain text information of the title, including string information, font name, and font size number; combine the information to generate feature item information, and perform MD5 encryption on the feature item information to generate a text content feature type.

[0026] A cell attribute judgment unit is configured to obtain attributes of each cell in a table, the attributes including: content attributes, position attributes, type attributes, and layout attributes.

[0027] According to the application, the structure of an unstructured table file can be extracted, summarized as a standardized data template, and supported to exist in the form of a structured database or a structured file. The template is matched with a non-structured table file of the same type, the data structure of an existing table is extracted in a standard specification, integrated with the structure in the template, and a structured data template is generated for storage, thereby preparing for later standardized data development and construction. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a method flowchart for generating a structured data template according to an embodiment of the present application;

[0029] Figure 2 is an information item structure of a data map according to an embodiment of the present application;

[0030] Figure 3 is a method flowchart for acquiring a cell content attribute according to an embodiment of the present application;

[0031] Figure 4 is a device flowchart for generating a structured data template according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] There are many scenarios in which a management department prepares a table in a unified format and issues it to each grassroots department, and personnel in the grassroots department fill in the table as necessary, and the management department periodically collects the filled-in table to manage, archive, and analyze it, such as a personnel information registration table. Due to the particularity of the table content, structured data cannot be directly exported. In the present application, a non-structured text table with a unified format is summarized and processed to generate a structured data template corresponding to the format type. The data template is applied to a table filled in with data, and can quickly extract structured data for retrieval, calculation, statistics, and analysis.

[0033] The specific implementation of the present application will be described in detail below in conjunction with the accompanying drawings.

[0034] The process of generating a structured data template in the present application is a method based on data map processing. The core of the data map is to describe the relationship between each cell in the table through multiple sets of structured key-value pairs. The data structure source of the data map is a standard title word library in a specific field.

[0035] Therefore, the basis of the present application is to construct a standard title word dictionary title-dictionary, and to expand the synonyms and abbreviations of each word and supplement the records to the dictionary. For example, the personnel management text data in a specific field mainly refers to the table text in the documents and other written materials, and the range of the standard title word dictionary includes commonly used information item title words and special information item title words in the documents and other written materials.

[0036] The information items in all text data are counted, converted into information item data structures one by one, and after deduplication, an information item warehouse is generated, which also includes the aggregation of all standard values and equivalent values for data types with fixed value ranges to obtain a value set, and then the mapping relationship between the data type and the value set is constructed, for example, the information item warehouse includes information item standard title name, information item value type, information item value range, and value validity check rules and other attributes.

[0037] For example, there is a data table as follows, which contains a table header "household information table" and three content information titles: household owner, name, and contact information. Next, this table is taken as an example for description.

[0038]

[0039] The information item standard title words of a large number of tables in the information item warehouse are extracted, and the data structure of the title item is obtained, for example, the information item standard title "household owner" is a string type; the information item standard title "contact information" is a string type with a length of 11; and the information item standard table "household information table" has a table header feature of bold 4.

[0040] Figure 1 is a method flowchart for generating a structured data template, as shown in Figure 1 The generation of the structured data template includes the following steps:

[0041] Step S110: defining data map template information, including text content feature types of the data map template and corresponding template data structures; wherein the text content features and data structures are agreed through the information item warehouse.

[0042] For example, different table files have different formats, and in the header of the table, the text content features are distinguished according to the size, name, and font size of the unified font. These differences in table formats constitute the text content feature types. Different text content feature types correspond to different table contents, that is, different template data structures, for example, the table header "household information table" of the present application can be identified according to the name, font size, and font name of the text content feature type.

[0043] The template data structure is a tree structure composed of information items, and the information item is the smallest data unit in the data map template; the tree structure is used to express the relationship between the information items, for example, in the household information table, the information "household owner" and "name" belong to the parent-child relationship of upper and lower layers.

[0044] The structure of the key-value pair is shown in Figure 2

[0045] ​In the present application, one information item includes multiple key-value pairs, one key-value pair is composed of information name and information value; the information name of the key-value pair includes information item title, title position, information item type; the information value corresponding to the information name of the key-value pair supports one or more normal data, information item.

[0046] When the information name of a key-value pair in one information item is information item type, the corresponding information value includes node and text;

[0047] When the information value of a key-value pair in one information item is text, the information item also includes a key-value pair with information name as value position;

[0048] When the information value of a key-value pair in one information item is node, the information item also includes a key-value pair with information name as sub-information item, and the corresponding information value of the sub-information item includes one or more information items.

[0049] For example, in the present application example, three information items are included to record the information of "householder", "name" and "contact information". Taking the information item corresponding to "householder" as an example, at least three groups of key-value pairs are included, which are information item title, title position and information item type. Since the information item type of this information item is "node", the information item also includes a key-value pair with information name as sub-information item. The table in the present example is expressed as follows according to the structure of Figure 2

[0050]

[0051] As can be seen from the above table, the data structure characteristics of the information item are constrained by the information item warehouse, for example, the characteristics of the householder are string, and the length of the contact information is 11.

[0052] Since there may be multiple tables in one table file, when confirming the position of a cell, three information of table number, column number and row number are needed. As can be seen from the present example, when the information name of the information item is title position, the corresponding information value is a multi-value combination, including table number, cell column position and cell row position;

[0053] The above content defines the text content characteristic type of the data map template in the data map template information and the corresponding template data structure, the attributes and relationships of each value in the data structure.

[0054] In the actual application scenario, there are a large number of table files with different formats, so it is necessary to summarize the templates of table files with the same format and define the corresponding text content characteristic type and the corresponding template data structure.

[0055] The following steps will introduce the application of the data map template structure in the process of generating the structured data template in detail. ​

[0056] Step S120: Obtain the table file, and summarize the text content feature type of the table file;

[0057] Summarizing the text content feature type of the table file refers to identifying the title of the table file, and obtaining the text information of the title, including string information, font name, and font size number.

[0058] The information is combined to generate feature item information, and the feature item information is MD5 encrypted to generate the text content feature type.

[0059] In this example, first, the blank string in the top title text of the personnel management text data is removed to obtain the title, and then the punctuation marks of the title are removed to obtain the pure string title "Household Information Table". The font name is bold and the font size number is 2, which are combined as the identification information "Household Information Table-Black-2". The identification information is MD5 encrypted to generate the ciphertext "AB51985448A6865EC8ECBC32E9E3C0F6" as the text content feature type. In this way, the length of the text content feature type can be fixed, the string comparison and retrieval can be facilitated, and the use efficiency can be improved.

[0060] Step S130: Define the corresponding template data structure according to the text content feature type of the table file;

[0061] According to the text content feature type obtained in step S120, the template data structure is extracted.

[0062] Step S140: According to the template data structure, parse the table file, extract the information item content of the template data structure, and construct the table content data map;

[0063] After determining the text content feature type of the table, the table file is parsed according to the template data structure to obtain the attributes of each cell in the table, including content attributes, position attributes, type attributes, and layout attributes. The content attribute is the text content of the specified cell extracted from the table file. The type attribute is the type of the content extracted from the cell, including the title cell and the value cell. The position attribute is the specific position of the cell extracted, including the table number, the column number, and the row number. It also includes the combination method of the cell, including aggregation combination, single-value combination, and multi-value combination.

[0064] In this example, without considering the title "Household Information Table", the table includes five cells. Based on the position attribute, the attributes of each cell are obtained as follows:

[0065] Location attribute Content attribute Type attribute Layout attribute 1,2,1 Householder Title Aggregation 1,2,2 Name Title Single value 1,2,3 Value Single value 1,3,2 Contact information Title Single value 1,3,3 Value Single value

[0066] Position attribute is the identification attribute of each cell, which indicates the specific position of each cell, including table number, column number and row number. For example, the cell of "Name" is in the first table, the second row and the second column.

[0067] Layout attribute includes aggregate combination, single-value combination and multi-value combination, which needs to combine the content of multiple adjacent cells to determine. The determination process is shown in the following table: Figure 3

[0068] The determination process takes the top-left corner of the table as the starting point of calculation, and uses the sequence from left to right and from top to bottom:

[0069] 1) Find the first cell with non-empty content and mark it as the title cell T0.

[0070] 2) Determine whether the content of the cell immediately to the right of T0 is empty, and check whether the height of the cell to the right is equal to that of T0. If the content is empty and the height is equal, mark it as a value cell V0, and associate T0 with V0 to form a single-value combination.

[0071] Continue to check to the right of V0, and repeat step 2). If the content of the adjacent cell is empty, mark it as a value cell V1, and associate T0 with V0 and V1 to form a multi-value combination. Repeat step 2) until the end of the row or the next cell has non-empty content.

[0072] If the next cell has non-empty content, i.e., the case of step 1), continue to repeat step 2) to determine the type of all cells in the row.

[0073] 3) If the above steps cannot find the adjacent value cell, check whether the content of the cell immediately below T0 is empty, and check whether the length of the cell below is equal to that of T0. If the content is empty and the length is equal, mark it as a value cell V0, and associate T0 with V0 to form a single-value combination.

[0074] Continue to check below V0, and repeat step 3). If the content of the adjacent cell is empty, mark it as a value cell V1, and associate T0 with V0 and V1 to form a multi-value combination. Repeat this step until the end of the column or the cell has non-empty content.

[0075] 4) For the title cell that cannot find the adjacent value cell to the right and below, record it as the title cell A0 of the aggregate combination.

[0076] 5) Check to the right of A0, and the height range is consistent with the height of A0. All single-value or multi-value combinations immediately to the right of A0 are recorded as being associated with A0, until a second aggregate title is encountered to the right of A0 or the table boundary is reached.

[0077] ​6) Examine downwards from A0, with the length range being the same as the length of A0. All single or multi-value combinations immediately below A0 are recorded as associated with A0, until a second aggregation header is encountered below A0 or the table boundary is reached.

[0078] After obtaining the attribute content of all cells in the table file, it is matched with the information item structure, that is, the information is extracted to populate the information items of the template data structure, including the following correspondence:

[0079] The information item title corresponds to the content attribute, the title position corresponds to the position attribute, the value position corresponds to the position attribute, and the information item type corresponds to the layout attribute.

[0080] Different text content feature types correspond to different template data structures, generating different structured data files; the structured data files support database table structures, custom protocol file formats, JSON file formats, and XML file formats.

[0081] In the example of this application, the final converted structured data file content of the data map corresponding to "AB51985448A6865EC8ECBC32E9E3C0F6" is as follows:

[0082] {

[0083] "title": "Head of Household"

[0084] "labelPosition":{"table":1,"rowNumber":2,"cellNumber":1},

[0085] "type":"node",

[0086] “items”:[{

[0087] "title": "Name",

[0088] "labelPosition":{"table":1,"rowNumber":2,"cellNumber":2},

[0089] "valuePosition":{"table":1,"rowNumber":2,"cellNumber":3},

[0090] "type":"text"

[0091] },{

[0092] "title": "Contact Information",

[0093] "labelPosition":{"table":1,"rowNumber":3,"cellNumber":2},

[0094] "valuePosition":{"table":1,"rowNumber":3,"cellNumber":3},

[0095] "type": "text",

[0096] “length”:11

[0097] }]

[0098] }

[0099] Step S150: Select and process the information items in the data map to generate a structured data template.

[0100] In this step, a corresponding structured data file is loaded based on the text content feature type. The structured data file is then cleaned, including extracting key-value pairs from information items, removing descriptive fields such as title location, value location, and data length, retaining the information name, and adding value data fields. The cleaned structured data file is then saved as a structured data template. The storage method can be XML, a database table, a JSON file, etc.

[0101] In this application example, the processed structured data template is in JSON file format, and its content is as follows:

[0102] {

[0103] "title": "Head of Household"

[0104] “items”:[{

[0105] "title": "Name",

[0106] “value”:null,

[0107] },{

[0108] "title": "Contact Information",

[0109] “value”:null,

[0110] }]

[0111] }

[0112] Figure 4 This is a structural diagram of the device for generating structured data templates, including the following modules:

[0113] P410: data map template structure definition module:

[0114] For defining data map template information, including text content feature types of data map template and corresponding template data structure; wherein, the template data structure is a tree structure composed of information items, and the information item is the smallest data unit; one information item includes a plurality of key-value pairs, and the key-value pair is composed of information name and information value;

[0115] P420: table file attribute extraction module: for obtaining a table file, summarizing text content feature types of the table file, defining corresponding template data structure; parsing the table file according to the template data structure, and extracting cell attribute information;

[0116] In the table file attribute extraction module, the specified table file is extracted, the attributes of the cells in the table are extracted in combination with the data map template structure defined in the data map template structure definition module, and the above functions are realized, including the following functional units:

[0117] P421: text content feature type generation unit: for identifying the title of the table file, obtaining the text information of the title, including string information, font name and font size number; combining the information to generate feature item information, and generating text content feature type by MD5 encryption of the feature item information.

[0118] P422: cell attribute judgment unit: for obtaining the attributes of each cell in the table, such as: obtaining the content of the cell to generate content attribute, obtaining the table number, column number and row number of the cell to generate position attribute, combining the content of adjacent cells to judge layout attribute, and judging type attribute according to the content.

[0119] P430: data map construction module: for filling the information item content of the template data structure with the cell attribute information, and constructing the table content data map; that is, combining the attributes of all cells in the table file with the data map template to generate a specific data map.

[0120] The data map is embodied in the form of a structured database, a JSON file and an XML file.

[0121] P440: structured data template generation module: for selecting and processing the information items of the data map to generate a structured data template.

[0122] The scheme in the application solves the structured processing of a large number of different format tables, for example, a unit collects and arranges a household registration table, a school-age child registration table and various types of tables, a template is generated through the scheme, an existing table file is generated into a structured template, the template is convenient to use in a structured scene, for example, a client terminal accesses information items for data processing, a batch of tables submitted by each household are processed into structured data, which is convenient for statistics and calculation, and is also convenient for later access of the formed file content to a comprehensive management system.

[0123] The above only discloses several specific embodiments of the application, but the application is not limited thereto, and any changes that can be thought of by those skilled in the art shall fall within the protection scope of the application.

Claims

1. A method for generating structured data templates, characterized in that, include: Define the information relationship of the data map template, which includes the text content feature type of the data map template and the corresponding template data structure; The text content features and data structure are defined through an information item repository; wherein, the template data structure is a tree structure composed of information items, and the information item is the smallest data unit; Obtain the table file and summarize the text content characteristics of the table file; Define the corresponding template data structure based on the text content feature type of the table file; Parse the table file according to the template data structure, extract cell attribute information to fill the information item content of the template data structure, and construct a table content data map. The information items of the data map are selected and processed to generate a structured data template; The step of parsing the table file according to the template data structure refers to obtaining the attributes of each cell in the table. These attributes include content attributes, position attributes, type attributes, and layout attributes. The content attribute is the text content of a specified cell extracted from the table file. The type attribute is the type of content extracted from the cell, including header cells and value cells. The position attribute is the specific location of the extracted cell, including table number, column number, and row number. The layout attribute is determined based on the attributes of a specified cell and the attributes of corresponding adjacent cells. Layout attributes include aggregated groups, single-value groups, and multi-value groups. The judgment process starts from the top left corner of the table and proceeds from left to right and top to bottom, including: Step 1, finding the first cell with non-empty content and marking it as the title cell T0; Step 2, checking if the content of the cell immediately to the right of T0 is empty, and simultaneously checking if the height of the cell to the right is equal to that of T0. If the content is empty and the height is equal, it is marked as the value cell V0, and T0 and V0 are associated to form a single-value group; continuing to check to the right of V0, repeating the judgment in Step 2, if the content of the immediately to the right of V0 is empty, it is marked as the value cell V0.

1. T0 is associated with V0 and V1 to form a multi-value combination; repeat step 2 until the end of a row or the next cell is not empty; Step 3. If the above steps cannot find an adjacent value cell, check if the cell immediately below T0 is empty, and also check if the length of the cell below is equal to T0. If the content is empty and the length is equal, mark it as value cell V0, and T0 is associated with V0 to form a single-value combination; continue checking below V0, repeating the judgment in step 3. If the adjacent cell is empty, mark it as value cell V1, and T0 is associated with V0 and V1 to form a multi-value combination; repeat this step until... Step 4: For header cells where no adjacent value cells can be found to the right or down, record them as header cells of the aggregated group, A0. Step 5: Check to the right of A0, with the height range consistent with the height of A0. Record all single or multi-value combinations adjacent to A0 on the right as associated with A0, until a second aggregated header is encountered to the right of A0 or the table boundary is reached. Step 6: Check below A0, with the length range consistent with the length of A0. Record all single or multi-value combinations adjacent below A0 as associated with A0, until a second aggregated header is encountered below A0 or the table boundary is reached.

2. The method for generating structured data templates according to claim 1, characterized in that, A single information item comprises multiple key-value pairs, each key-value pair consisting of an information name and an information value. The information name of the key-value pair includes the information item title, title position, and information item type; the information value corresponding to the information name of the key-value pair supports one or more ordinary data items. When an information item includes key-value pairs whose information names are of type information item, the corresponding information values ​​include nodes and text; When an information item includes a key-value pair whose information value is text, the information item also includes a key-value pair whose information name is the position of the value. When an information item includes a key-value pair whose information value is a node, the information item also includes a key-value pair whose information name is a sub-information item, and the information value corresponding to the sub-information item includes one or more information items.

3. The method for generating structured data templates according to claim 2, characterized in that, The text content feature type of the summarized table file refers to: Identify the title of the table file and obtain the text information of the title, including string information, font name, and font size number; The information is combined to generate feature information, and the feature information is then encrypted using MD5 to generate a text content feature type.

4. The method for generating structured data templates according to claim 1, characterized in that, The step of extracting information to fill the information item content of the template data structure refers to obtaining the attribute content of all cells in the table file and matching it with the information item structure, including: The information item title corresponds to the content attribute, the title position corresponds to the position attribute, the value position corresponds to the position attribute, and the information item type corresponds to the layout attribute.

5. The method for generating structured data templates according to claim 4, characterized in that, Building a table content data map refers to: To realize the content of the information items, a structured data file corresponding to the table file is synthesized; The structured data file can be implemented using database table structures, custom protocol file formats, JSON file formats, and XML file formats.

6. The method for generating structured data templates according to claim 5, characterized in that, The step of selecting and processing information items from the data map to generate a structured data template refers to: Load the structured data file and clean its content, including extracting key-value pairs from information items, clearing information values, and retaining information names; After cleaning up the content, save the structured data file as a structured data template.

7. An apparatus for generating structured data templates, characterized in that, include: Data map template structure definition module: used to define the information relationship of the data map template, the information relationship including the text content feature type of the data map template and the corresponding template data structure; wherein, the template data structure is a tree structure composed of information items, and the information item is the smallest data unit; one information item includes multiple key-value pairs, and the key-value pairs are composed of information name and information value; Table file attribute extraction module: used to obtain the table file, summarize the text content feature types of the table file, define the corresponding template data structure; parse the table file according to the template data structure, and extract cell attribute information; Data map construction module: used to fill the information items of the template data structure with the cell attribute information to construct a table content data map; Structured data template generation module: used to select and process information items of the data map to generate structured data templates; The step of parsing the table file according to the template data structure refers to obtaining the attributes of each cell in the table. These attributes include content attributes, position attributes, type attributes, and layout attributes. The content attribute is the text content of a specified cell extracted from the table file. The type attribute is the type of content extracted from the cell, including header cells and value cells. The position attribute is the specific location of the extracted cell, including the table number, column number, and row number. The layout attribute is determined based on the attributes of a specified cell and the attributes of its corresponding adjacent cells. The layout attributes include aggregated combinations, single-value combinations, and multi-value combinations. The judgment process uses the top-left corner of the table as the starting point and follows a left-to-right, top-to-bottom order, including: Step 1, finding the first cell with non-empty content and marking it as the title cell T0; Step 2, determining whether the content of the cell immediately to the right of T0 is empty, and simultaneously checking whether the height of the cell to the right is equal to that of T0. If the content is empty and the height is equal, it is marked as the value cell V0, and T0 and V0 are associated to form a single-value combination; continuing to check to the right of V0, repeating the judgment in Step 2, if the content of the immediately to the right of V0 is empty, it is marked as the value cell. V1, T0 is associated with V0 and V1 to form a multi-value combination; repeat step 2 until the end of a row or the next cell is not empty; step 3, if the above steps cannot find the adjacent value cell, check if the content of the cell immediately below T0 is empty, and at the same time check if the length of the cell below is equal to that of T0. If the content is empty and the length is equal, mark it as value cell V0, and associate T0 with V0 to form a single-value combination; continue to check below V0, repeating the judgment in step 3. If the content of the adjacent cell is empty, mark it as value cell V1, and associate T0 with V0 and V1 to form a multi-value combination; repeat this step until... Step 4: For header cells where no adjacent value cells can be found to the right or down, record them as header cells of the aggregated group, A0. Step 5: Check to the right of A0, with the height range consistent with the height of A0. Record all single or multi-value combinations adjacent to A0 on the right as associated with A0, until a second aggregated header is encountered to the right of A0 or the table boundary is reached. Step 6: Check below A0, with the length range consistent with the length of A0. Record all single or multi-value combinations adjacent below A0 as associated with A0, until a second aggregated header is encountered below A0 or the table boundary is reached.

8. The apparatus for generating structured data templates according to claim 7, characterized in that, The table file attribute extraction module includes: Text content feature type generation unit: used to identify the title of the table file, obtain the text information of the title, including string information, font name, and font size number; combine the information to generate feature item information, MD5 encrypt the feature item information, and generate text content feature type; Cell attribute determination unit: used to obtain the attributes of each cell in the table, including: content attribute, position attribute, type attribute, and layout attribute.

Citation Information

Patent Citations

  • Method for extracting and organizing unstructured sheet document data under big data environment

    CN105630916A