Table data processing method, device, equipment, medium and computer program product

Abstract

The application discloses a table data processing method, device, equipment, medium and computer program product. The application relates to the technical field of data processing. The method comprises the following steps: acquiring a table file and a file identifier, and analyzing the table file; generating a table storage identifier of the table file; acquiring a structure identifier in the content of the table file, and taking the structure identifier, the file identifier and the table storage identifier as metadata of the table file; storing the content in the table file in a storage space according to a table structure; and storing the content stored in the storage space in the table file and the metadata of the table file correspondingly. The technical scheme of the application realizes automatic synchronization of the table file, improves the synchronization efficiency of the table file, and guarantees the synchronization accuracy of the table file.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of data processing, and particularly relate to a table data processing method, device, equipment, medium and computer program product. BACKGROUND

[0002] With the continuous development of information technology, more and more data files need to be synchronized. Common data files are mainly table files, which can be used for data statistics and records. How to quickly synchronize the table data is an urgent problem to be solved.

[0003] In the prior art, the synchronization of table files depends on the proficiency of users in the functions of table files, and it is difficult to realize automatic synchronization of table files, and there is a problem of low synchronization efficiency and difficult to guarantee synchronization accuracy of table files. SUMMARY

[0004] Embodiments of the present application provide a table data processing method, device, equipment, medium and computer program product, which realizes automatic synchronization of table files, improves the synchronization efficiency of table files, and guarantees the synchronization accuracy of table files.

[0005] In a first aspect, the embodiments of the present application provide a table data processing method, comprising:

[0006] obtaining a table file and a file identifier, and parsing the table file;

[0007] generating a table storage identifier of the table file;

[0008] obtaining a structure identifier in the content of the table file, and taking the structure identifier, the file identifier and the table storage identifier as metadata of the table file;

[0009] storing the content in the table file in a storage space according to the table structure;

[0010] storing the content stored in the storage space in the table file and the metadata of the table file correspondingly.

[0011] In a second aspect, the embodiments of the present application further provide a table data processing device, comprising:

[0012] a table file obtaining module configured to obtain a table file and a file identifier, and parse the table file;

[0013] a table storage identifier generating module configured to generate a table storage identifier of the table file;

[0014] a structure identifier obtaining module configured to obtain a structure identifier in the content of the table file, and take the structure identifier, the file identifier and the table storage identifier as metadata of the table file;

[0015] a table file content storage module, configured to store the content in the table file into the storage space according to the table structure;

[0016] a table file storage module, configured to store the content stored in the storage space in the table file and the metadata of the table file correspondingly.

[0017] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the table data processing method according to any of the embodiments of the present application when executing the program.

[0018] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the program is executable on a processor to implement the table data processing method according to any of the embodiments of the present application.

[0019] In a fifth aspect, a computer program product is provided, which includes a computer program, and the computer program is executable on a processor to implement the table data processing method according to any of the embodiments of the present application.

[0020] The technical scheme of the embodiments of the present application solves the problem that it is difficult to realize automatic synchronization of table files, and the synchronization efficiency of table files is low and the synchronization accuracy is difficult to guarantee, by obtaining a table file and a file identifier, analyzing the table file to generate a table storage identifier of the table file, obtaining a structure identifier in the content of the table file, storing the structure identifier, the file identifier, and the table storage identifier as metadata of the table file, storing the content in the table file into the storage space according to the table structure, and storing the content stored in the storage space in the table file and the metadata of the table file correspondingly, realizes automatic synchronization of table files, improves the synchronization efficiency of table files, and guarantees the synchronization accuracy of table files. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical scheme of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be considered as a limitation on the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0022] Figure 1 A flowchart of a table data processing method provided by the embodiments of the present application;

[0023] Figure 2 A flowchart of a table data processing method provided by the embodiments of the present application;

[0024] Figure 3 A flow chart of a table data processing method provided by an embodiment of the present application is shown in FIG. 1. The table data processing method can be applied to processing of table data. The table data processing method can be executed by a table data processing apparatus, which can be implemented in the form of hardware and / or software. The table data processing apparatus can be configured in an electronic device.

[0025] Figure 4 A principle diagram of saving of a Sheet mapping relationship provided by an embodiment of the present application is shown in FIG. 2.

[0026] Figure 5 A principle diagram of saving of a column mapping relationship of a single-page table provided by an embodiment of the present application is shown in FIG. 3.

[0027] Figure 6 A principle diagram of saving of contents of a table file provided by an embodiment of the present application is shown in FIG. 4.

[0028] Figure 7 A process schematic diagram of creating a wide table provided by an embodiment of the present application is shown in FIG. 5.

[0029] Figure 8 A structure schematic diagram of a table data processing apparatus provided by an embodiment of the present application is shown in FIG. 6.

[0030] Figure 9 A structure schematic diagram of an electronic device suitable for a table data processing method provided by an embodiment of the present application is shown in FIG. 7. DETAILED DESCRIPTION

[0031] The present application will be further described below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely intended for explaining the present application but not limiting the present application. In addition, it should be noted that only the parts related to the present application are shown in the accompanying drawings for the convenience of description.

[0032] It should be noted that similar reference numerals and letters represent similar items in the accompanying drawings, and thus, once an item is defined in one drawing, it need not be further defined and explained in subsequent drawings. Meanwhile, in the description of the present application, the terms "first", "second", and the like are merely used for distinguishing description but cannot be understood as indicating or implying relative importance. The acquisition, storage, use, processing, and the like of data in the technical solutions of the embodiments of the present application all comply with the relevant provisions of national laws and regulations.

[0033] Figure 1 A flow chart of a table data processing method provided by an embodiment of the present application is shown in FIG. 1. The table data processing method can be applied to processing of table data. The table data processing method can be executed by a table data processing apparatus, which can be implemented in the form of hardware and / or software. The table data processing apparatus can be configured in an electronic device. Figure 1 The table data processing method shown in FIG. 1 includes:

[0034] S110, acquire the table file and the file identifier, and parse the table file.

[0035] The table file can be used to record and count data. For example, the table file can include an Excel file or a CSV file. Alternatively, the table file can be used to record log data of an application, a webpage or a system. The file identifier is used to uniquely identify the table file. For example, the file identifier can be a file name.

[0036] Specifically, the table file and the file identifier of the table file can be acquired, and the table file can be parsed by using a table file parsing tool. For example, the table file parsing tool can be a third-party open source library DocumentFormat.OpenXml.

[0037] S120, generate a table storage identifier of the table file.

[0038] The table storage identifier can be a mapping identifier of the structure in the table file in the storage space. The table storage identifier has a unique correspondence with the structure in the table file. It can be understood that the structure of one table file uniquely corresponds to one table storage identifier, the table storage identifiers of different structures of the same table file are different, and the table storage identifiers of the structures of different table files are also different. Alternatively, one table file can include at least one structure, and accordingly, one table file can correspond to at least one table storage identifier. Alternatively, the content of the table storage identifier can include at least one of letters, numbers and characters. For example, the table storage identifier can be "EN1", wherein EN is a letter and 1 is a number, i.e. the number of the table storage identifier.

[0039] Specifically, the storage identifier of the table file can be automatically generated according to a preset rule. The preset rule can include the content form contained in the table storage identifier, the combination method of the content form contained in the table storage identifier, and the coding order of different table storage identifiers.

[0040] S130, acquire the structure identifier in the content of the table file, and take the structure identifier, the file identifier and the table storage identifier as the metadata of the table file.

[0041] The content of the table file can include a structure of the table file and data in the table file. The structure identifier can be used to identify the data structure of the table file. For example, the table file can be an Excel file, and the structure identifier can include a single table identifier of each single page table of the table file, a row identifier in a single single page table, and a column identifier in the single single page table, etc. The metadata of the table file is used to represent the mapping relationship between the structure of the table file and the table storage identifier in the storage space. It can be understood that, through the file identifier, it can be determined which table file is processed; through the structure identifier, it can be determined the structure of the table file being processed; and through the table storage identifier, it can be determined the mapping identifier of the structure of the table file being processed in the storage space.

[0042] Specifically, the structure identifier in the content of the table file can be acquired, and the structure identifier, the file identifier and the table storage identifier are stored in association to obtain the metadata of the table file.

[0043] S140, the content in the table file is stored in the storage space according to the table structure.

[0044] The storage space can be used to store the content of the table file and the metadata of the table file. For example, the storage space can include a hard disk storage space, a network disk storage space, a cache storage space or a memory storage space, etc.

[0045] Specifically, the content in the table file can be stored in the storage space according to the table structure.

[0046] S150, the content stored in the storage space in the table file is stored in correspondence with the metadata of the table file.

[0047] Specifically, the content stored in the storage space in the table file can be stored in correspondence with the metadata of the table file in the storage space.

[0048] The technical scheme of the embodiment of the application obtains a table file and a file identifier, analyzes the table file, generates a table storage identifier of the table file, obtains a structure identifier in the content of the table file, and stores the content in the table file in the storage space according to the table structure, corresponding storage of the content stored in the storage space in the table file and the metadata of the table file is performed, automatic synchronization of the table file is realized, the mapping relationship between the structure of the table file and the table storage identifier in the storage space is established by determining the metadata of the table file, the structure analysis of the table file and the structure synchronization of the table file are realized based on the structure identifier, meanwhile, the mapping relationship between the data is simpler compared with other manners, the synchronization efficiency of the table file is improved, and the accuracy of the content of the table file stored in the storage space is ensured.

[0049] In an optional embodiment of the application, the structure identifier in the content of the table file is obtained, and the structure identifier, the file identifier and the storage identifier are taken as the metadata of the table file, which is specifically that the single table identifier of the single page table in the table file is obtained, and the single table identifier is determined as the structure identifier; the file identifier, the single table identifier and the table storage identifier of the table file are spliced by using a preset separator to form the metadata of the table file.

[0050] The table file can include at least one single page table. The single page table can refer to an independent table in the table file. The single table identifier is used to identify the single page table in the table file. For example, the table file can be an Excel file; the single page table can be a Sheet table in an Excel file; and the single table identifier can be the name of the Sheet table. Optionally, the table storage identifier can be a mapping identifier of the single table identifier in the table file in the storage space. The table storage identifier has a unique corresponding relationship with the single page table in the table file. The preset separator can be a separator between the file identifier, the single table identifier and the table storage identifier of the table file which is preset. Optionally, the preset separator can include at least one character. For example, the preset separator can be “|@|”.

[0051] Specifically, the single table identifier of the single page table in the table file can be obtained, and the single table identifier is directly determined as the structure identifier. The file identifier, the preset separator, the single table identifier, the preset separator and the table storage identifier of the table file can be spliced in sequence to obtain the metadata of the table file corresponding to the single page table. Optionally, the sequence of the file identifier, the structure identifier and the table storage identifier can be adjusted, which is not described herein. Optionally, the metadata of the table file corresponding to each single page table can be determined in sequence to obtain the metadata of the table file corresponding to all single page tables of the table file.

[0052] The single table identifier of the single page table in the table file is directly determined as the structure identifier, the determination process of the structure identifier is simplified, the preset delimiter is used for splicing the identifiers, the content of the metadata is enriched, the generation efficiency of the metadata of the table file is further improved, meanwhile, the preset delimiter is used for distinguishing different identifiers, and the different identifiers are stored together, the mapping relationship between the identifiers included in the metadata is simplified, and the accuracy of the content of the table file in the storage space is further improved.

[0053] In an optional embodiment of the present application, the column identifiers in each single page table are acquired and determined as structure identifiers, the column storage identifiers of the table file are generated, the first list corresponding to the table storage identifier is established in the storage space, the column storage identifiers of the table file and the column identifiers are spliced by using the preset delimiter and stored in the first list corresponding to the table storage identifier, and the first list corresponding to the table storage identifier is added to the metadata of the table file.

[0054] The single page table can include at least one column. The column identifier can be used to identify the column in the single page table of the table file. The column storage identifier can be the mapping identifier of the column in the single page table in the storage space. The column storage identifier has a unique corresponding relationship with the column in the single page table. The first list can be used to store the mapping relationship between the columns of the single page table corresponding to the table storage identifier and the column storage identifiers.

[0055] Specifically, the column identifier in the single page table can be acquired, and the column identifier in the single page table is directly determined as the structure identifier. The column storage identifiers of the table file can be automatically generated according to the preset rule. Optionally, the first list corresponding to the table storage identifier can be established in the storage space, and the first list and the table storage identifier are associatedly stored; or the table storage identifier can be added to the list name of the first list, and the first list corresponding to the table storage identifier is established in the storage space. The column identifier, the preset delimiter and the column storage identifier can be spliced in the order to obtain the splicing result corresponding to the single column in the single page table. The splicing result corresponding to each column in the single page table can be sequentially determined to obtain the splicing result corresponding to all columns of the single page table. The splicing result corresponding to all columns of the single page table can be stored in the first list corresponding to the table storage identifier. The first list corresponding to the table storage identifier can be added to the metadata of the table file.

[0056] The scheme simplifies the corresponding relationship between the columns in the same single-page table and the mapping relationship between the column identifier and the column storage identifier of a single column, and adds the first list corresponding to the table storage identifier to the metadata of the table file, so as to facilitate the determination of the column structure of the single-page table of the table file and further improve the synchronization efficiency of the table file.

[0057] In an optional embodiment of the present application, after the content stored in the table file is stored in correspondence with the metadata of the table file, the method further comprises: receiving a table data query request; generating a table data query statement according to the table data query request; extracting a file identifier to be queried from the table data query statement; querying a table storage identifier to be queried corresponding to the file identifier to be queried; replacing the file identifier to be queried with the table storage identifier to be queried in the table data query statement; and querying target table data corresponding to the table data query request in the storage space according to the table data query statement after the replacement.

[0058] The table data query request can be used to query table data. Optionally, the table data query request can include a file identifier to be queried. The table data query statement is used to query target table data. For example, the table data query statement can be a SQL (Structured Query Language, structured query language) statement. The file identifier to be queried can be used to identify a table file to be queried. For example, the file identifier to be queried can be the file name of the table file to be queried. The table storage identifier to be queried can be a table storage identifier having a mapping relationship with the file identifier to be queried.

[0059] Specifically, the table data query request triggered by the user can be received. The table data query statement (for example, a SQL statement) can be generated according to the file identifier to be queried contained in the table data query request. The file identifier to be queried can be extracted from the table data query statement. The table storage identifier to be queried can be queried in the storage space according to the file identifier to be queried and the mapping relationship between the file identifier and the table storage identifier. The file identifier to be queried can be replaced with the table storage identifier to be queried in the table data query statement. The target table data corresponding to the table data query request can be queried in the storage space according to the table data query statement with the table storage identifier after the replacement.

[0060] The scheme realizes replacement between the file identifier and the table storage identifier in the table data query statement through the mapping relationship between the file identifier and the table storage identifier, does not need the user to manually input the table storage identifier, can realize the query of the table data according to the table data query request, reduces the complexity of the table data query in the storage space, improves the convenience and the readability of the table data query, and improves the universality of the table data query.

[0061] In an optional embodiment of the present application, the target table data corresponding to the table data query request is queried in the storage space according to the replaced table data query statement, and is specifically embodied as follows: a new table file is generated, and at least one single-page table is generated in the new table file; and the queried target table data is filled in the at least one single-page table.

[0062] The new table file can be used for recording the target table data obtained by the query. The new table file can include at least one single-page table.

[0063] Specifically, the Apache POI (the Java API for Microsoft Documents) framework can be used to generate the new table file in the XSSFWorkbook mode, at least one single-page table is generated in the table file, and the queried target table data is filled in the at least one single-page table.

[0064] Optionally, if the table data query request is a detailed data query request, the queried target table data can be filled in the at least one single-page table in a detailed manner; if the table data query request is a statistical data query request, the queried target table data can be filled in the at least one single-page table in a summarized manner.

[0065] The scheme generates a new table file, generates at least one single-page table in the new table file, fills the queried target table data in the at least one single-page table, and generates a table data query result, thereby facilitating subsequent analysis and processing of the table data and further improving the processing efficiency of the table data.

[0066] In an optional embodiment of the present application, the storage space is specifically embodied as a memory storage space.

[0067] Exemplarily, the memory storage space can be a Redis memory storage space. Compared with other storage spaces, the memory storage space has the characteristic of fast data read-write speed.

[0068] The scheme specifically embodies the storage space as a memory storage space, greatly improves the read-write speed of the table data in the table file, and thereby improves the synchronization speed of the table file and the query speed of the table data.

[0069] Figure 2 A flow chart of a table data processing method is provided for an embodiment of the present application. The embodiment further refines the storing of the content in the table file into the storage space according to the table structure on the basis of the above-mentioned embodiment. Referring to Figure 2 The table data processing method shown in the figure comprises:

[0070] S210, acquiring a table file and a file identifier, and parsing the table file.

[0071] S220, generating a table storage identifier of the table file.

[0072] S230, acquiring a structure identifier in the content of the table file, and taking the structure identifier, the file identifier and the table storage identifier as metadata of the table file.

[0073] S240, dividing the content in the table file into at least one row.

[0074] Specifically, the content in the table file can be divided into at least one row according to the row number.

[0075] S250, using a preset separator to splice the different column values included in each row to obtain the row data of the row.

[0076] Each row of the table data can include at least one column value. The row data of the row can be used to store the different column values of the same row.

[0077] Specifically, for each row, the preset separator can be used to splice the different column values included in the row to obtain the row data of the row.

[0078] S260, establishing a second list corresponding to the table storage identifier in the storage space.

[0079] The second list can be used to store the content of the table file in the storage space. The second list corresponds to the table storage identifier.

[0080] Optionally, the second list corresponding to the table storage identifier can be established in the storage space, and the second list and the table storage identifier can be stored in association; or the table storage identifier can be added in the list name of the second list, and the second list corresponding to the table storage identifier can be established in the storage space.

[0081] S270, storing the row data of each row into the second list according to the row sequence of the table file.

[0082] Specifically, the row data of each row can be sequentially stored into the second list according to the row sequence of the table file.

[0083] S280, a second list corresponding to the table storage identifier is stored, and is added to the metadata of the table file.

[0084] S290, the content stored in the storage space in the table file is stored in correspondence with the metadata of the table file.

[0085] The technical scheme of the embodiment of the application obtains a table file and a file identifier, parses the table file, generates a table storage identifier of the table file, obtains a structure identifier in the content of the table file, and stores the structure identifier, the file identifier and the table storage identifier as metadata of the table file. The content in the table file is divided into at least one row, a preset separator is used, for each row, different column values included in the row are spliced to obtain row data of the row, a second list corresponding to the table storage identifier is established in the storage space, the row data of each row is stored in the second list in the row order of the table file, the second list corresponding to the table storage identifier is added to the metadata of the table file, the content stored in the storage space in the table file is stored in correspondence with the metadata of the table file, the same row different column values in the table file are spliced by using the preset separator to obtain the row data of the row, the mapping process of each data position in the content of the table file is simplified, the complexity of the content synchronization process of the table file is reduced, the synchronization efficiency and the synchronization accuracy of the content of the table file are improved, the correspondence between the content of the table file and the table storage identifier is determined through the correspondence between the table storage identifier and the second list, and the synchronization efficiency of the table file is further improved.

[0086] In an optional embodiment of the application, after the row data of each row is stored in the second list in the row order of the table file, the following steps are further included: in the second list, the primary key and the foreign key are detected; an associated file associated with the table file is obtained; in the stored content of the associated file, data corresponding to the primary key is queried; the data corresponding to the primary key is counted according to at least one statistical type to obtain primary key added data corresponding to each statistical type; data corresponding to the foreign key is queried in the associated file, and foreign key added data of the foreign key is determined; the primary key added data corresponding to each statistical type and the foreign key added data of each foreign key are added to the second list.

[0087] The primary key can be used to uniquely identify the row data in the second list. In the second list, there is no duplicate data in the column data corresponding to the primary key. The foreign key can be other data in the second list except the primary key. The associated file can be another file having an association relationship with the table file. Optionally, the content of the associated file can include the primary key of the table file, the foreign key of the table file, the primary key of the second list, or the foreign key of the second list. The storage content of the associated file can be the content stored in the storage space by the associated file. The statistical type can include quantity, maximum value, minimum value, and mean value. The primary key added data can be data obtained after the data corresponding to the primary key is counted according to the statistical type. The foreign key added data can be data of other associated attributes corresponding to the foreign key in the associated file.

[0088] Specifically, in the second list, the primary key and the foreign key can be detected according to whether the row data in the second list can be uniquely identified. The associated file associated with the table file can be obtained according to the primary key and the foreign key. The data corresponding to the primary key can be queried in the storage content of the associated file according to the content of the primary key. The primary key added data corresponding to the statistical type can be determined by counting the data corresponding to the primary key according to the statistical type. The data corresponding to the foreign key can be queried in the associated file according to the content of the foreign key, and the foreign key added data of the foreign key can be determined. The primary key added data corresponding to the statistical type and the foreign key added data of each foreign key can be added to the second list.

[0089] The scheme detects the primary key in the second list, obtains the associated file associated with the table file, queries the data corresponding to the primary key in the storage content of the associated file, counts the data corresponding to the primary key according to at least one statistical type, obtains the primary key added data corresponding to each statistical type, adds the primary key added data corresponding to the statistical type to the second list, detects the foreign key in the second list, obtains the associated file associated with the table file, queries the data corresponding to the foreign key in the associated file, determines the foreign key added data of the foreign key, and adds the foreign key added data of each foreign key to the second list. The content of the second list is further expanded, the data corresponding to the primary key in the storage content of the associated file is pre-counted, and the data corresponding to the foreign key in the storage content of the associated file is pre-acquired, which enriches the content of the table file in the storage space, improves the comprehensiveness of the data in the storage space, avoids content analysis based on multiple files (i.e., the table file and the associated file), facilitates further analysis of the content of the table file and the associated file, and further improves the efficiency and accuracy of the table file analysis and processing.

[0090] In an alternative embodiment of the present application, the primary key addition data corresponding to each statistical type and the foreign key addition data of each foreign key are added to the second list, which is embodied as follows: generating a first column identifier according to the statistical type; adding the first column identifier and the primary key addition data corresponding to the statistical type as the same column data to the second list; generating a second column identifier according to the associated file corresponding to the foreign key; and adding the second column identifier and the foreign key addition data of the foreign key as the same column data to the second list.

[0091] The first column identifier can be an identifier field of the primary key addition data. The first column identifier can be used to identify the category of the corresponding primary key addition data. Alternatively, the content of the first column identifier can include the primary key and / or the statistical type. For example, the statistical type can be quantity; and the first column identifier can be primary key calculation result 1 (quantity ranking). The second column identifier can be an identifier field of the foreign key addition data. The second column identifier can be used to identify the category of the corresponding foreign key addition data. Alternatively, the content of the second column identifier can include the foreign key and / or the associated attribute in the associated file. For example, the associated attribute in the associated file of the foreign key can include name and rating; accordingly, the second column identifier can include foreign key associated attribute 1 and foreign key associated attribute 2; and the second column identifier can also include foreign key name and foreign key rating.

[0092] Specifically, the first column identifier can be generated according to the statistical type and the primary key. The first column identifier can be added to the second list as an attribute field of the primary key addition data, and the primary key addition data corresponding to the statistical type can be added to the second list as the same column data of the attribute field in the order of the primary key. The second column identifier can be generated according to the associated attribute in the associated file corresponding to the foreign key. The second column identifier can be added to the second list as an attribute field of the foreign key addition data, and the foreign key addition data of the foreign key can be added to the second list as the same column data of the attribute field in the order of the foreign key.

[0093] The present scheme generates the first column identifier according to the statistical type, adds the first column identifier and the primary key addition data corresponding to the statistical type as the same column data to the second list, uses the first column identifier to clearly define the relationship between the primary key addition data and the statistical type, generates the second column identifier according to the associated file corresponding to the foreign key, adds the second column identifier and the foreign key addition data of the foreign key as the same column data to the second list, and uses the second column identifier to clearly define the corresponding relationship between the foreign key addition data and the associated file. In this way, the content comprehensiveness of the table file in the storage space is improved, and the processing efficiency of the table file is further improved.

[0094] In an alternative embodiment of the present application, the primary key and the foreign key are detected in the second list, which is embodied as follows: whether the data in the same column in each row of data is repeated and non-empty is detected to obtain target column data; the primary key of the table file is determined according to the target column data; and the value of the data in the same column is determined as the foreign key of the primary key in the column data other than the primary key.

[0095] The target column data can be the alternative column data corresponding to the primary key of the table file. It can be understood that the target column data can include the column data corresponding to the primary key of the table file. Alternatively, the data in the same column in each row of the target column data is non-empty and does not exist repeated data. Alternatively, the number of columns of the target column data can be at least one.

[0096] Specifically, whether the data in the same column in each row of data is repeated and non-empty can be detected in the second list, and the column data in which the data in the same column in each row of data is non-empty and does not exist repeated data is determined as the target column data. Each target column data is screened to determine a target column data, and the column value of the target column data is determined as the primary key of the table file. The value of the data in the same column can be determined as the foreign key of the primary key in the column data corresponding to the primary key.

[0097] The present scheme determines the target column data by detecting whether the data in the same column in each row of data is repeated and non-empty, determines the primary key of the table file according to the target column data, and determines the value of the data in the same column as the foreign key of the primary key in the column data corresponding to the primary key. The primary key and the foreign key of the table file are determined in a more simple way, which improves the efficiency of determining the primary key and the foreign key, and further improves the synchronization efficiency of the table data.

[0098] In an alternative embodiment of the present application, the primary key of the table file is determined according to the target column data, which is embodied as follows: when the number of columns of the target column data is one, the target column data is determined as the primary key of the table file; when the number of columns of the target column data is at least two, each target column data is spliced according to the row to form key column data with the same number of rows as the target column data, and is added to the second list; and the key column data is determined as the primary key of the table file.

[0099] The primary key of the table file has uniqueness. The column number of the target column data is one, and it can be understood that the table file is a single primary key file, and the target column data can be directly determined as the primary key of the table file. The column number of the target column data is at least two, and it can be understood that the table file is a multi-primary key file, and the target column data can be further processed to determine the primary key of the table file. The key column data can be column data obtained by splicing at least two target column data in the same row. The number of rows of the key column data is the same as the number of rows of the target column data. The key column data includes data of the same row of each target column data.

[0100] Specifically, when the column number of the target column data is one, the target column data can be directly determined as the primary key of the table file. When the column number of the target column data is at least two, each target column data can be spliced according to the row to obtain associated column data with the same number of rows as the target column data, and the key column data can be added to the second list. The key column data is determined as the primary key of the table file.

[0101] The scheme determines the target column data as the primary key of the table file when the column number of the target column data is one, and splices each target column data according to the row to form key column data with the same number of rows as the target column data and adds it to the second list when the column number of the target column data is at least two. The key column data is determined as the primary key of the table file. Different column numbers of the target column data are processed respectively, which avoids determining the primary key of the table file without difference for different column numbers of the target column data, and improves the flexibility of determining the primary key of the table file while considering the uniqueness of the primary key of the table file.

[0102] It should be noted that the parts not described in detail in the embodiments of the present application can be referred to the description of the foregoing embodiments.

[0103] Figure 3 A flowchart of a table data processing method provided in an embodiment of the present application. The embodiment of the present application provides a preferred embodiment of a table data processing method. Referring to the table data processing method shown in Figure 3 The table data processing method comprises the following steps.

[0104] S310, synchronizing the data file to the memory.

[0105] The data file can be a table file. For example, an Excel file or a CSV file. To analyze the data file, the data file is first synchronized to the in-memory storage space. The present solution can include a file management module. In the file management module, the data file can be uploaded to the server. Optionally, the data file can be uploaded individually or in batches. After uploading the data file, the upload time of the data file can be viewed. When the data file is updated, the data file with the same file name can be uploaded repeatedly, and the system can automatically determine whether the data file already exists. If the data file already exists, the system automatically performs a replacement, and after the data file is replaced, the system updates the upload time of the data file. Unwanted data files can be deleted in the system. The data file uploaded to the server needs to be synchronized to the in-memory storage space, and the present solution includes an automatic data synchronization module. The present solution can use Redis as an in-memory database (i.e., in-memory storage space), or other types of in-memory databases can be selected. The system can set a timing task to scan the data file on the server at a short time interval periodically. When a user uploads a new data file, the timing task can read the new data file and write it to Redis. When a user modifies a data file, the timing task can read the modified data file, the system first deletes the old data in Redis, and then writes the new data to Redis. When a user deletes a data file, the system synchronously deletes the data in Redis. The data synchronization situation and the data synchronization status of each file can be displayed on the system page in real time. The display content of the data synchronization situation can include the number of synchronized files, the number of files in synchronization, and the number of unsynchronized files. The data synchronization status can include synchronized, in synchronization, and unsynchronized.

[0106] Synchronizing the data file to the in-memory storage space can include synchronizing the metadata of the data file and synchronizing the row data of the data file. Because the format of the CSV file is relatively simple, the system can regard the CSV file as a Sheet table (i.e., a single-page table) in the Excel file.

[0107] The present solution can regard the Sheet table of the Excel file as a single-page table, and the data file can be read and written through the Sheet table. Figure 4The method shown saves the sheet mapping relationships (i.e., the metadata of the table files) to an in-memory database. Specifically, a list called `tableList` can be set up in the Redis in-memory database to store the sheet mapping relationships. Each row in `tableList` represents a sheet mapping (i.e., the metadata of a table file). Each row of the sheet mapping can be obtained by concatenating three values ​​(i.e., file identifier, single-sheet identifier, and table storage identifier) ​​using the preset separator `|@|`. The first value can be the filename of the Excel file (i.e., the file identifier of the table file); the second value is the sheet name in the Excel file (i.e., the single-sheet identifier of the table file); and the third value can be an automatically generated English table name (i.e., the table storage identifier). This English table name is unique within the system.

[0108] Optionally, the metadata also includes column identifiers for each sheet. This can be achieved through methods such as... Figure 5 The method shown saves the column mapping relationship of a single-page table (i.e., the mapping relationship between column identifiers and column storage identifiers) to an in-memory database. For example... Figure 5 As shown, each English table name (i.e., table storage identifier) ​​has a first list in Redis to store the column mapping relationships of that table file. Specifically, each row in the tableEnColList collection represents a column mapping. Each column mapping can be obtained by concatenating two values ​​(i.e., column identifier and column storage identifier) ​​using the preset separator |@|. In a column mapping, the first value can be the column name (i.e., column identifier); the second value can be an English column name automatically generated internally by the system (i.e., column storage identifier).

[0109] Optionally, it can be done through, for example Figure 6 The method shown saves the contents of the table file to an in-memory database. Specifically, for each English table name (i.e., table storage identifier), a second list is set up in Redis to store the contents of that table file. Each row in the tableEnDataList collection corresponds to a row in the table file in sequence. The column values ​​of each column belonging to the same row in the table file can be concatenated using the preset delimiter |@| to obtain each row in the tableEnDataList collection.

[0110] S320: Automatically create a table based on the table header.

[0111] After opening the query page, the Spark system starts the Spark JDBC (Java DataBase Connectivity). After starting the connection, the system reads the data in Redis for initialization loading. Using the relevant Java interface of Redis, all the data of the tableEnDataList collection can be loaded into Spark, and a Spark RDD (Resilient Distributed Dataset) is obtained, each RDD corresponding to a single-page table. The metadata tableList and tableEnColList in Redis can be read. Among them, tableList is the table-level information (i.e. the metadata of the table file); tableEnColList is the column information (i.e. the column mapping of the single-page table). According to the column information of each table, each RDD (i.e. the single-page table with the content of the table file) can be converted into a Spark Data Frame (uniform structured interface object) (i.e. an object generated by combining the column name and the content of the table file), and the column name used for automatic table building is a unique English column name (i.e. column storage identifier) automatically generated by the system. According to the table-level information (i.e. the metadata of the table file), each Spark DataFrame (uniform structured interface object) can be converted into a temporary table of Spark SQL. The method for creating a temporary table uses the create Or Replace Temp View() interface of DataFrame (uniform structured interface object). The table name used for automatic table building is a unique English table name (i.e. table storage identifier) automatically generated by the system, so as to ensure that the created temporary table does not conflict.

[0112] S330, automatically identifying the primary key and the foreign key.

[0113] If the data file is not a single-primary-key file (i.e. the number of target column data is at least two), a column (i.e. key column data) can be added to the data file, the combined primary key (i.e. the splicing result of the target column data according to the row) is determined as the key column data, and the key column data is determined as the primary key of the data file.

[0114] Based on the second list that has been created, each column in the second list is judged for repetition and non-empty. If a column meets the requirements of the primary key (i.e. non-empty and no repeated data), the column data is determined as the primary key. Finally, each table identifies a primary key.

[0115] Each non-primary-key column is judged one by one. If the value range of a certain non-primary-key column is all within a certain primary key, the non-primary-key column is taken as a foreign key, and the foreign key points to the primary key.

[0116] S340, automatically creating a wide table.

[0117] Based on the second list, the primary key and the foreign key that have been created, a wide table can be automatically created (expanding the second list).

[0118] Figure 7 is a schematic diagram of the process of creating a wide table. On the left is the source table (i.e. the second list). The source table includes 5 fields; field 1 is the primary key, and field 3 is the foreign key. On the right is the wide table created after (i.e. the expanded second list), which has 4 additional fields based on the original 5 fields. Specifically, the process of creating a wide table is as follows: when a primary key of a second list has a foreign key corresponding to the storage content of other tables (i.e. associated files), such as field 1, the information of the foreign key is automatically calculated and associated to the second list to generate a wide table.

[0119] For example, the calculation process of the primary key calculation result 1 (quantity ranking) is as follows: first, the number of records of the primary key value (i.e. field 1) in other tables (i.e. the storage content of associated files) is calculated, and then the number of records is ranked with other primary key values to obtain the quantity ranking.

[0120] For example, the calculation process of the primary key calculation result 2 (average ranking) is as follows: first, the average value corresponding to the primary key value (i.e. field 1) in other tables (i.e. the storage content of associated files) is calculated, and then the average value is ranked with other primary key values to obtain the average ranking.

[0121] For example, when a table has a foreign key (e.g. field 3), the relevant primary key table field corresponding to the foreign key can be automatically associated to form a wide table. For example, the number is a foreign key, and the name and customer attributes corresponding to the number will be automatically associated.

[0122] S350, SQL field conversion.

[0123] After the initialization data is loaded, SQL statements can be written to query table data. The SQL language supported by the present scheme is Spark SQL syntax. Spark SQL is a general-purpose language.

[0124] The present scheme is optimized in SQL statement writing: for table names and field names, the user only needs to input the Chinese name in the data file, and does not need to input the English name generated internally by the system (i.e. table storage identifier or column storage identifier). Optionally, the Chinese table name can be placed between the curly braces of "${Chinese table name}"; the Chinese field name can be placed between the square brackets of "$[Chinese field name]".

[0125] After the system receives the query SQL, the Chinese table name and field name are automatically replaced. The system can find the corresponding English name of the Chinese table name and field name according to the mapping relationship in Reids. The system can replace the ${Chinese table name} with the English table name; and replace the [Chinese field name] with the English field name.

[0126] S360, query result export.

[0127] The Apache POI framework can be used to export the query result as an Excel file. Optionally, the XSSFWorkbook method can be used to export the query result. Specifically, the query result export process is as follows:

[0128] 1. Create an Excel file.

[0129] Specifically, an Excel file can be created, and at least one Sheet table in the Excel file can be created. Optionally, the Sheet name can be input.

[0130] 2. Create a row in the Sheet table.

[0131] It can be judged whether the first row has been created. If yes, the second row is created; otherwise, the first row is created. This is to avoid covering the previously created data.

[0132] 3. Create a cell.

[0133] 4. Create cell content.

[0134] 5. Export the Excel file.

[0135] This scheme can support the export of detailed data and statistical data: when the query SQL statement is a detailed data query statement, the exported result can be an Excel file of detailed data; when the query SQL statement is a statistical data query statement, the exported result can be a statistical data Excel file. After exporting the Excel file, the exported result can be further analyzed.

[0136] The scheme has great universality by lowering the threshold of data file analysis, and users do not need to learn various Excel functions and pivot table knowledge, and do not need to master specific programming languages such as VBA code, but only need to master general SQL language to analyze data files, and the SQL statement is optimized, only the Chinese name in the data file needs to be input, and the system will automatically replace it into English name, and the English name generated by the system (i.e. table storage identifier or column storage identifier) does not need to be input, further improving the convenience and readability; at the same time, the functions of Excel functions and pivot table have certain limitations and cannot meet some complex analysis requirements, and the functions of Excel data analysis are greatly improved by using Spark SQL syntax; the scheme also provides a convenient data file import function, users can upload data files through the page, the system can automatically identify whether the file is new or modified, and the system can automatically import the new or modified data file into the database; in addition, the primary key and foreign key are automatically identified, the content of the table file is facilitated to be queried, the important information is extracted in advance by automatically creating a wide table, the available derivative information is provided in advance, a large amount of unnecessary manual correlation analysis is reduced, and the analysis speed of large files is accelerated; the metadata and content in the data file are stored in the Redis in-memory database in advance, based on the pure in-memory database, the rapid access to data is realized, when the table data is queried, the system can directly query the data in Redis, and the data does not need to be read from the data file, the data reading speed is improved, and support is provided for the analysis of large files; finally, the SQL analysis steps are simplified, the user does not need to build a table in advance, and the automatic table building function can be realized, compared with other data file analysis systems, the metadata (i.e. configuration information) of the data file needs to be defined in advance, the table English name and column English name can be automatically generated according to the column information in the data file, the table can be automatically built according to the generated table English name and column English name, and the related configuration information does not need to be provided by the user.

[0137] It should be noted that the parts not described in detail in the embodiments of the application can refer to the descriptions of the foregoing embodiments.

[0138] Figure 8 A structural schematic diagram of a table data processing device provided by the embodiments of the application. The embodiments can be applicable to the case of processing table data, the device can execute the table data processing method, the device can be realized in the form of hardware and / or software, and the table data processing device can be configured in an electronic device. Referring to Figure 8The table data processing apparatus shown comprises a table file acquisition module 810, a table storage identification generation module 820, a structure identification acquisition module 830, a table file content storage module 840 and a table file storage module 850. The table file acquisition module 810 is configured to acquire a table file and a file identification, and to parse the table file; the table storage identification generation module 820 is configured to generate a table storage identification of the table file; the structure identification acquisition module 830 is configured to acquire a structure identification in the content of the table file, and to take the structure identification, the file identification and the table storage identification as metadata of the table file; the table file content storage module 840 is configured to store the content in the table file into a storage space according to a table structure; and the table file storage module 850 is configured to store the content stored in the storage space in the table file and the metadata of the table file correspondingly.

[0139] The technical scheme of the embodiment of the application realizes automatic processing of the table file by acquiring a table file and a file identification, parsing the table file, generating a table storage identification of the table file, acquiring a structure identification in the content of the table file, taking the structure identification, the file identification and the table storage identification as metadata of the table file, storing the content in the table file into a storage space according to a table structure, and storing the content stored in the storage space in the table file and the metadata of the table file correspondingly. The mapping relationship between the structure of the table file and the table storage identification in the storage space is established by determining the metadata of the table file. The structure analysis of the table file and the structure synchronization of the table file are realized based on the structure identification. Meanwhile, the structure synchronization of the table file is realized by the structure identification. Compared with other manners, the mapping relationship between data is simpler, the synchronization efficiency of the table file is higher, the storage efficiency of the table file is improved, and the accuracy of the table file stored in the storage space is ensured.

[0140] In an optional embodiment of the application, the structure identification acquisition module 830 comprises a single table identification acquisition unit configured to acquire a single table identification of a single page table in the table file, and to determine the single table identification as the structure identification; and a metadata generation unit configured to splice the file identification, the single table identification and the table storage identification of the table file by using a preset separator, and to form the metadata of the table file.

[0141] In an optional embodiment of the present application, the structure identifier obtaining module 830 further comprises: a column identifier obtaining unit, configured to obtain column identifiers in each single-page table, and determine the column identifiers as the structure identifiers; a column storage identifier generating unit, configured to generate column storage identifiers of the table file; a first list establishing unit, configured to establish a first list corresponding to the table storage identifier in the storage space; a column identifier splicing unit, configured to splice the column storage identifiers of the table file and the column identifiers by using a preset delimiter, and store the spliced result in the first list corresponding to the table storage identifier; and a first list adding unit, configured to add the first list corresponding to the table storage identifier to the metadata of the table file.

[0142] In an optional embodiment of the present application, the table file content storage module 840 comprises: a table file content dividing unit, configured to divide the content in the table file into at least one row; a row data determining unit, configured to splice different column values included in each row by using a preset delimiter, and obtain row data of the row; a second list establishing unit, configured to establish a second list corresponding to the table storage identifier in the storage space; and a row data storage unit, configured to store the row data of each row in the second list according to the row sequence of the table file.

[0143] In an optional embodiment of the present application, after storing the row data of each row in the second list according to the row sequence of the table file, the table file content storage module 840 further comprises: a primary key detecting unit, configured to detect the primary key and the foreign key in the second list; an associated file obtaining unit, configured to obtain an associated file associated with the table file; a primary key data querying unit, configured to query data corresponding to the primary key in the storage content of the associated file; a primary key added data determining unit, configured to determine primary key added data corresponding to each statistical type by performing statistics on the data corresponding to the primary key according to at least one statistical type; a foreign key added data determining unit, configured to query data corresponding to the foreign key in the associated file, and determine foreign key added data of the foreign key; and a second list data adding unit, configured to add the primary key added data corresponding to each statistical type and the foreign key added data of each foreign key to the second list.

[0144] In an optional embodiment of the present application, the second list data adding unit comprises: a first column identifier generating subunit, configured to generate a first column identifier according to the statistical type; a same column data adding subunit, configured to add the first column identifier and the primary key added data corresponding to the statistical type to the second list as same column data; a second column identifier generating subunit, configured to generate a second column identifier according to the associated file corresponding to the foreign key; and a second list data adding subunit, configured to add the second column identifier and the foreign key added data of the foreign key to the second list as same column data.

[0145] In an alternative embodiment of the present application, the primary key detection unit comprises: a target column data determination subunit configured to detect whether the data in the same column in each row of data is repeated and non-empty, and obtain target column data; a table file primary key determination subunit configured to determine the primary key of the table file according to the target column data; and a table file foreign key determination subunit configured to determine, in the column data other than the primary key, the value of the data in the same column as the column data corresponding to the primary key as a foreign key pointing to the primary key.

[0146] In an alternative embodiment of the present application, the table file primary key determination subunit is specifically configured to: when the number of columns of the target column data is one, determine the target column data as the primary key of the table file; when the number of columns of the target column data is at least two, splice each target column data according to the row in which the target column data is located to form key column data with the same number of rows as the target column data, and add the key column data to the second list; and determine the key column data as the primary key of the table file.

[0147] In an alternative embodiment of the present application, after the table file storage module 850 stores the content stored in the table file in correspondence with the metadata of the table file, the device further comprises: a table data query request receiving module configured to receive a table data query request; a table data query data generation module configured to generate a table data query statement according to the table data query request; a file identifier to be queried extraction module configured to extract a file identifier to be queried from the table data query statement; a table storage identifier query module configured to query a table storage identifier to be queried corresponding to the file identifier to be queried according to the file identifier to be queried; a table storage identifier replacement module configured to replace the file identifier to be queried with the table storage identifier to be queried in the table data query statement; and a target table data query module configured to query target table data corresponding to the table data query request in the storage space according to the table data query statement after the replacement.

[0148] In an alternative embodiment of the present application, the target table data query module comprises: a new table file generation unit configured to generate a new table file and generate at least one single-page table in the new table file; and a target table data filling unit configured to fill the target table data queried in the at least one single-page table.

[0149] In an alternative embodiment of the present application, the storage space comprises: a memory storage space.

[0150] The table data processing device provided in the embodiments of the present application can execute the table data processing method provided in any of the embodiments of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0151] It should be noted that the parts not described in detail in the embodiments of the present application can be referred to the descriptions of the foregoing embodiments.

[0152] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 9 A block diagram is shown that is suitable for implementing embodiments of the present invention. Figure 9 The device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0153] like Figure 9 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0154] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0155] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0156] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory (cache 32). Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 9 Not shown; usually referred to as a "hard drive"). Although Figure 9 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

[0157] Program / utility 40 having a set of program modules 42 can be stored in system memory 28 by way of example, such program modules 42 include an operating system, one or more application programs, other program modules, and program data, each or some combination thereof, which can include implementation of the network environment as in each of the above examples or some combination thereof. Program modules 42 generally carry out the functions and / or methodologies described in embodiments of the present application.

[0158] Electronic device 12 can also communicate with one or more external devices 14 such as a keyboard or pointing device, a display 24, etc.; one or more devices that enable a user to interact with electronic device 12; and / or one or more devices that enable electronic device 12 to communicate with one or more other computing devices. Such communication can occur via input / output (I / O) interface(s) 22. Still yet, electronic device 12 can communicate with one or more networks, such as one or more local area networks (LANs), wide area networks (WANs), and / or the Internet, through network adapter 20. As depicted, network adapter 20 communicates with the other components of electronic device 12 via bus 18. It should be appreciated that although not shown, other hardware and / or software modules could be used in conjunction with electronic device 12. Such as, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0159] Processing unit 16 executes various program applications and data processing by running programs stored in system memory 28, such as implementing the table data processing method provided by embodiments of the present application.

[0160] Embodiments of the present application also provide a computer readable storage medium, which stores a computer program (or computer executable instructions) for executing the table data processing method provided by embodiments of the present application when executed by a processor.

[0161] The computer storage medium of the embodiments of the present application can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples (non-exhaustive list) of the computer readable storage medium include: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device.

[0162] The computer readable signal medium can include a data signal propagated in baseband or propagated as a carrier wave in a propagated data signal, in which the computer readable program code is contained. Such propagated data signal can take many forms, including but not limited to electro-magnetic, optical or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport program for use by or in connection with an instruction execution system, apparatus or device.

[0163] The program code contained on the computer readable medium can be transmitted using any suitable medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination thereof.

[0164] The computer program code for carrying out operations of the embodiments of the present application can be written in one or more programming languages or combinations of languages including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages such as "C" or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). The embodiments of the present application can be implemented in a computing system that includes a back end component, for example, as a data server, or that includes a middleware component, for example, an application server, or that includes a front end component, for example, a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the application, or any combination of the above. Such a system can include any or all of the components of the embodiments of the present application, for example, the components of the system of the embodiments of the present application.

[0165] The embodiment of the present application further provides a computer program product comprising a computer program which, when executed by a processor, implements the table data processing method provided by any of the embodiments of the present application.

[0166] The computer program product can be written in any one of a number of programming languages or combinations thereof, including an object oriented programming language such as Java, Smalltalk, C++, or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0167] It should be noted that the above only describes the preferred embodiments of the present application and the principles of the applied technology. It is understood by those skilled in the art that the present application is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A method for processing tabular data, characterized in that, include: Obtain the table file and its file identifier, and parse the table file; The table storage identifier for generating the table file; Obtain the structure identifier from the content of the table file, and use the structure identifier, file identifier, and table storage identifier as the metadata of the table file; The contents of the table file are stored in the storage space according to the table structure; The content stored in the table file in the storage space is stored in correspondence with the metadata of the table file; The step of storing the contents of the table file into the storage space according to the table structure includes: Divide the contents of the table file into at least one row; Using a preset delimiter, for each row, the different column values ​​included in the row are concatenated to obtain the row data of the row; Establish a second list corresponding to the table storage identifier in the storage space; According to the row order of the table file, store the row data of each row into the second list; In the second list, primary keys and foreign keys are detected; Retrieve the associated file linked to the table file; Query the data corresponding to the primary key in the storage content of the associated file; According to at least one statistical type, the data corresponding to the primary key is statistically analyzed to obtain the primary key addition data corresponding to each statistical type. Query the data corresponding to the foreign key in the associated file, and determine the data to be added to the foreign key. Add the primary key data corresponding to each statistical type and the foreign key data corresponding to each foreign key to the second list; Add the second list corresponding to the table storage identifier to the metadata of the table file.

2. The method according to claim 1, characterized in that, The step of obtaining the structure identifier from the content of the table file and using the structure identifier, file identifier, and storage identifier as the metadata of the table file includes: Obtain the single-table identifier of the single-page table in the table file and determine it as the structure identifier; Using a preset delimiter, the file identifier, single table identifier, and table storage identifier of the table file are concatenated to form the metadata of the table file.

3. The method according to claim 2, characterized in that, Also includes: Obtain the column identifiers from each of the single-page tables and determine them as structure identifiers; Generate column storage identifiers for the table file; Establish a first list corresponding to the table storage identifier in the storage space; Using the preset delimiter, the column storage identifier and the column identifier of the table file are concatenated and stored in the first list corresponding to the table storage identifier; Add the first list corresponding to the table storage identifier to the metadata of the table file.

4. The method according to claim 1, characterized in that, The step of adding primary key data corresponding to each of the statistical types and foreign key data corresponding to each of the foreign keys to the second list includes: Generate a first column identifier based on the statistical type; Add the data from the primary key corresponding to the first column identifier and the statistical type as the same column data to the second list; Generate a second column of identifiers based on the associated files corresponding to the foreign keys; Add the foreign key data of the second column identifier and the foreign key data of the foreign key as the same column data to the second list.

5. The method according to claim 1, characterized in that, The detection of primary keys and foreign keys in the second list includes: Detect whether the data in the same column of each row is duplicated and not empty, and obtain the target column data; Determine the primary key of the table file based on the target column data; In the foreign key column data, the values ​​of the data in the same column that correspond to the primary key are determined as foreign keys pointing to the primary key.

6. The method according to claim 5, characterized in that, Determining the primary key of the table file based on the target column data includes: When the number of columns in the target column is one, the target column is determined as the primary key of the table file; When the number of columns in the target column data is at least two, the target column data is concatenated according to the row it is in to form key column data with the same number of rows as the target column data, and then added to the second list; The key column data is determined as the primary key of the table file.

7. The method according to claim 1, characterized in that, After storing the contents of the table file in correspondence with the metadata of the table file, the method further includes: Received a table data query request; Generate a table data query statement based on the table data query request; Extract the file identifier to be queried from the table data query statement; Based on the file identifier to be queried, query the corresponding table storage identifier to be queried; In the table data query statement, the file identifier to be queried is replaced with the table storage identifier to be queried; Based on the replaced table data query statement, query the target table data corresponding to the table data query request in the storage space.

8. The method according to claim 7, characterized in that, The step of querying the target table data corresponding to the table data query request in the storage space based on the replaced table data query statement includes: Generate a new table file, and generate at least one single-page table in the new table file; Fill the target table data to be queried into at least one single-page table.

9. The method according to claim 1, characterized in that, The storage space includes: memory storage space.

10. A tabular data processing device, characterized in that, include: The table file acquisition module is used to acquire the table file and its file identifier, and to parse the table file. The table storage identifier generation module is used to generate the table storage identifier of the table file. The structure identifier acquisition module is used to acquire the structure identifier in the content of the table file, and use the structure identifier, file identifier and table storage identifier as the metadata of the table file; The table file content storage module is used to store the content of the table file into the storage space according to the table structure; A table file storage module is used to store the contents of the table file stored in the storage space in correspondence with the metadata of the table file; The table file content storage module includes: A table file content partitioning unit is used to divide the content of the table file into at least one row; The row data determination unit is used to concatenate the different column values ​​included in each row using a preset delimiter to obtain the row data of the row; The second list creation unit is used to create a second list corresponding to the table storage identifier in the storage space. A row data storage unit is used to store the row data of each row into the second list according to the row order of the table file; A primary key detection unit is used to detect primary keys and foreign keys in the second list; The associated file acquisition unit is used to acquire the associated files associated with the table file; The primary key data query unit is used to query the data corresponding to the primary key in the storage content of the associated file; A primary key data addition determination unit is used to perform statistics on the data corresponding to the primary key according to at least one statistical type, and obtain primary key addition data corresponding to each statistical type; A foreign key data addition determination unit is used to query the data corresponding to the foreign key in the associated file and determine the foreign key data to be added to the foreign key. The second list data adding unit is used to add primary key data and foreign key data corresponding to each of the statistical types to the second list. The metadata adding unit is used to add the second list corresponding to the table storage identifier to the metadata of the table file.

11. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the tabular data processing method as described in any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the tabular data processing method as described in any one of claims 1-9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the tabular data processing method as described in any one of claims 1-9.

Images