Multi-source data granulation fusion and index classification layering processing method

Abstract

The invention discloses a multi-source data granulation fusion and index classification layering processing method, which comprises the following steps of: classifying a large number of diversified redundant indexes in a table, establishing a storage library, unifying various forms of tables into a standard table, identifying a title area and a numerical value area, and respectively extracting related attribute information of numerical values to form granulated and standardized data; and then, constructing an index access structure, according to personalized selection of a user, presenting numerical items, indexes and hierarchies corresponding to numerical values, index classification description and hierarchies, application scenes of index and index classification description and other implicit attributes in a spreadsheet form, so that accurate expression of two-dimensional table data is realized. According to the method, the multi-source data can be normalized and stored, so that theindex name uniqueness, the hierarchical relationship multisource, the index classification description multisource and the application scene multisource are realized, and a basis is provided for flexible, diversified and rapid presentation of the multi-source data.

Description

technical field [0001] The invention belongs to the technical field of big data processing, in particular to a multi-source data granular fusion and index classification and layered processing method. Background technique [0002] In the process of big data governance, the sources of data are diverse, and tabular data files are one of the common types. Typical two-dimensional spreadsheet data includes business (survey) and yearbooks. For business (survey) data, due to its diversity, most existing technologies use the form of sub-database storage, separate query, and separate expression. Unified solution; For yearbook data, although the existing technology has basically realized the unified storage and joint query of yearbook data, most of them adopt the direct combination of X axis and Y axis, that is, XY (or YX) combination index method, and there are index names Too long, irregular, inaccurate, poor readability and many other defects. [0003] The reasons are mainly refl...

AI Technical Summary

Problems solved by technology

[0004] 1) The diversity of multi-source data makes it difficult to deal with it uniformly

Since multi-source data comes from different sources, these data are of various types, and their index names and styles are different. Most of the existing technologies adopt different storage and expression schemes for different tabular data, and it is difficult to achieve unified storage and Express

[0005] 2) The redundancy of multi-source indicators makes it difficult to store them uniformly

Due to the diversity of two-dimensional spreadsheets, the same indicator name appears in different data tables at the same time. The existing technology uses separate storage methods, resulting in redundant indicators and cannot ensure the uniqueness of indicator names.

[0006] 3) The diversity of application scenarios of multi-source indicators makes it difficult to retrieve them uniformly

Because the existing processing technology does not store the hierarchical relationship and application scenarios between indicators and indicator classification descriptions, it is impossible to perform unified retrieval for different application scenarios of multi-source indicators

[0007] 4) The direct combination of XY (or YX) is difficult to accurately express the meaning of the data

Due to the defects in the storage method of the existing processing technology, the indicators are presented in the form of combined indicators. There are many defects such as too long indicator names, irregularities, inaccuracies, and poor readability. It is difficult to accurately express the meaning of the data.

Images