A multi-source data granular fusion and index classification and hierarchical processing method

Abstract

The invention discloses a multi-source data granulation fusion and index classification and layering processing method, by classifying a large number of diverse and redundant indexes in the table, and establishing a storage library, and then unifying various forms of tables into a standard table , Identify the title area and the value area, and extract the relevant attribute information of the value respectively, so as to form granular and standardized data; then, by constructing the index index structure, according to the user's personalized choice, the value item and the value are presented in the form of a spreadsheet Corresponding indicators and levels, indicator classification descriptions and levels, indicators and indicator classification description application scenarios and other hidden attributes, to achieve accurate expression of two-dimensional tabular data. The present invention can normalize and store multi-source data, so that the uniqueness of index names, multi-sources of hierarchical relationships, multi-sources of index classification descriptions, and multi-sources of application scenarios provide flexibility, diversity, and Rapid rendering provides the foundation.

Description

technical field [0001] The invention belongs to the technical field of big data processing, in particular to a multi-source data granular fusion and index classification and layered processing method. Background technique [0002] In the process of big data governance, the sources of data are diverse, and tabular data files are one of the common types. Typical two-dimensional spreadsheet data includes business (survey) and yearbooks. For business (survey) data, due to its diversity, most existing technologies use the form of sub-database storage, separate query, and separate expression. Unified solution; For yearbook data, although the existing technology has basically realized the unified storage and joint query of yearbook data, most of them adopt the direct combination of X axis and Y axis, that is, XY (or YX) combination index method, and there are index names Too long, irregular, inaccurate, poor readability and many other defects. [0003] The reasons are mainly refl...

AI Technical Summary

Problems solved by technology

[0004] 1) The diversity of multi-source data makes it difficult to deal with it uniformly

Since multi-source data comes from different sources, these data are of various types, and their index names and styles are different. Most of the existing technologies adopt different storage and expression schemes for different tabular data, and it is difficult to achieve unified storage and Express

[0005] 2) The redundancy of multi-source indicators makes it difficult to store them uniformly

Due to the diversity of two-dimensional spreadsheets, the same indicator name appears in different data tables at the same time. The existing technology uses separate storage methods, resulting in redundant indicators and cannot ensure the uniqueness of indicator names.

[0006] 3) The diversity of application scenarios of multi-source indicators makes it difficult to retrieve them uniformly

Because the existing processing technology does not store the hierarchical relationship and application scenarios between indicators and indicator classification descriptions, it is impossible to perform unified retrieval for different application scenarios of multi-source indicators

[0007] 4) The direct combination of XY (or YX) is difficult to accurately express the meaning of the data

Due to the defects in the storage method of the existing processing technology, the indicators are presented in the form of combined indicators. There are many defects such as too long indicator names, irregularities, inaccuracies, and poor readability. It is difficult to accurately express the meaning of the data.

Images