Mass heterogeneous multi-dimensional data acquisition-oriented quality big data system
A multi-dimensional data and big data technology, applied in the field of data processing, can solve the problems of incomplete data and inaccurate quality data analysis.
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment 1
[0042] This embodiment provides a quality big data system for massive heterogeneous multi-dimensional data collection, such as figure 1 shown, including:
[0043] Massive heterogeneous multi-dimensional data collection module, used to collect massive heterogeneous quality data;
[0044] The quality data governance module is used for data governance of massive heterogeneous quality data collected;
[0045] The ultra-large-scale quality data storage module is used to store massive heterogeneous quality data after data governance;
[0046] The quality data intelligent diagnosis and analysis module is used to perform data intelligent diagnosis and analysis on massive heterogeneous quality data after data governance through various data models;
[0047] The quality data multi-dimensional dynamic perception module is used to visualize and display the mass heterogeneous quality data after data management according to the intelligent diagnosis results of the data;
[0048] The spec...
Embodiment 2
[0054] On the basis of the above-mentioned embodiment 1, this embodiment provides a quality big data system, and the quality data management module includes:
[0055] A data disassembly unit, configured to intelligently disassemble complex information in the massive heterogeneous quality data;
[0056] The data processing unit is used to standardize the disassembled complex information according to the preset data association and intelligent matching methods;
[0057] The data cleaning unit is used to clean the standardized complex information.
[0058] The beneficial effects of the above technical solutions are: data governance solves the problems of inconsistent standards, data redundancy, missing, conflicts, etc., ensures the shareability, high quality, and consistency of quality data, and ensures the accuracy and efficiency of quality big data analysis. The data governance module, through data management, intelligently disassembles complex information, cleans data through...
Embodiment 3
[0060] On the basis of the above-mentioned embodiment 2, this embodiment provides a quality big data system, and when the data cleaning unit cleans the standardized data, it also includes:
[0061] Obtaining the heterogeneous multidimensional quality data, and horizontally segmenting the heterogeneous multidimensional quality data according to preset data disassembly rules, to obtain N horizontal data subsets;
[0062] At the same time, longitudinally segmenting the heterogeneous multi-dimensional quality data to obtain M longitudinal data subsets;
[0063] Among them, the value of N is the same as M;
[0064] determining the degree of coupling and similarity between the subset of horizontal data and the subset of longitudinal data;
[0065] sorting the horizontal data subsets and the vertical data subsets according to the degree of importance according to the coupling degree and similarity through a preset algorithm, and assigning corresponding labels to obtain the data to be ...
PUM
Login to View More Abstract
Description
Claims
Application Information
Login to View More