Data analysis method based on big data

A data analysis and big data technology, applied in the field of data processing, can solve problems such as slowing down of database processing speed, increase of database index, and no solution proposed

Inactive Publication Date: 2015-04-29
SICHUAN ZHONGKE TENGXIN TECH
View PDF3 Cites 39 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] (2) The amount of data can only be reduced to the acceptable range of the memory by sampling, and the sampling of data usually leads to the loss of data information;
[0006] (3) The ever-increasing data can easily lead to the continuous increase of the database index, and the hysteresis of the index increase can easily lead to a decrease in the processing speed of the database
[0007] Therefore, the database system cannot meet the explosive growth of the current large data volume in terms of data processing efficiency and the amount of data that can be processed.
For the above-mentioned problems existing in related technologies, effective solutions have not yet been proposed

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data analysis method based on big data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0020] The following and accompanying appendices illustrating the principles of the invention Figure 1 A detailed description of one or more embodiments of the invention is provided together. The invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details.

[0021] The invention proposes an extensible data analysis and mining method. By defining reusable data mining components, multiple data sources can be reused to improve the agility of the data mining process.

[0022] figure 1 It is a...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data analysis method based on big data. The data analysis method includes steps of receiving user-defined data mining process and component information to realize graphical data flow processing; generating codes conforming to Hadoop standards, converting a user-defined data mining process model into operable codes on the Hadoop; connecting data mining components to form the data mining process; utilizing a frame provided by Hadoop as a data mining process executing platform, submitting the codes to the executing frame and applying parallel computing capability of a cloud platform to parallelize the mining process. By the defined component model, the data mining process can be quickly and visually defined by users, and data access to various data storage systems is realized.

Description

technical field [0001] The invention relates to data processing, in particular to a big data analysis and mining method. Background technique [0002] Facing the rapidly growing amount of data, how to discover useful information from the data has become a problem faced by most current data mining systems. The characteristics that the data analysis and mining system of large data sets need to have include (1) data adaptability: the system can accept various types of data, and avoid the system's strong requirements on the type, structure and data integrity of the stored data, so as to avoid Problems caused by the strong data requirements of the usual data warehouse; (2) agility: the system can adapt to the application scenarios with increasing data and frequent updates; (3) depth of analysis: provide multi-angle and multi-faceted analysis of data analysis, Complex probability statistics and machine learning algorithms can be easily added to meet the needs of multi-type data a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/285
Inventor 谢叔阳
Owner SICHUAN ZHONGKE TENGXIN TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap