Hadoop-based network data mining and analysis platform and method thereof

A network data and platform technology, applied in network data retrieval, database distribution/replication, electronic digital data processing, etc., can solve the scalability barriers of traditional relational data management systems, storage performance and scalability constraints restrict the effective application of data bottlenecks, To deal with problems such as new data power failure, to achieve the effect of reducing the amount of model calculation, easy portability, and improving algorithm efficiency
CN106934014AActive Publication Date: 2017-07-07南方电网互联网服务有限公司
8 Cites 24 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
南方电网互联网服务有限公司
Publication Date
2017-07-07

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a Hadoop-based network data mining and analysis platform and a method thereof. The platform comprises a data collection layer, a data storage layer, a business application layer and a user layer, wherein the data collection layer adopts a distributed directional collection system architecture, collects original network data by taking terminal sites in different networks as basic task units of network data collection, and transmits the data to the data storage layer in a gathering manner, wherein each basic task unit adopts independent collection rule and policy; the data storage layer is used for finishing gathering, storage and original processing of the original network data and providing different types of function call services; the data storage layer is realized by adopting a Hadoop framework; and the business application layer is used for calling and analyzing the network data processed by the data storage layer to realize separation of a public component and a personalized business application component, and transmitting a result after network data analysis to the user layer for performing real-time display.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention belongs to the field of network data processing, and in particular relates to a Hadoop-based network data mining and analysis platform and a method thereof. Background technique

[0002] In recent years, "big data" has become a hot spot in the scientific and technological circles and business circles. Data has become an equally important strategic resource as natural resources and human resources. of great importance. If these big data are effectively organized and used, it will play a huge role in promoting the development of society and economy. These rapidly increasing data mainly come from people's daily life, especially the Internet has become my country's largest public information distribution center and social group platform. Compared with traditional communication media such as newspapers, wireless radio and television, online media has comprehensive characteristics such as low barriers to entry, super-large-scale information, ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More