HBase-based mobile internet mass data processing system and method

A mass data processing and mobile Internet technology, applied in the field of big data, can solve problems such as inability to effectively store mass data, and achieve the effect of improving query efficiency
CN107943890AInactive Publication Date: 2018-04-20WUHAN HONGXU INFORMATION TECH
3 Cites 4 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
WUHAN HONGXU INFORMATION TECH
Publication Date
2018-04-20
Estimated Expiration
Not applicable · inactive patent

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses an HBase-based mobile internet mass data processing system and method, and belongs to the field of big data. The system comprises a data polling module (10), a data analysis module (20), a data cleaning module (30), an ETL module (40), a data table record packaging module (50), an index table record packaging module (60), an HBase writing module (70), an HBase cluster module (80) and a Region control module (90), wherein the Region control module (90) is used for controlling distribution, in the HBase cluster module (80), of Regions. According to the system and method,an HBase distributed database is used and an HDFS is used as a file system, so that the disk extension problem is solved; secondary index is realized, so that the retrieval speed is high; data table Regions are in one-to-one correspondence with index table Regions, on a same Region Server, RPC calling is decreased for once during the query, so that the query efficiency is improved; and effective technical support can be provided for mobile internet mass data processing.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention belongs to the field of big data, and in particular relates to an HBase-based mobile Internet massive data processing system and a method thereof. Background technique

[0002] In recent years, mobile terminals such as mobile phones and PCs have become more and more popular, and the mobile Internet has developed rapidly, resulting in a sharp increase in the amount of information. How to store and retrieve massive data has increasingly measured the success of an enterprise, and data storage capabilities will become the core competitiveness of enterprises. The importance of data is unquestionable, but as data is generated faster and faster, and the amount of data is getting bigger and bigger, the challenges of data processing technology are also getting bigger and bigger.

[0003] HBase is a distributed, column-oriented, scalable open source database mainly used for unstructured data storage purposes. HBase is an open source implementatio...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More