HBase-based Internet-of-things big data storing and accessing method

An access method and big data technology, applied in database models, relational databases, digital data processing, etc., can solve problems such as high costs, many user restrictions, and single query methods, and achieve efficient storage and scalability. , taking into account the effect of data balance

Active Publication Date: 2017-05-31
江苏海平面数据科技有限公司
View PDF5 Cites 12 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Traditional IoT solutions usually store data in relational databases. The outstanding problem is that it cannot solve the high-frequency insertion and query of large amounts of data, and the cost of the solution is high and the scalability is not...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • HBase-based Internet-of-things big data storing and accessing method
  • HBase-based Internet-of-things big data storing and accessing method
  • HBase-based Internet-of-things big data storing and accessing method

Examples

Experimental program
Comparison scheme
Effect test

specific Embodiment

[0044] A kind of HBase-based Internet of Things big data access method of the present invention comprises:

[0045] 1) Create an HBase table according to the following storage rules

[0046]Create an HBase table for each type of terminal, set the name of the column family in each table to "1", the name of the column qualifier to cmdID, the content of the uploaded message body to be value (as shown in Table 1), and specify the HBase Region The split policy is "KeyPrefixRegionSplitPolicy", and the length of the prefix is ​​2.

[0047] Table I

[0048]

[0049] The rowkey generation rules for each piece of data stored in HBase (such as figure 1 shown) as follows:

[0050] a. The selection of the modulus value is closely related to the number of regions in the cluster and will directly affect the scalability of the cluster. Here we can define the size of the modulus value as 32767 to ensure that the generated prefix is ​​2 bytes;

[0051] b. Calculate the corresponding hash...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to an HBase-based Internet-of-things big data storing and accessing method, and belongs to the technical field of Internet-of-things data processing. The method includes the following steps that 1, HBase lists are created, each terminal type corresponds to one corresponding HBase list, the name of each terminal type serves as the name of the corresponding HBase list, and column family names (Column Family), column qualifier names (Column) and a Region splitting strategy are specified; 2, reported data is introduced to the HBase lists, and a set of rowkey generation scheme is designed; 3, data query is conducted. The method fully considers the features of Internet-of-things application fields and the characteristics of HBase column type storage, efficient storage of the data is achieved, requirements of users under different application scenarios are met, and the scalability and data balancing of a system are effectively taken into account.

Description

technical field [0001] The present invention relates to an HBase-based Internet of Things big data access method, belonging to the technical field of Internet of Things data processing, in particular to an HBase database-based Internet of Things big data access method. Background technique [0002] The Internet of things (IoT), that is, through GPS, RFID, sensors and other equipment, according to the agreed protocol, connects any item with the Internet for information exchange and communication, so as to realize intelligent identification, positioning, tracking and monitoring and management of a network. Simply put, the Internet of Things is the "Internet of Things Connected". Among them, the items connected to the Internet are terminals. Usually a group of terminals with the same communication protocol, data format and command set are defined as a terminal type. By defining the terminal type, it can provide a basis for subsequent operations such as table division and aut...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/2282G06F16/2471G06F16/284
Inventor 孙杰李鹏飞王欣然陈智也
Owner 江苏海平面数据科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap