IoT heterogeneous data storage method and system

Abstract

The invention discloses an Internet of things heterogeneous data storage method and a storage system. The internet of things heterogeneous data storage method comprises the steps that various types of data are stored in a column-based mode through an HBase distributed storage system, an MySQL metadatabase carries out registration management on the stored data, and finally the data access layer is configured in a centralized manner, so that applications based on the database can use the unified data interface. According to the Internet of things heterogeneous data storage method and the storage system, the unified storage mode is adopted through the unified data interface, the storage of different types of data and data from different sources is effectively supported; the column-oriented storage mechanism is extended by utilizing the HBase technology, so that the storage volume can be extended horizontally; the memory mapping file mechanism is adopted to effectively improve the storage and management efficiency of large files; and the data redundancy storage technology is adopted to greatly improve the system reliability and the accessibility. The Internet of things heterogeneous data storage method and the storage system can well solve the storage problem of various types of data, has efficient storage and utilization capability on large files, and is also low in cost and high in robustness.

Description

technical field

[0001] The present invention relates to a method in the field of data storage, in particular to a method and system for storing heterogeneous data in the Internet of Things. Background technique

[0002] The Internet of Things (IoT: Internet of Things) is an important part of the new generation of information technology. On today's Internet, data interaction has almost become our ultimate appeal, but the ensuing data diversity, distributed storage and loose coupling of information, and the geometric expansion of data volume have also brought data The difficulty of organization is increasing. At the same time, with the development of Ajax, RIA and service-oriented network applications, the complexity of the client data processing logic required by them continues to increase, making development more difficult.

[0003] Traditional data storage technologies are based on relational databases, mainly through mathematical concepts and methods such as set algebra t...

Examples

Embodiment Construction

[0028] The technical solutions of the present invention will be further explained below in conjunction with the drawings and embodiments, but the following content is not intended to limit the protection scope of the present invention.

[0029] like figure 1 As shown, this embodiment provides a heterogeneous data storage system for the Internet of Things. The system is composed of five modules: an interface module, a data storage module, a capacity expansion module, a large file storage module, and a redundant storage module. The specific details of each module The implementation application is as follows:

[0030] (1) Interface module—unified analysis of data in different formats, receiving specified unified query and feedback data types, and realizing loose coupling between the data providing layer and the data access layer;

[0031] (2) Data storage module - use HBase technology to build a large-scale storage cluster, and store data in the designated column family (ColumnF...

technical field

[0001] The present invention relates to a method in the field of data storage, in particular to a method and system for storing heterogeneous data in the Internet of Things. Background technique

[0002] The Internet of Things (IoT: Internet of Things) is an important part of the new generation of information technology. On today's Internet, data interaction has almost become our ultimate appeal, but the ensuing data diversity, distributed storage and loose coupling of information, and the geometric expansion of data volume have also brought data The difficulty of organization is increasing. At the same time, with the development of Ajax, RIA and service-oriented network applications, the complexity of the client data processing logic required by them continues to increase, making development more difficult.

[0003] Traditional data storage technologies are based on relational databases, mainly through mathematical concepts and methods such as set algebra t...

Examples

Embodiment Construction

[0028] The technical solutions of the present invention will be further explained below in conjunction with the drawings and embodiments, but the following content is not intended to limit the protection scope of the present invention.

[0029] like figure 1 As shown, this embodiment provides a heterogeneous data storage system for the Internet of Things. The system is composed of five modules: an interface module, a data storage module, a capacity expansion module, a large file storage module, and a redundant storage module. The specific details of each module The implementation application is as follows:

[0030] (1) Interface module—unified analysis of data in different formats, receiving specified unified query and feedback data types, and realizing loose coupling between the data providing layer and the data access layer;

[0031] (2) Data storage module - use HBase technology to build a large-scale storage cluster, and store data in the designated column family (ColumnF...

Images