HBase-based highly ordered queue operation method and device

Abstract

The invention provides an HBase-based highly ordered queue operation method and device. The method includes acquiring queue operation requests sent by a plurality of client terminals, wherein queues targeted by the queue operation request are disposed in an HBase distributive database; performing competition queue lock on the queue operation requests if the plurality of the queue operation requests target at one queue at the same time; performing queue operation corresponding to the queue operation requests with lock success on the queue; after the queue operation is implemented, removing the queue lock and performing the above steps again until the execution of the queue operation requests are implemented so as to realize HBase-based highly ordered queue operation. Compared with the prior art, the invention has advantages of high performance, high orderliness, high practicability, and being capable of supporting mass equipment and insusceptible of data loss and the like, and can be applied to cloud IoT access service, and is good in performance, high in reliability, high in horizontal expandability and meets requirements for access of mass users and equipment.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a technology of HBase-based strong ordered queue operation. Background technique [0002] On the public cloud, IoT services are often multi-tenant, each user has a different device, and each subscribing device has its own message queue. As the number of users and devices increases, the number of queues and the size of queue data will also increase dramatically, often becoming a performance bottleneck. [0003] Most of the existing distributed queue solutions are implemented with Redis or Kafka. First of all, from a distributed perspective, both support cluster mode and can form a distributed system. Secondly, from the perspective of queues, Redis supports queue structures, and Kafka's topics can be used as queues to meet the orderly writing of messages. Finally, from the perspective of multi-tenancy, logical division can be made. For example, user+client can be used t...

AI Technical Summary

Problems solved by technology

[0004] However, Redis and Kafka can meet small-scale requirements to a certain extent, but cannot meet larger-scale and higher-reliability scenarios

First of all, from the perspective of Redis, it is a memory-based database. For mass storage, the cost will be quite high. Its cluster mode is master-slave, and there is a risk of data loss. For example, the master node is down during master-slave replication. , or both master and slave nodes are down, data will be inconsistent and lost

Secondly, from the perspective of Kafka, there is a performance bottleneck problem. Because Kafka stores topic and other information in zookeeper, when the number of topics reaches a certain scale, that is, the number of znodes is large, the limitation of zookeeper will cause Kafka performance to drop sharply

It can be seen that the current public cloud IoT service providers cannot support massive access scale, nor can they achieve high availability and high reliability

Images