Redis cluster mass data quick cleaning system and method based on message queue

Abstract

The invention discloses a Redis cluster mass data quick cleaning system and method based on a message queue. The system includes a management node, a Redis cluster node, a message queue node and a temporary cache node; for the Redis cluster node, Redis cluster services are deployed on different servers, at least three Redis primary libraries are deployed, and each Redis primary library correspond to at least one Redis secondary library; for the message queue node, message queue distributed clusters are deployed on the different servers to serve as intermediate vectors used for processing Redis key values; for the temporary cache node, a Pika primary library and a Pika secondary library are deployed on the two servers respectively and used for storing cleaned Redis cluster data; for the management node, all metainformation is stored, at the same time, the Web application service is deployed, a Redis cluster, a message queue cluster and the Pika primary library are accessed, and the Redis key values are deleted. The processing speed of the system is close to the memory level speed, mass data can be quickly cleaned, the system is easy and reliable to use and has little influence on a Redis cluster service, and data roll-backing can be achieved through one click.

Description

technical field [0001] The invention belongs to the technical field of computers, is mainly applied to the application scene of a Redis database, and is used for quickly cleaning up massive data of a Redis cluster. Background technique [0002] For Internet companies, the Redis database is used more and more widely as a cache and high-performance storage solution, especially the Redis cluster, which has been widely used due to its support for distributed high availability and horizontal expansion. However, the characteristics of Redis clusters determine the complexity of cleaning up massive data. Existing data cleaning solutions for Redis clusters have the following disadvantages: [0003] 1) The cleaning speed is slow and cannot meet the demand for cleaning massive data in a short period of time, thus affecting the normal operation of production environment services. [0004] 2) It has a great impact on Redis cluster services. For high-concurrency Redis clusters, cleaning ...

AI Technical Summary

Problems solved by technology

[0003] 1) The cleaning speed is slow, unable to meet the needs of cleaning up massive data in a short period of time, thus affecting the normal operation of production environment services

[0004] 2) It has a great impact on Redis cluster services. For high-concurrency Redis clusters, cleaning up massive data will affect the normal operation of the production environment business

[0005] 3) Poor ease of use, high requirements on the ability of operators, and poor operation convenience

Images