Implementation method of redis database migration tool on cloud platform

Abstract

The invention provides a method for realizing a Redis database migration tool on a cloud platform. The method comprises the following steps: creating a Redis database migration tool instance on the cloud platform through a Redis-shake tool; a user checks the log and progress of the migration task through the TaskID, and monitors the state and progress of data migration in real time; the system obtains the performance index of the migration process through an HTTP protocol, and stores the performance index into a Prometheus time sequence database; and a user creates a Redis migration tool instance through the cloud platform console, configures Redis instance information of a source end and a target end, and starts a migration task to realize data migration. The implementation method of the redis database migration tool on the cloud platform has the beneficial effects that the rapid and convenient migration from the under-cloud Redis instance to the on-cloud Redis instance is realized.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to an implementation method of a redis database migration tool on a cloud platform. Background Art

[0002] With the development of cloud computing, more and more companies are migrating their databases to cloud platforms. As a high-performance cache database, Redis is widely used in various systems. On cloud platforms, you can create Redis instances with multiple architectures, such as single-node Redis instances, sentinel cluster Redis instances, and Redis Cluster instances.

[0003] Existing Redis data migration tools, such as Redis-shake, can support the migration and filtering of Redis data. Redis-shake is easy to use, high-performance, and highly reliable. It is suitable for Redis data migration scenarios of various scales, and can support the migration of full and incremental data, and supports synchronization between single nodes, master-slave versions, and cluster versions.

[0004] The existing technology has the following shortcomings: configuring the Redis migration task is cumbersome, and no visual management tool is provided, which makes the migration task management inconvenient; the existing tools cannot view the Redis migration progress in real time, making it difficult to control the entire migration process; there is a lack of performance monitoring of the Redis migration process, and it is impossible to identify and handle bottleneck problems in the migration in a timely manner. Summary of the invention

[0005] In view of this, the present invention aims to propose an implementation method of a redis database migration tool on a cloud platform to at least solve one problem in the background technology.

[0006] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0007] A method for implementing a redis database migration tool on a cloud platform, comprising:

[0008] Use the Redis-shake tool to create a Redis database migration tool instance on the cloud platform;

[0009] Users can view the log and progress of migration tasks through TaskID and monitor the status and progress of data migration in real time;

[0010] The system obtains the performance indicators of the migration process through the HTTP protocol and stores them in the Prometheus time series database;

[0011] The user creates a Redis migration tool instance through the cloud platform console, configures the source and target Redis instance information, and starts the migration task to implement data migration.

[0012] Furthermore, the creation of a Redis database migration tool instance on the cloud platform by using the Redis-shake tool includes:

[0013] The user configures the connection information of the source database and the target database through the cloud platform interface, creates a migration task in the virtual machine, and generates a unique TaskID for the task;

[0014] The system creates a working directory for each migration task and records the configuration, progress, and log information of the migration task.

[0015] Furthermore, the creation of a Redis database migration tool instance on the cloud platform by using the Redis-shake tool includes:

[0016] The user obtains the monitoring port information of the migration task through TaskID and checks the progress of the migration task in real time, including the number of data entries read from the source database and the number of data entries written to the target database.

[0017] The monitoring indicators of the migration task are obtained through the HTTP protocol and stored in the Prometheus time series database. The monitoring indicators include the amount of data read per second and the amount of data written per second.

[0018] Furthermore, the user creates a Redis migration tool instance through the cloud platform console, configures the source and target Redis instance information, and starts the migration task to achieve data migration, including:

[0019] Install Redis-shake into the CentOS operating system and make it into an operating system image;

[0020] The user creates a Redis migration tool in the cloud platform console. The service layer selects the corresponding Redis-shake system image and creates a virtual machine on the cloud to complete the creation of the Redis migration tool.

[0021] Furthermore, the user creates a Redis migration tool instance through the cloud platform console, configures the source and target Redis instance information, and starts the migration task to achieve data migration, including:

[0022] The user selects the created Redis migration tool instance and enters the information of the source and target Redis instances;

[0023] The system tests connectivity with the source and target Redis databases to ensure correct configuration;

[0024] Start the Redis migration task and begin data migration.

[0025] Furthermore, the information of the target Redis instance includes IP, port, user name, and password.

[0026] Furthermore, the present solution discloses an electronic device, comprising a processor and a memory that is communicatively connected to the processor and is used to store executable instructions of the processor, wherein the processor is used to execute an implementation method of a redis database migration tool on a cloud platform.

[0027] Furthermore, the present solution discloses a server, comprising at least one processor, and a memory communicatively connected to the processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor so that the at least one processor executes an implementation method of a redis database migration tool on a cloud platform.

[0028] Furthermore, the present solution discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements a method for implementing a redis database migration tool on a cloud platform.

[0029] Compared with the prior art, the implementation method of the redis database migration tool on a cloud platform described in the present invention has the following beneficial effects:

[0030] (1) The implementation method of the redis database migration tool on a cloud platform described in the present invention realizes the rapid and convenient migration of the off-cloud Redis instance to the on-cloud Redis instance;

[0031] (2) The implementation method of the redis database migration tool on a cloud platform described in the present invention provides a visual operation interface, simplifies the configuration and management of Redis migration tasks, and improves the user experience;

[0032] (3) The implementation method of the redis database migration tool on a cloud platform described in the present invention monitors the migration progress and performance in real time, which helps users to promptly discover bottlenecks in the migration process and optimize them. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0034] Figure 1 A schematic diagram of the creation process of the Redis migration tool according to an embodiment of the present invention;

[0035] Figure 2 A schematic diagram of a migration task creation process according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0037] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0038] Use redis-shake as the engine to create a redis migration tool instance on the cloud platform. After creating the instance, you can create a migration task. The specific implementation is as follows:

[0039] Migration tool functions:

[0040] 1. Visualize the configuration migration task:

[0041] Through the user configuration of the connection information of the source database and the target database, a migration task is created in the virtual machine where the migration tool is located, a globally unique TaskID is generated for the task, and a working directory is created for the migration task with the TaskID as the directory name. The ID of the migration process, the configuration information of the migration task, and the log of the migration task are recorded in the working directory.

[0042] 2. View the migration task log:

[0043] Search for the migration task log by TaskID.

[0044] 3. Check the progress of the migration task:

[0045] The monitoring service port monitored by the migration task is obtained through the migration task configuration information recorded in the TaskID working directory, and then the migration progress of the migration task is obtained, including: the number of data read from the source database and the number of data written to the target database.

[0046] 4. Migration tool monitoring:

[0047] Obtain the monitoring service port monitored by the migration task process as described in [3], obtain the monitoring indicators through the HTTP protocol, and store the monitoring indicators in the Prometheus time series database to provide users with monitoring information for querying migration tasks. The monitoring indicators include: the amount of data read from the source database per second and the amount of data written to the target database per second.

[0048] Covered redis migration scenarios:

[0049] 1. Migrate the off-cloud Redis instance to the on-cloud Redis instance, that is, the source Redis is off-cloud and the target Redis is on-cloud.

[0050] 2. Migrate the redis instance within the cloud, that is, the source redis is on the cloud and the target redis is also on the cloud.

[0051] Create a migration tool on the cloud platform:

[0052] 1. Install redis-shake into the Centos operating system and make it into an operating system image.

[0053] 2. The user creates the redis migration tool on the cloud platform page, and the service layer selects the corresponding redis-shake system image to create a virtual machine on the cloud. This completes the creation of the redis migration tool.

[0054] Create a redis migration task on the cloud platform:

[0055] 1. Select the created redis migration tool

[0056] 2. Enter the information of the source redis instance, including:

[0057] The host, port, username, password and other information of the source redis

[0058] 3. Enter the target redis instance information, including the target host, port, username, password, etc.

[0059] 4. Create a migration task.

[0060] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0061] In the several embodiments provided in the present application, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the division of the units described above is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The above-mentioned units may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

[0063] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for implementing a redis database migration tool on a cloud platform, characterized in that: include: Use the Redis-shake tool to create a Redis database migration tool instance on the cloud platform; Users can view the log and progress of migration tasks through TaskID and monitor the status and progress of data migration in real time; The system obtains the performance indicators of the migration process through the HTTP protocol and stores them in the Prometheus time series database; The user creates a Redis migration tool instance through the cloud platform console, configures the source and target Redis instance information, and starts the migration task to implement data migration.

2. According to the implementation method of a redis database migration tool on a cloud platform according to claim 1, it is characterized in that: The Redis database migration tool instance is created on the cloud platform by using the Redis-shake tool, including: The user configures the connection information of the source database and the target database through the cloud platform interface, creates a migration task in the virtual machine, and generates a unique TaskID for the task; The system creates a working directory for each migration task and records the configuration, progress, and log information of the migration task.

3. According to the implementation method of a redis database migration tool on a cloud platform according to claim 1, it is characterized in that: The Redis database migration tool instance is created on the cloud platform by using the Redis-shake tool, including: The user obtains the monitoring port information of the migration task through TaskID and checks the progress of the migration task in real time, including the number of data entries read from the source database and the number of data entries written to the target database. The monitoring indicators of the migration task are obtained through the HTTP protocol and stored in the Prometheus time series database. The monitoring indicators include the amount of data read per second and the amount of data written per second.

4. The method for implementing a redis database migration tool on a cloud platform according to claim 1, characterized in that: The user creates a Redis migration tool instance through the cloud platform console, configures the source and target Redis instance information, and starts the migration task to achieve data migration, including: Install Redis-shake into the CentOS operating system and make it into an operating system image; The user creates a Redis migration tool in the cloud platform console. The service layer selects the corresponding Redis-shake system image and creates a virtual machine on the cloud to complete the creation of the Redis migration tool.

5. The method for implementing a redis database migration tool on a cloud platform according to claim 1, characterized in that: The user creates a Redis migration tool instance through the cloud platform console, configures the source and target Redis instance information, and starts the migration task to achieve data migration, including: the user selects the created Redis migration tool instance and enters the source and target Redis instance information; The system tests connectivity with the source and target Redis databases to ensure correct configuration; Start the Redis migration task and begin data migration.

6. The method for implementing a redis database migration tool on a cloud platform according to claim 5, characterized in that: The information of the target Redis instance includes IP, port, user name, and password.

7. An electronic device, comprising a processor and a memory connected to the processor for storing instructions executable by the processor, characterized in that: The processor is used to execute the implementation method of the redis database migration tool on a cloud platform described in any one of claims 1-6 above.

8. A server, characterized in that: It includes at least one processor and a memory communicatively connected to the processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor so that the at least one processor executes a method for implementing a redis database migration tool on a cloud platform as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for implementing a redis database migration tool on a cloud platform as described in any one of claims 1 to 6 is implemented.

Images