Methods for converting between Redis master-slave instances and Redis cluster instances

Abstract

This application provides a method for converting between Redis master-slave instances and Redis cluster instances, relating to the field of server cluster technology. The method includes: when user business grows, scaling up according to pre-defined conversion steps, i.e., from a first Redis master-slave instance to a first Redis master-slave instance and from a Trove cluster instance to a first Redis single-shard master-slave cluster instance to a first Redis cluster instance, thereby improving the concurrency capability of the Redis service; and when user business is at a low point, scaling down according to the steps of a second Redis cluster instance to a second Redis single-shard master-slave cluster instance to a second Redis master-slave instance and from a Trove cluster instance to a second Redis master-slave instance, thereby reducing user costs. The method of this application achieves the technical effects of improving the response speed of scaling up and down requests and reducing resource allocation costs.

Description

Technical Field

[0001] This application relates to the field of server cluster technology, and in particular to a method for converting between Redis master-slave instances and Redis cluster instances. Background Technology

[0002] With the rapid development of communication and network technologies, the load requirements of most communication and network systems have become higher, and the need for expansion and contraction will arise with changes in business. In order to cope with the impact of expansion and contraction caused by business fluctuations, the system generally adopts a cluster form.

[0003] Redis, a high-performance open-source key-value database, features high access performance, rich data types, and good scalability. Furthermore, Redis can provide users with key-value storage and query capabilities through external services.

[0004] In existing technologies, Redis employs sharding technology, storing data in shards across multiple Redis instances, distributing data across Redis instances on various devices to reduce the storage pressure on a single Redis instance. When business requirements change, cluster scaling is achieved by creating new clusters or replacing existing ones. However, creating new clusters or replacing existing ones prolongs scaling time, is inflexible, inefficient, and cannot quickly respond to business scaling needs. Furthermore, this method requires migrating all resources from the original cluster to the new cluster, making targeted resource migration impossible. Therefore, existing technologies suffer from slow response times to scaling requirements and high resource access costs. Summary of the Invention

[0005] This application provides a method for converting between Redis master-slave instances and Redis cluster instances, in order to solve the technical problems of slow response speed and high resource call cost in the prior art for scaling up and down.

[0006] Firstly, this application provides a method for converting a Redis master-slave instance into a Redis cluster instance, including:

[0007] Determine whether the first Redis master-slave instance contains a first load balancer; if yes, transfer the first load balancer to the first Redis master-slave and Trove cluster instance corresponding to the first Redis master-slave instance; if no, obtain the first virtual IP address of the first Redis master-slave instance, create the first load balancer based on the first virtual IP address, and transfer the first load balancer to the first Redis master-slave and Trove cluster instance.

[0008] Obtain the first master instance, the first proxy instance, and various first configuration resources of the first Redis master-slave instance; determine the first target configuration resource to be transferred among the first configuration resources, and delete the first configuration resources other than the first target configuration resource; wherein, the first master instance is bound to the first virtual IP address;

[0009] Transfer the first master instance, the first proxy instance, and the first target configuration resources to the first Redis master-slave and Trove cluster instance;

[0010] Unbind the first virtual IP address from the first master instance, and create a corresponding second virtual IP address in the first Redis master-slave and Trove cluster instances based on the first virtual IP address; bind the second virtual IP address to the first master instance.

[0011] Based on the first Redis master-slave and Trove cluster instances, generate a corresponding first Redis single-shard master-slave cluster instance; based on the first preset conditions, add a first preset number of first target shards and first proxy instances to the first Redis single-shard master-slave cluster instance; and configure the first load balancer accordingly based on all the first proxy instances.

[0012] Determine the first data slot in the first Redis single-shard master-slave cluster instance used for storing data; based on the first target shard and all shards of the first Redis single-shard master-slave cluster instance, reallocate the first data slot to generate the corresponding first Redis cluster instance.

[0013] Secondly, this application provides a method for converting a Redis cluster instance into a Redis master-slave instance, including:

[0014] Identify all second data slots contained in all shards of the second Redis cluster instance, as well as the second target shard pre-reserved in the second Redis cluster instance; migrate all second data slots to the second target shard, and delete all shards other than the second target shard;

[0015] Delete a second preset number of second proxy instances from the second Redis cluster instance based on the second preset conditions; configure the second load balancer in the second Redis cluster instance based on the deleted second proxy instances; shrink the second Redis cluster instance to a second Redis single-shard master-slave cluster instance;

[0016] Based on the second Redis single-shard master-slave cluster instance, generate the corresponding second Redis master-slave and Trove cluster instances, and create the corresponding second Redis master-slave instance based on the second Redis master-slave and Trove cluster instances;

[0017] Obtain the second configuration resource from the second Redis master-slave and Trove cluster instances, determine the second target configuration resource to be transferred from the second configuration resource; transfer the second target configuration resource to the second Redis master-slave instance, and delete the second configuration resource other than the second target configuration resource;

[0018] Obtain the second master-slave instance ID of the second Redis master-slave instance, and create a corresponding third virtual IP address in the second Redis master-slave instance based on the second master-slave instance ID; bind the third virtual IP address to the second master instance in the second Redis master-slave instance.

[0019] The method for converting between Redis master-slave instances and Redis cluster instances provided in this application allows for the scaling up of Redis master-slave instances into Redis cluster instances as user business grows. First, the first load balancer, first master instance, first proxy instance, and the first target configuration resource to be transferred from the first Redis master-slave instance, along with various first configuration resources, are transferred to the first Redis master-slave and Trove cluster instances. Then, a first Redis single-shard master-slave cluster instance is generated based on the first Redis master-slave and Trove cluster instances, and a first preset number of first target shards and first proxy instances are added to the first Redis single-shard master-slave cluster instance. Based on the first target shard and all shards of the first Redis single-shard master-slave cluster instance, the first data slots used for data storage in the first Redis single-shard master-slave cluster instance are reallocated to generate the first Redis cluster instance. During periods of low user activity, following the steps from a second Redis cluster instance to a second Redis single-shard master-slave cluster instance to a second Redis master-slave instance and from a Trove cluster instance to a second Redis master-slave instance, the second Redis cluster instance is scaled down to a second Redis master-slave instance. During this conversion, the resources to be transferred within the second Redis cluster instance are also migrated along with the entire conversion process. The conversion between Redis master-slave instances and Redis cluster instances is based on the existing instances, migrating the data from the original instances rather than creating a completely new instance to replace the original. This allows for dynamic scaling up or down according to user needs, without requiring users to change their Redis client or modify their business code. Furthermore, during the switch between two different Redis deployment models, targeted resource transfer is achieved by automatically deleting resources unnecessary for the target model and creating the necessary resources, thus improving the response speed for scaling up and down requests and reducing resource access costs. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] Figure 1 This is a schematic diagram of the structure of a Redis master-slave instance provided in the embodiments of this application;

[0022] Figure 2 This application provides a schematic diagram of the structure of a Redis master-slave and Trove cluster instance as illustrated in the embodiments of this application.

[0023] Figure 3 This is a schematic diagram of the structure of a Redis single-shard master-slave cluster instance provided in the embodiments of this application;

[0024] Figure 4 This is a schematic diagram of the structure of a Redis cluster instance provided in the embodiments of this application;

[0025] Figure 5 A flowchart illustrating a method for converting a Redis master-slave instance into a Redis cluster instance, provided in an embodiment of this application.

[0026] Figure 6 A schematic diagram illustrating a scenario for converting a Redis master-slave instance into a Redis cluster instance, provided as an embodiment of this application;

[0027] Figure 7 A flowchart illustrating a method for converting a Redis cluster instance into a Redis master-slave instance, provided in an embodiment of this application.

[0028] Figure 8 This is a schematic diagram illustrating a scenario of converting a Redis cluster instance into a Redis master-slave instance, as provided in an embodiment of this application.

[0029] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0030] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0031] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein.

[0032] In this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0033] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0034] To address the scaling needs arising from business fluctuations in current communication and network systems, Redis is typically used for system scaling. As a high-performance open-source key-value database, Redis employs sharding technology to distribute data across multiple Redis instances. When business requirements change, scaling is achieved by creating new clusters or replacing existing ones, while simultaneously migrating all resources from the original cluster to the new one. However, this approach is inflexible, prolonging scaling time and hindering rapid response to business demands. Furthermore, it lacks the ability to selectively migrate resources, resulting in slow response times and high resource access costs.

[0035] First, the implementation scenarios involved in this application will be explained.

[0036] Redis can provide key-value storage and query functionality as an external service. Developers only need to connect to the Redis service through a client to easily and quickly read and write various types of data. In Redis master-slave mode, one Redis service (slave instance) can be specified to synchronize data with another Redis service (master instance). Under the China Unicom cloud architecture, Redis master-slave mode exposes a Virtual IP Address (VIP) to users. This address is always bound to the master instance, making the master instance visible to users through the VIP, while the slave instance is not visible to users. When the master instance fails, a slave instance is selected to become the new master instance and bound to the aforementioned VIP to provide service.

[0037] Redis cluster mode allows you to enable cluster mode for one or more Redis services, adding them to the cluster and synchronizing data on specified nodes. In Redis cluster mode, data is distributed across 16384 slots based on the key, and then allocated to the appropriate nodes according to the slot distribution. A node and all (zero or more) nodes synchronizing data with that node constitute a shard, with the master node and slave node of that shard being the master and slave nodes, respectively. Typically, the master node is read-write, while slave nodes are read-only. If the master node fails, the Redis cluster automatically promotes one of the slave nodes in that shard to become the new master. In the China Unicom cloud architecture, all shards, master nodes, and slave nodes in Redis cluster mode are invisible to users. However, when users need to connect directly to the Redis cluster, they must provide the IP addresses of all master and slave nodes.

[0038] For Redis master-slave instances, the proxy is optional and used to provide read-write separation services. For Redis cluster instances, the proxy is always created to distribute user requests to the actual Redis data nodes, allowing users to access the cluster as if it were a single Redis service. Additionally, Redis cluster mode creates a load balancer for all proxies within the same cluster and exposes their IP addresses to provide services.

[0039] Infrastructure as a Service (IaaS) cloud computing platforms, exemplified by OpenStack, pool the hardware resources (CPU, memory, etc.) of multiple servers, allowing users to purchase the required amount of computing resources at the time they need. These resources are delivered rapidly using virtual machines or similar methods, while maintaining isolation between resources purchased by different users. Furthermore, OpenStack's Trove service allows the creation and management of database services on the OpenStack cloud pool, supporting features such as Redis master-slave replication and Redis cluster sharding by data slot.

[0040] The method for converting between Redis master-slave instances and Redis cluster instances provided in this application scales up Redis master-slave instances to Redis cluster instances as user business grows, following the steps from a first Redis master-slave instance to a first Redis master-slave instance and from a Trove cluster instance to a first Redis single-shard master-slave cluster instance and back to a first Redis cluster instance. During this conversion process, resources to be transferred are determined and new resources are created based on business needs, thus enabling targeted resource transfer. Conversely, during periods of low user business activity, the method scales down second Redis cluster instances to second Redis master-slave instances, following the steps from a second Redis cluster instance to a second Redis single-shard master-slave cluster instance to a second Redis master-slave instance and from a Trove cluster instance to a second Redis master-slave instance. This conversion process also requires determining resources to be transferred and migrating them along with the entire conversion process. Furthermore, the conversion between Redis master-slave instances and Redis cluster instances is based on the existing instance, migrating the data of the existing instance rather than creating a completely new instance to replace the original instance. This allows for dynamic scaling up or down according to user needs, and users do not need to change their Redis client or modify their business code. This achieves the technical effect of improving the response speed of scaling up and down needs and reducing the cost of resource access.

[0041] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0042] This application provides a method for converting between Redis master-slave instances and Redis cluster instances. It should be noted that the instances involved in the conversion process include Redis master-slave instances, Redis master-slave and Trove cluster instances, Redis single-shard master-slave cluster instances, and Redis cluster instances.

[0043] For clarity, both Redis master and slave instances share a virtual IP address. During the conversion of a Redis master / slave instance to a Redis cluster instance, this virtual IP address can be used to create a load balancer for the converted Redis cluster instance. This ensures that users can access the database through the same virtual IP address (the master / slave VIP before conversion, and the load balancer VIP after conversion) before and after the conversion. If future product iterations support creating load balancers from Redis master / slave instances, the load balancer will not be recreated during the conversion process, and users will still be able to access the database through the same virtual IP address (load balancer VIP) before and after the conversion.

[0044] Figure 1 This is a schematic diagram of the Redis master-slave instance provided in the embodiments of this application, as shown below. Figure 1As shown, a Redis master-slave instance mainly includes the following: (1) a unified master-slave instance ID (Group ID) used to manage the following resources; (2) a master database instance (master instance or master database), including an OpenStack Nova virtual machine, a Cinder data volume, two virtual network cards provided by Neutron (connected to the business VPC and management VPC respectively), and the operating system, database service, management service, etc. running in the virtual machine; among them, the database service is such as mysqld of MySQL, redis-server of Redis, etc., and the management service is such as trove-guestagent; in addition, each master instance has a unified ID used to manage the above resources and services; (3) zero or more slave database instances (slave instances or slave databases), similar to the master instance, including a slave instance ID, a virtual machine, a data volume, two virtual network cards (connected to the business VPC and management VPC respectively), and the operating system, database service, management service, etc.; (4) zero or more slave database instances (slave instances or slave databases), similar to the master instance, including a slave instance ID, a virtual machine, a data volume, two virtual network cards (connected to the business VPC and management VPC respectively), and the operating system, database service, management service, etc.; A read database instance (read-only instance or read-only database) is similar to the primary instance, containing a read-only instance ID, a virtual machine, a data volume, two virtual network cards (connected to the business VPC and the management VPC respectively), as well as an operating system, database services, management services, etc.; the read-only instance runs in read-only mode, connects to the primary instance through the management VPC, and establishes a master-slave replication relationship; at the same time, the read-only instance exposes the IP address of its own business VPC network card, and users can use this address to perform read operations on the database, but cannot perform write operations; (5) 0 or more database proxy instances (proxy instances or proxy) are similar to the primary instance, containing a proxy instance ID, a virtual machine, a data volume, two virtual network cards (connected to the business VPC and the management VPC respectively), as well as an operating system, database proxy services, management services, etc.;Among them, database proxy services such as Cetus, Predixy, etc.; (6) 0 or 1 load balancer (LB or load), the load balancer itself has a virtual IP address (LB VIP), expose the address and listen on a specific port of the address (consistent with the port number of the database service, such as port 6379 of Redis). Users can access the load balancer as if they were accessing a single database instance. The backend configuration of the load balancer is the IP address of the proxy instance business VPC. The load balancer can forward user requests to the appropriate proxy instance for processing according to the load balancing algorithm, thereby allowing the database instance to create or maintain more connections. Currently, a load balancer can only be created when two or more proxy instances are created, and at least one proxy instance must be created before the load balancer can be configured. Otherwise, the created load balancer will fail. (7) One virtual IP address, always bound to the business VPC network card of the master instance, is visible to users. When the master instance fails, a master-slave switch is performed, and the virtual IP address is synchronously bound to the new master instance. (8) Master-slave replication security group, bound to the management VPC network card of the master instance, slave instance, and read-only instance, allows these instances to access the database service port of the master instance through the network. (9) 0 or 1 Nova server group (server When creating Redis master-slave instances, if affinity or anti-affinity is required, the corresponding server group is created in Nova. When creating master, slave, and proxy instances, the virtual machine must belong to that server group. Database instances typically specify anti-affinity and generate corresponding server groups to ensure that the database remains available through master-slave failover if one host machine crashes. Virtual machines can only be added to a server group during creation; they cannot be added after creation. Once a virtual machine is added to a server group, both can be deleted individually without cascading deletion of the other. Read-only instances do not use this setting; virtual machines can be arbitrarily scheduled by Nova without being restricted by server groups. In addition, when users create and use Redis master-slave instances, they usually include the following, but these are not managed by the master-slave instance ID: (10) User-defined security groups, bound to the business VPC network cards of the master instance, slave instance, read-only instance, and proxy instance, allowing users to access the database service ports of these instances through the network; users can call the interface to change the security group themselves; (11) Zero or more configuration templates, bound to the master instance, slave instance, and read-only instance respectively, with each instance bound to zero or one configuration template. The configuration templates bound to the master instance and slave instance are the same, while the configuration templates bound to the read-only instance can be the same as or different from those bound to the master instance and slave instance. The configuration templates bound to multiple read-only instances can also be the same or different; a configuration template contains one or more database configuration parameters and their corresponding values. When bound to an instance, Trove writes these parameters and values ​​into the database service configuration file and applies them to the database process.

[0045] Figure 2 This is a schematic diagram of the structure of the Redis master-slave and Trove cluster instances provided in the embodiments of this application, as shown below. Figure 2 As shown, Redis master-slave and Trove cluster instances mainly include the following: (1) a unified Redis master-slave and Trove cluster instance ID (Cluster ID) used to manage the following resources; (2) a master database instance (master instance or master database); (3) zero or more slave database instances (slave instances or slave databases). Ideally, slave instances should run in read-only mode, connect to the master instance through the business VPC, and establish a master-slave replication relationship. In actual operation, since Redis master-slave and Trove cluster instances are only an intermediate state, the master-slave replication relationship at this time is not considered; neither the master instance nor the slave instance exposes its own IP address, and there is no virtual IP address of the Redis master-slave instance in the Redis master-slave and Trove cluster instances, so users cannot directly access the master instance and slave instance; (4) zero or more database proxy instances (proxy instances or proxies). The instance is configured with the IP addresses of all the business VPC network cards of the master instance and slave instance, which can perform read and write operations on the database and can separate the read operations to the slave instance to realize the read and write separation function; the proxy instance exposes the IP address of its own business VPC network card, and users can access the proxy instance as if they were accessing a single database instance; (5) User-defined security groups are bound to the business VPC network cards of the master instance, slave instance and proxy instance, allowing users to access the database service ports of these instances through the network, as well as the database services to access specific ports with each other; users can call the interface to change the security group themselves; (6) 0 or 1 configuration templates are bound to all master instances and slave instances, and the configuration templates bound to these instances are the same. In addition, Redis master-slave and Trove cluster instances also include the following, but they are not managed by the Redis master-slave and Trove cluster instance IDs: (7) 0 or 1 load balancers (LB or load), since there may be no proxy instance in the current state, it cannot be guaranteed that the load balancer is available to the user at this time; however, during the conversion between Redis master-slave instances and Redis cluster instances, the load balancer must exist and always occupy the same virtual IP address to ensure that the user can still access the virtual IP address after the conversion is completed.

[0046] Figure 3 This is a schematic diagram of the structure of a Redis single-shard master-slave cluster instance provided in the embodiments of this application, as shown below. Figure 3As shown, a Redis single-shard master-slave cluster instance mainly includes the following: (1) a unified Redis single-shard master-slave cluster instance ID (Cluster ID) used to manage the following resources; (2) a master database instance (master instance or master database); (3) zero or more slave database instances (slave instances or slave databases). Both the master instance and the slave instances are enabled in cluster mode. Then, the slave instances execute the cluster meet and cluster replicate commands to form a Redis single-shard master-slave cluster. The cluster has only one shard, and there is a master-slave replication relationship within the shard. At the same time, Trove allocates an initial Shard to the shard. ID; At this time, the master instance and slave instance no longer establish a traditional (non-cluster) master-slave replication relationship. Both the master instance and slave instance can expose their own IP address. However, since they are currently in an intermediate state, it cannot be guaranteed that users can access the master instance and slave instance; (4) 0 or more database proxy instances (proxy instances or proxies). The proxy instances are configured with the IP addresses of the business VPC network cards of the master instance and slave instance. They can perform read and write operations on the database and can separate the read operations to the slave instance to achieve read and write separation function; The proxy instances expose the IP address of their own business VPC network cards. Users can access the proxy instances as if they were accessing a single database instance; (5) User-defined security groups are bound to the business VPC network cards of the master instance, slave instance and proxy instance. Users can access the database service ports of these instances through the network and the database services can access specific ports with each other; Users can call the interface to change the security group; (6) 0 or 1 configuration templates are bound to all master instances and slave instances. The configuration templates bound to these instances are the same. The Redis single-shard master-slave cluster instance also includes the following, but they are not managed by the Redis single-shard master-slave cluster instance ID: (7) 0 or 1 load balancers (LB or load), since there may be no proxy instance in the current state, it cannot be guaranteed that the load balancer is available to the user at this time; however, during the conversion between the Redis master-slave instance and the Redis cluster instance, the load balancer must exist and always occupy the same virtual IP address to ensure that the user can still access the virtual IP address after the conversion is completed.

[0047] Figure 4 This is a schematic diagram of the structure of a Redis cluster instance provided in the embodiments of this application, as shown below. Figure 4As shown, a Redis cluster instance mainly includes the following: (1) a unified Redis cluster instance (Cluster ID) used to manage the following resources; (2) one or more database shards, each shard containing a Shard ID assigned by Trove, a master database instance (master instance or master database), and zero or more slave database instances (slave instances or slave databases); each instance contains an OpenStack Nova virtual machine, a Cinder data volume, two virtual network cards provided by Neutron (connected to the business VPC and management VPC respectively), and the operating system, database service, management service, etc. running in the virtual machine; in addition, each instance has a unified ID used to manage the above resources and services; all master instances and slave instances of all shards communicate using the business VPC, are interconnected through the Redis cluster meet command to form a cluster, and allocate the 16384 data slots of the Redis cluster instance to each master instance; slave instances communicate through the Redis cluster meet command. The replicate command specifies the master instance and replicates the data slots and corresponding data stored in the master instance. The master instance can read and write, while the slave instance is read-only. The master instance and slave instance can expose their own IP addresses for users to access. However, in actual products, the IP addresses of the master instance and slave instance are written into the proxy instance and are not visible to users. When the master instance fails, the Redis cluster instance will select a slave instance in the same shard to promote it to the master instance. (3) 0 or more database proxy instances (proxy instances or proxies) include a proxy instance ID, a virtual machine, a data volume, two virtual network cards (connected to the business VPC and management VPC respectively), as well as the operating system, database proxy service, management service, etc. The proxy instance is configured with the IP addresses of the business VPC network cards of all master instances and slave instances. It can perform read and write operations on the database and can also Read operations are separated to the slave instance to achieve read-write separation; the proxy instance exposes the IP address of its own business VPC network card, and users can access the proxy instance as if it were a single database instance, without having to directly connect to each node of the cluster; (4) User-defined security groups are bound to the business VPC network cards of the master instance, slave instance, and proxy instance, allowing users to access the database service ports of these instances through the network, as well as the database services to access specific ports with each other; users can call the interface to change the security group themselves; (5) 0 or 1 configuration templates are bound to all master instances and slave instances. The configuration templates bound to these instances are the same. A configuration template contains one or more database configuration parameters and their corresponding values; when bound to an instance, Trove writes these parameters and values ​​into the configuration file of the database service and applies them to the database process.In addition, the cloud-based Redis database cluster products currently providing services usually come with the following, but they are not managed by the Redis cluster instance ID: (6) 0 or 1 load balancers (LB or load), the load balancer itself has a virtual IP address (LB VIP), exposes the address and listens on a specific port of the address, and users can access the load balancer as if they were accessing a single database instance; the backend configuration of the load balancer is the IP address of the proxy instance business VPC network card, and the load balancer can forward user requests to the appropriate proxy instance for processing according to the load balancing algorithm, thereby allowing the database instance to create or maintain more connections; technically, at least 1 proxy instance needs to be created before a load balancer can be created or configured, otherwise the created load balancer will fail.

[0048] When user business grows, the Redis master-slave instance is scaled up to a Redis cluster instance, following the steps from the first Redis master-slave instance to the first Redis master-slave and Trove cluster instance to the first Redis single-shard master-slave cluster instance and back to the first Redis cluster instance. During this conversion process, resources to be transferred, such as the first master instance, first proxy instance, slave instances, user-defined security groups, configuration templates, etc., are determined based on business needs, and new resources are created, such as creating necessary shards and the first proxy instance, to perform targeted resource transfer. When user business is at a low point, the second Redis cluster instance is scaled down to a second Redis master-slave instance, following the steps from the second Redis cluster instance to the second Redis single-shard master-slave cluster instance to the second Redis master-slave and Trove cluster instance to the second Redis master-slave instance. During this conversion process, resources to be transferred, such as the second master instance, second proxy instance, slave instances, second load balancer, etc., are determined and migrated along with the entire conversion process, while some unnecessary resources are deleted. Furthermore, when converting between Redis master-slave instances and Redis cluster instances, the conversion is performed on the basis of the original instance, migrating the data of the original instance rather than creating a completely new instance to replace the original instance. This allows for dynamic scaling up or down according to user needs, and users do not need to change their Redis client or modify their business code. This achieves the technical effect of improving the response speed of scaling up and down needs and reducing the cost of resource access.

[0049] Figure 5 This is a flowchart illustrating a method for converting a Redis master-slave instance into a Redis cluster instance, as provided in an embodiment of this application. Figure 5 As shown in the embodiments of this application, a method for converting a Redis master-slave instance into a Redis cluster instance is provided, including:

[0050] S501. Determine whether the first Redis master-slave instance contains a first load balancer; if yes, transfer the first load balancer to the first Redis master-slave and Trove cluster instance corresponding to the first Redis master-slave instance; if no, obtain the first virtual IP address of the first Redis master-slave instance, create the first load balancer based on the first virtual IP address, and transfer the first load balancer to the first Redis master-slave and Trove cluster instance.

[0051] Specifically, cluster information is created in the Trove database to obtain the IDs of the first Redis master-slave instance and the Trove cluster instance. At the same time, the first load balancer is created or reused to ensure that at least one IP address of the instance before the conversion remains available after the conversion. If the first Redis master-slave instance has a first load balancer, the ID of the first load balancer is recorded directly and reused in subsequent conversion processes.

[0052] If the first Redis master-slave instance does not have a first load balancer, then the first load balancer is created using the first virtual IP address. Since the information of the first load balancer is very easy to lose during the conversion process, the ID of the first load balancer needs to be returned to the cloud computing platform via HTTP synchronization as soon as possible. Therefore, the operation of creating the first load balancer should be carried out as early as possible.

[0053] S502, Obtain the first master instance, the first proxy instance, and various first configuration resources of the first Redis master-slave instance; Determine the first target configuration resource to be transferred among the first configuration resources, and delete the first configuration resources other than the first target configuration resource; wherein, the first master instance is bound to the first virtual IP address;

[0054] S503, Transfer the first master instance, the first proxy instance, and the first target configuration resources to the first Redis master-slave and Trove cluster instance;

[0055] Specifically, as many resources as possible from the first Redis master-slave instance should be transferred to the first Redis master-slave and Trove cluster instances, including the first master instance, first proxy instance, slave instances, user-defined security groups, configuration templates, and other resources within the first Redis master-slave instance. The replication source settings on the slave instances need to be modified to specify replication from the IP address of the business VPC of the first master instance; simultaneously, the configured backend IP of the first proxy instance should also be changed to the IP address of the business VPC of the first master and slave instances. However, since the first Redis master-slave and Trove cluster instances are in a temporary state, these two steps are not strictly required.

[0056] Figure 6This application provides a schematic diagram illustrating a scenario for converting a Redis master-slave instance into a Redis cluster instance, as shown in the embodiments of this application. Figure 6 As shown, redundant read-only instances, master-slave replication security groups, server groups, and Redis master-slave instance IDs should be deleted. Currently, considering the difficulty in controlling master-slave replication relationships within each shard of a Redis cluster instance, if the Redis cluster instance continues to use server groups, all nodes on all shards would need to be set to anti-affinity, which could easily lead to conversion failures. Therefore, server groups have been deleted, and affinity or anti-affinity will no longer be considered. Furthermore, if the first Redis master-slave instance has a first load balancer, the first virtual IP address is no longer needed and should be deleted simultaneously.

[0057] The process of converting the first Redis master-slave instance into the first Redis master-slave and Trove cluster instance does not require shutting down the first master instance and has an RDB format backup, thus maintaining data integrity to the greatest extent.

[0058] S504. Unbind the first virtual IP address from the first master instance, and create a corresponding second virtual IP address in the first Redis master-slave and Trove cluster instances based on the first virtual IP address; bind the second virtual IP address to the first master instance;

[0059] S505. Generate a corresponding first Redis single-shard master-slave cluster instance based on the first Redis master-slave and Trove cluster instances; add a first preset number of first target shards and first proxy instances to the first Redis single-shard master-slave cluster instance based on the first preset conditions; configure the first load balancer accordingly based on all the first proxy instances;

[0060] Specifically, cluster mode is enabled for the first master instance, the first proxy instance, and the slave instance. The process of converting the first Redis master-slave and Trove cluster instance into the first Redis single-shard master-slave cluster instance requires shutting down the first master instance and clearing the slave instance data. Therefore, data integrity can only be guaranteed by relying on the RDB format backup of the first master instance.

[0061] In the first example, the steps to enable cluster mode for the first master instance are as follows: Set the `cluster-enabled yes` command in the Redis configuration file and restart. After restarting, the Redis process will reload the data from the RDB backup and distribute it across the 16384 data slots of the first Redis single-shard master-slave cluster instance. The steps to enable cluster mode for the first proxy instance are as follows: Adjust the backend type to cluster in the configuration file (e.g., Predixy) and set the backend IP to the IP address of the VPC network interface card of the first master and slave instances, then restart. The steps to enable cluster mode for the slave instance are as follows: Clear the `slaveof` setting in the Redis configuration file, set the `cluster-enabled yes` command, then execute the `flushall` command to clear the data, and restart. At this point, the slave instance is in cluster mode but has no data. Then execute the `cluster meet` and `cluster replicate` commands to connect to the first master instance and restart replication, replicating the data from the 16384 data slots.

[0062] Understandably, the necessary first proxy instance is created, added to the cluster, and the configuration of the existing first proxy instance is modified to add the newly created first target shard; at the same time, the backend resource pool of the first load balancer is configured, and the IP addresses of the business VPC network cards of all first proxy instances are added.

[0063] S506. Determine the first data slot used for storing data in the first Redis single-shard master-slave cluster instance; reallocate the first data slot based on the first target shard and all shards of the first Redis single-shard master-slave cluster instance; generate the corresponding first Redis cluster instance.

[0064] In the second example, the first master instance joins the cluster by executing the `cluster meet` command, while the slave instances establish a master-slave replication relationship by executing the `cluster meet` and `cluster replicate` commands. Then, `Trove` assigns a ShardID to each instance. After all the first master and slave instances have joined the Redis cluster, one instance is randomly selected to execute the `redis-cli —cluster rebalance` command, which distributes the data slots in the cluster to the shards in a specific way. This command depends on the amount of data in the cluster and may be time-consuming, requiring a monitoring mechanism to wait for its completion. For example, the `redis-cli —cluster rebalance` command could be executed in round-robin fashion, and the returned results checked. At this point, a complete first Redis cluster instance, along with the first load balancer, is delivered to the user.

[0065] Figure 7This is a flowchart illustrating a method for converting a Redis cluster instance into a Redis master-slave instance, as provided in an embodiment of this application. Figure 7 As shown in the embodiments of this application, a method for converting a Redis cluster instance into a Redis master-slave instance is provided, including:

[0066] S701. Determine all second data slots contained in all shards of the second Redis cluster instance, and the second target shard pre-reserved in the second Redis cluster instance; migrate all second data slots to the second target shard, and delete all shards other than the second target shard;

[0067] In the third example, assuming Shard 1 is the shard to be retained, obtain the master instance ID of the master instance in this shard, and then execute the `redis-cli --cluster rebalance` command to migrate all data slots of all shards to the master instance of Shard 1. This command depends on the amount of data in the cluster and may be time-consuming, requiring a monitoring mechanism to wait for its completion. For example, poll the execution of the `redis-cli --cluster rebalance` command and check the returned results. Additionally, execute the `cluster forget` command to remove shards other than Shard 1 from the second Redis cluster instance, and then delete the corresponding master and slave instances.

[0068] It should be noted that all Redis cluster products come with a load balancer. When converting a Redis cluster instance to a Redis master-slave instance, the load balancer can be included in the management scope of the Redis master-slave instance, thereby ensuring that users can access the database through the same virtual IP address (LB VIP) before and after the conversion.

[0069] S702: Delete a second preset number of second proxy instances from the second Redis cluster instance based on the second preset conditions; configure the second load balancer in the second Redis cluster instance based on the deleted second proxy instances; shrink the second Redis cluster instance to a second Redis single-shard master-slave cluster instance;

[0070] Understandably, if necessary, some second proxy instances may be removed from the second Redis cluster instance, and the backend resource pool of the second load balancer may be configured to remove the IP addresses of the corresponding business VPC network interfaces of the second proxy instances. Simultaneously, the configuration of existing second proxy instances may be modified to remove the IP addresses of the deleted shards. It should be noted that the above process relies on Redis commands to ensure data integrity during data slot migration. It is also necessary to ensure that the deleted instances belong to shards that do not need to be retained, and that the data on them has been migrated; otherwise, unmigrated data will be lost along with the instance deletion.

[0071] S703. Generate corresponding second Redis master-slave and Trove cluster instances based on the second Redis single-shard master-slave cluster instance, and create corresponding second Redis master-slave instances based on the second Redis master-slave and Trove cluster instances;

[0072] Specifically, cluster mode is disabled for the second master instance, second proxy instance, and slave instance, and the master-slave replication relationship is re-established. The process of converting the second Redis single-shard master-slave cluster instance into a second Redis master-slave and Trove cluster instance requires shutting down the second master instance and clearing the slave instance data. Therefore, data integrity can only be guaranteed by relying on the RDB format backup of the second master instance.

[0073] In the fourth example, the steps to disable cluster mode for the second master instance are as follows: Execute the `cluster forget` command to remove the node corresponding to the second master instance from the cluster, then delete `cluster-enabled yes` from the Redis configuration file of the slave instance, and restart. After restarting, the Redis process will reload the data from the RDB backup. The steps to disable cluster mode for the second proxy instance are as follows: Adjust the backend type to standalone mode in the configuration file (e.g., Predixy), and set the backend IP to the IP address of the business VPC network interface of the second master instance and slave instance, then restart. The steps to disable cluster mode for the slave instance are as follows: Set `slaveof` to point to the slave instance in the Redis configuration file, delete `cluster-enabled yes`, then execute the `flushall` command to clear the data, and restart. If `slaveof` is set to the IP address of the business VPC network interface of the second master instance, the slave instance will connect to the second master instance and restart master-slave replication. If it is set to the IP address of the management VPC network interface of the second master instance, the slave instance will also attempt to connect to the second master instance, but since the corresponding master-slave replication security group has not yet been created, the connection will fail.

[0074] S704. Obtain the second configuration resource in the second Redis master-slave and Trove cluster instances, determine the second target configuration resource to be transferred in the second configuration resource; transfer the second target configuration resource to the second Redis master-slave instance, and delete the second configuration resource other than the second target configuration resource;

[0075] S705. Obtain the second master-slave instance ID of the second Redis master-slave instance, and create a corresponding third virtual IP address in the second Redis master-slave instance based on the second master-slave instance ID; bind the third virtual IP address to the second master instance in the second Redis master-slave instance.

[0076] Figure 8 This application provides a schematic diagram illustrating a scenario for converting a Redis cluster instance into a Redis master-slave instance, as shown in the embodiments of this application. Figure 8 As shown, master-slave information is created in the Trove database to obtain the second master-slave instance ID of the second Redis master-slave instance. Simultaneously, a corresponding third virtual IP address and master-slave replication security group are created, and the corresponding rules are configured. Furthermore, as many resources as possible from the second Redis cluster instance need to be transferred to the second Redis master-slave instance, including the second master instance, slave instance, second proxy instance, and second load balancer. The slave instance needs to modify its replication source settings to specify the IP address of the VPC network interface managed by the second master instance for replication. Since the master-slave replication security group is configured at this time, the slave instance can replicate normally. The second proxy instance should change its configured backend IP address to the third virtual IP address of the second Redis master-slave instance. If the second Redis cluster instance has associated server groups, these are recorded in the second Redis master-slave instance. Finally, the IDs of the second Redis master-slave instance and the Trove cluster instance are deleted, resulting in a complete second Redis master-slave instance. Users can then create read-only instances and other resources for the Redis master-slave instance as needed.

[0077] In this Redis master-slave instance, the second master instance does not expose its own IP address. Instead, it exposes a third virtual IP address as the address of the second master instance. Users can perform read and write operations on the database through the third virtual IP address. Since the third virtual IP address is always bound to the second master instance, user operations are directed to the second master instance and cannot access the slave instance. In addition, the proxy instance in the Redis master-slave instance is configured with a virtual IP address and the IP address of the read-only instance's business VPC network interface. It can perform read and write operations on the database, and in the case of only a read-only instance, read operations are separated to the read-only instance to achieve read-write separation. The proxy instance exposes the IP address of its own business VPC network interface, and users can access the proxy instance as if it were a single database instance.

[0078] The method for converting between Redis master-slave instances and Redis cluster instances provided in this application allows for the scaling up of Redis master-slave instances into Redis cluster instances as user business grows. First, a first load balancer is created or reused in the first Redis master-slave instance. The first master instance, first proxy instance, and the first target configuration resource to be transferred from various first configuration resources are obtained from the first Redis master-slave instance. The first load balancer, first master instance, first proxy instance, and first target configuration resource are then transferred to the first Redis master-slave and Trove cluster instances. Second, a first Redis single-shard master-slave cluster instance is generated based on the first Redis master-slave and Trove cluster instances, and a first preset number of first target shards and first proxy instances are added to it. Finally, data is allocated based on all shards within the first Redis single-shard master-slave cluster instance to generate the first Redis cluster instance. When user business is at a low point, it is permissible to scale down a Redis cluster instance to a Redis master-slave instance. First, all data from the second Redis cluster instance is migrated to a pre-reserved second target shard, and all shards except the second target shard, along with a pre-defined number of second proxy instances, are deleted to generate a second Redis single-shard master-slave cluster instance. Next, a second Redis master-slave and Trove cluster instance is created based on the second Redis single-shard master-slave cluster instance. By disabling cluster mode on the second Redis master-slave and Trove cluster instances and re-establishing the master-slave replication relationship, the second Redis master-slave instance is generated. Finally, the second target configuration resource to be transferred is determined, transferred to the second Redis master-slave instance, and all resources except the second target configuration resource are deleted. The conversion between Redis master-slave instances and Redis cluster instances is based on the existing instances, migrating the data from the original instances rather than creating a completely new instance to replace the original ones. This allows for dynamic scaling up or down according to user needs, and users do not need to change their Redis clients or modify their business code. In addition, during the switching between the two different deployment models of Redis, targeted resource transfer is achieved by automatically deleting resources that are not needed in the target model and creating the resources required for it. This achieves the technical effect of improving the response speed of scaling up and down needs and reducing the cost of resource access.

[0079] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0080] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0081] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0082] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0083] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0084] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0085] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0086] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0087] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for converting a Redis master-slave instance into a Redis cluster instance, characterized in that, The method comprises the following steps: determining whether the first Redis master-slave instance contains a first load balancer; if yes, transferring the first load balancer to a first Redis master-slave and Trove cluster instance corresponding to the first Redis master-slave instance; if no, obtaining a first virtual IP address of the first Redis master-slave instance, creating the first load balancer based on the first virtual IP address, and transferring the first load balancer to the first Redis master-slave and Trove cluster instance; obtaining a first master instance, a first proxy instance and a plurality of first configuration resources of the first Redis master-slave instance; determining a first target configuration resource to be transferred in the first configuration resources, and deleting the first configuration resources other than the first target configuration resource; wherein the first master instance is bound to the first virtual IP address; transferring the first master instance, the first proxy instance and the first target configuration resource to the first Redis master-slave and Trove cluster instance; unbinding the first virtual IP address from the first master instance, creating a corresponding second virtual IP address in the first Redis master-slave and Trove cluster instance based on the first virtual IP address, and binding the second virtual IP address to the first master instance; generating a corresponding first Redis single-shard master-slave cluster instance based on the first Redis master-slave and Trove cluster instance; adding a first preset number of first target shards and first proxy instances to the first Redis single-shard master-slave cluster instance based on a first preset condition; and configuring the first load balancer based on all the first proxy instances. determining a first data slot for storing data in the first Redis single-shard master-slave cluster instance; and reallocating the first data slot based on the first target shards and all the shards of the first Redis single-shard master-slave cluster instance, to generate a corresponding first Redis cluster instance.

2. A method for converting a Redis cluster instance into a Redis master-slave instance, characterized in that, The method comprises the following steps: determining all second data slots contained in all shards of a second Redis cluster instance, and a second target shard pre-reserved in the second Redis cluster instance; migrating all the second data slots to the second target shard, and deleting all the shards other than the second target shard; deleting a second preset number of second proxy instances in the second Redis cluster instance based on a second preset condition; configuring a second load balancer in the second Redis cluster instance based on the second proxy instances after the deletion; scaling down the second Redis cluster instance to a second Redis single-shard master-slave cluster instance; generating a corresponding second Redis master-slave and Trove cluster instance based on the second Redis single-shard master-slave cluster instance, and creating a corresponding second Redis master-slave instance based on the second Redis master-slave and Trove cluster instance; obtaining a second configuration resource in the second Redis master-slave and Trove cluster instance, determining a second target configuration resource to be transferred in the second configuration resource; transferring the second target configuration resource to the second Redis master-slave instance, and deleting the second configuration resource except the second target configuration resource; obtaining a second master-slave instance ID of the second Redis master-slave instance, creating a corresponding third virtual IP address in the second Redis master-slave instance based on the second master-slave instance ID; and binding the third virtual IP address with a second master instance in the second Redis master-slave instance.

Images