Redis cluster federation data routing method and device, equipment, medium and product

Abstract

The present disclosure relates to a Redis cluster federation data routing method, device, equipment, medium and product, the method comprising: dividing the hash slots of the Redis cluster federation to obtain a plurality of hash slot directories; wherein each hash slot directory contains a plurality of hash slots; constructing a static routing table; the static routing table is used to represent a first mapping relationship between each hash slot directory and each Redis cluster under the Redis cluster federation, and a second mapping relationship between each hash slot under each hash slot directory and the nodes in each Redis cluster; when receiving an operation request of a client to a first node, finding the routing path of the first node in the Redis cluster federation based on the static routing table, and forwarding the operation request to the first node according to the routing path, so that the first node performs the corresponding operation according to the operation request. The present disclosure can realize efficient data routing and seamless expansion of large-scale Redis cluster federation, thereby improving the performance and operation and maintenance efficiency of the Redis cluster federation.

Description

Technical Field

[0001] This disclosure relates to the field of distributed caching systems, and more particularly to a method, apparatus, device, medium, and product for routing federated data in a Redis cluster. Background Technology

[0002] With the continuous expansion of internet business, distributed caching systems have become the core infrastructure supporting high-concurrency access. Redis Remote Dictionary Service clusters are widely used in distributed caching systems because they perform well in small and medium-sized deployments and can provide efficient data access.

[0003] In related technologies, traditional routing methods for Redis clusters have inherent scalability bottlenecks. Large-scale deployments drastically increase operational complexity, leading to performance degradation. Therefore, there is an urgent need for a routing method for federated data in Redis clusters to ensure cluster performance while managing large-scale clusters. Summary of the Invention

[0004] This disclosure provides a method, apparatus, device, medium, and product for routing federated data in a Redis cluster.

[0005] According to a first aspect of this disclosure, a method for routing Redis cluster federated data is provided, wherein the Redis cluster federation includes multiple Redis clusters, each Redis cluster including multiple nodes, the method comprising:

[0006] The hash slots of the Redis cluster federation are divided into multiple hash slot directories; each hash slot directory contains multiple hash slots.

[0007] Construct a static routing table; the static routing table is used to represent the first mapping relationship between each hash slot directory and each Redis cluster under the Redis cluster federation, and the second mapping relationship between each hash slot under each hash slot directory and each node in each Redis cluster;

[0008] When a client requests an operation on the first node, the system searches for the routing path of the first node in the Redis cluster federation based on the static routing table, and forwards the operation request to the first node according to the routing path, so that the first node performs the corresponding operation according to the operation request.

[0009] Further, the step of finding the routing path of the first node in the Redis cluster federation based on the static routing table includes:

[0010] Perform a hash operation on the address information of the first node to obtain the first hash value;

[0011] Perform bitwise operations on the first hash value to obtain the target hash slot directory number and the target offset;

[0012] The routing path is determined in the static routing table based on the target hash slot directory number and the target offset.

[0013] Further, the step of performing bitwise operations on the first hash value to obtain the target hash slot directory number and the target offset includes:

[0014] The target offset is obtained by performing a modulo operation on the total number of hash slots contained in the Redis cluster federation based on the first hash value;

[0015] The target offset is shifted based on a preset value to obtain the target hash slot directory number.

[0016] Further, determining the routing path in the static routing table based on the target hash slot directory number and the target offset includes:

[0017] Based on the first mapping relationship in the static routing table, the Redis cluster associated with the target hash slot directory number is found to obtain the target Redis cluster;

[0018] Based on the second mapping relationship in the static routing table, the node associated with the target offset is found in the target Redis cluster to obtain the target node, and the routing path is determined based on the location of the target node.

[0019] Furthermore, after forwarding the operation request to the first node according to the routing path, the method further includes:

[0020] In response to a data migration instruction for the second node, a first Redis cluster associated with the second node is determined in the Redis cluster federation; wherein the first Redis cluster includes the Redis cluster to which the second node belongs, and / or the Redis cluster to which the destination node of the second node belongs, the destination node being the endpoint for the data migration of the second node;

[0021] Determine the network status metrics of the first Redis cluster; wherein the network status metrics include at least one of the following: network latency, bandwidth, and packet loss rate;

[0022] The migration strategy is determined based on the network status indicators; wherein, the migration strategy is used to indicate whether to implement data migration through an intermediary module, the intermediary module including the data transmission switch and data transmission router in the Redis cluster federation;

[0023] The data in the second node is migrated to the destination node based on the migration strategy.

[0024] Further, determining the migration strategy based on the network state indicators includes:

[0025] Based on the network status indicators, the first Redis cluster is evaluated to obtain the evaluation results;

[0026] The migration strategy is determined based on the evaluation results.

[0027] Further, the evaluation of the first Redis cluster based on the network status indicators to obtain the evaluation results includes:

[0028] Assign weights to each of the network state indicators;

[0029] The evaluation result is obtained by weighting each of the network state indicators based on the weights.

[0030] Further, determining the migration strategy based on the evaluation results includes:

[0031] If the evaluation result is greater than a preset threshold, the migration strategy is determined as the first migration strategy; wherein, the first migration strategy is to instruct the second node to migrate data to the destination node;

[0032] If the evaluation result is less than or equal to a preset threshold, the migration strategy is determined as the second migration strategy; wherein, the second migration strategy is to indicate that the data in the second node is migrated to the destination node through the intermediary module.

[0033] Further, the migration strategy is a first migration strategy, and the migration of data from the second node to the destination node based on the migration strategy includes:

[0034] A target transmission path is determined in the network topology between the second node and the destination node; wherein, the network topology is determined based on the topological relationship between the nodes in the Redis cluster to which the second node belongs and the nodes in the Redis cluster to which the destination node belongs.

[0035] The data to be migrated in the second node is fragmented to obtain fragmented data, and the fragmented data is transmitted in parallel to the destination node through the target transmission path.

[0036] Furthermore, the method also includes:

[0037] During the process of migrating data from the second node to the destination node according to the migration strategy, the change information of the network status indicators is obtained;

[0038] If the change information is determined to exceed a preset change range, the migration strategy is switched from the first migration strategy to the second migration strategy, and the data that has not been migrated is migrated to the destination node according to the second migration strategy; the first migration strategy is to migrate data from the second node to the destination node; the second migration strategy is to migrate data from the second node to the destination node through the intermediary module.

[0039] According to a second aspect of this disclosure, a routing apparatus for Redis cluster federated data is provided, the apparatus comprising:

[0040] The first building module is used to divide the hash slots of the Redis cluster federation into multiple hash slot directories; wherein each hash slot directory contains multiple hash slots;

[0041] The second construction module is used to construct a static routing table; the static routing table is used to represent the first mapping relationship between each hash slot directory and each Redis cluster under the Redis cluster federation, and the second mapping relationship between each hash slot under each hash slot directory and the node in each Redis cluster;

[0042] The forwarding module is used to, when receiving an operation request from a client for the first node, look up the routing path of the first node in the Redis cluster federation based on the static routing table, and forward the operation request to the first node according to the routing path, so that the first node can perform the corresponding operation according to the operation request.

[0043] According to a third aspect of this disclosure, an electronic device is provided. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.

[0044] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the methods described above.

[0045] According to a fifth aspect of this disclosure, a computer program product is provided. The computer program product includes a computer program that, when executed by a processor, implements the methods described above in this disclosure.

[0046] This disclosure provides a routing method, apparatus, device, medium, and product for Redis cluster federated data. In this embodiment, firstly, the hash slots of the Redis cluster federation are divided to obtain multiple hash slot directories; each hash slot directory contains multiple hash slots; a static routing table is constructed; the static routing table represents a first mapping relationship between each hash slot directory and each Redis cluster under the Redis cluster federation, and a second mapping relationship between each hash slot under each hash slot directory and a node in each Redis cluster; when a client's operation request for a first node is received, the routing path of the first node is found in the Redis cluster federation based on the static routing table, and the operation request is forwarded to the first node according to the routing path, so that the first node performs the corresponding operation according to the operation request.

[0047] As described above, the technical solution disclosed herein divides the hash slots of the Redis cluster federation into multiple hash slot directories, each containing multiple hash slots. Then, it constructs a static routing table containing a first mapping relationship between the hash slot directories and the Redis cluster, and a second mapping relationship between each hash slot in the hash slot directory and nodes in the Redis cluster. When a client's operation request for the first node is received, the routing path to the first node can be quickly found based on this static routing table, and the operation request can be forwarded. This technical solution significantly simplifies the routing decision-making process by leveraging a clear two-level mapping relationship, reducing the complexity and time consumption of route lookup, and significantly improving the response speed of client operation requests. Furthermore, this... The structured hash slot directory partitioning and static routing table design in the publicly available technical solution ensure a balanced distribution of data across Redis clusters and nodes in the Redis cluster federation, effectively avoiding overload of a single node or cluster and guaranteeing overall system performance stability. Furthermore, the fixed mapping of the static routing table not only reduces resource consumption from dynamic routing calculations but also lowers the operational complexity of routing management. It can also accurately forward operation requests to the target first node, avoiding data access anomalies caused by routing errors, further improving the reliability and accuracy of data access in the Redis cluster federation and providing strong support for efficient data routing in large-scale Redis cluster federation. Attached Figure Description

[0048] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0049] Figure 1A flowchart illustrating a routing method for Redis cluster federated data provided as an exemplary embodiment of this disclosure;

[0050] Figure 2 A schematic diagram of the Redis cluster federation structure provided as an exemplary embodiment of this disclosure;

[0051] Figure 3 A schematic diagram of a multi-level hash slot structure provided in an exemplary embodiment of this disclosure;

[0052] Figure 4 A schematic diagram of the hash slot directory memory layout provided for an exemplary embodiment of this disclosure;

[0053] Figure 5 A flowchart illustrating a method for routing Redis cluster federated data as provided in another exemplary embodiment of this disclosure;

[0054] Figure 6 A flowchart illustrating a data migration method provided as an exemplary embodiment of this disclosure;

[0055] Figure 7 A flowchart for determining a migration strategy based on network state indicators, provided as an exemplary embodiment of this disclosure;

[0056] Figure 8 A flowchart of a data migration method provided as another exemplary embodiment of this disclosure;

[0057] Figure 9 A schematic block diagram of the functional modules of a routing apparatus for Redis cluster federated data provided in an exemplary embodiment of this disclosure;

[0058] Figure 10 A structural block diagram of an electronic device provided as an exemplary embodiment of this disclosure;

[0059] Figure 11 A structural block diagram of a computer system provided as an exemplary embodiment of this disclosure;

[0060] Figure 12 A structural block diagram of a computer program product provided for an exemplary embodiment of this disclosure. Detailed Implementation

[0061] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0062] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0063] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0064] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0065] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0066] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0067] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0068] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device. It is understood that the above notification and user authorization process is merely illustrative and does not constitute a limitation on the implementation of this disclosure; other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0069] In one embodiment, such as Figure 1 As shown, a method for routing federated data in a Redis cluster is provided, including the following steps:

[0070] Step 101: Divide the hash slots of the Redis cluster federation to obtain multiple hash slot directories.

[0071] Here, the executing entity can divide the hash slots of the Redis cluster federation into multiple hash slot directories, where each hash slot directory contains multiple hash slots.

[0072] In one possible implementation, the Redis cluster federation comprises multiple Redis clusters, each containing multiple nodes, such as... Figure 2 As shown, Figure 2 The diagram illustrates the structure of a Redis cluster federation. In this federation, the top-level Federation Cluster Manager (FCM) serves as the core component. Relying on modules such as cluster registration, cluster discovery, status monitoring, load balancing, migration scheduling, and fault tolerance and recovery, it enables cross-cluster data migration, load balancing, and full lifecycle management of the clusters. The intermediate Federation Proxy (FP), based on the global slot mapping and routing table maintained by the FCM, undertakes cross-cluster global management, data routing, load balancing, and synchronizes cluster discovery and slot mapping information. For Redis Cluster 1 and Redis Cluster 2 in the federation, the Cluster Interface Component Layer (CICL) is used for internal management of each cluster. CICL interacts with the FCM and manages the data migration switch, cluster monitoring components, and cluster proxy (CP) within the cluster through modules such as cluster internal management, communication, data management, status reporting, and node discovery.

[0073] In addition to the above, a Redis cluster federation may also include a first-stage data migration module and a cluster monitoring component. The cluster monitoring component can monitor the health status within the cluster, such as node CPU and memory usage, request latency, etc., and make corresponding adjustments and optimizations. It can also ensure data consistency within the cluster, including data replication, conflict resolution, and consistency checks. The first-stage data migration module can manage the slot migration process within the cluster, including data migration from the source node to the target node and updating slot mappings. It should be noted that the above embodiment is only an exemplary schematic diagram of the Redis cluster federation structure. In practical applications, a Redis cluster federation can include more components, which is not limited here.

[0074] In one possible embodiment, FCM can act as the execution entity to divide the hash slots of the Redis cluster federation into multiple hash slot directories. Each hash slot directory contains multiple hash slots. For example, in a certain Redis cluster federation, there are a total of 16384 hash slots. Dividing them according to the rule of one hash slot directory for every 1024 hash slots, we finally get 16 hash slot directories. Hash slot directory 0 contains hash slots 0 to 1023, hash slot directory 1 contains hash slots 1024 to 2047, and so on. Hash slot directory 15 contains hash slots 15360 to 16383. Each hash slot directory contains 1024 consecutive hash slots.

[0075] Step 102: Construct a static routing table.

[0076] Here, after the execution entity divides the hash slots of the Redis cluster federation into multiple hash slot directories, it can construct a static routing table. The static routing table is used to represent the first mapping relationship between each hash slot directory and each Redis cluster under the Redis cluster federation, as well as the second mapping relationship between each hash slot under each hash slot directory and the nodes in each Redis cluster.

[0077] In one possible implementation, FCM, acting as the execution entity, can establish a global multi-level hash slot structure among the Redis clusters in a Redis cluster federation to coordinate hash slots across different Redis clusters. Then, a static routing table is configured through this multi-level hash slot structure. Specifically, FCM first splits a complete hash slot into a "hash slot directory number" and a "directory number offset," such as... Figure 3 As shown, Figure 3An exemplary diagram of a multi-level hash slot structure is shown, where the hash slot directory number occupies m bits, and the directory number offset occupies n bits. The hash slot directory number identifies the hash slot directory to which the hash slot belongs; that is, different hash slot directories can be assigned to different Redis clusters. The directory number offset identifies the specific location of the hash slot, used for fine-grained routing within the Redis cluster. Then, FCM uses a static slot mapping algorithm to allocate hash slots to different Redis clusters, such as... Figure 4 As shown, Figure 4 An exemplary diagram of the hash slot directory memory layout is shown. The hash slot directory in FCM is laid out using static routing. Start records the ID number of the Redis cluster corresponding to the page directory table at position 0, and so on, eventually occupying a size of m×4 bytes. FCM can quickly find the IP address and other information of the cluster CP through the ID number of the Redis cluster. Finally, FCM obtains the first mapping relationship between each hash slot directory and each Redis cluster under the Redis cluster federation, as well as the second mapping relationship between each hash slot under each hash slot directory and the node in each Redis cluster by configuring the static routing table.

[0078] For example, in a Redis cluster federation containing Redis cluster A, Redis cluster B, and Redis cluster C, FCM first splits each complete hash slot in the federation into a 4-bit "hash slot directory number" and a 12-bit "directory number offset" to construct a multi-level hash slot structure. Assume a total of 16 hash slot directories are generated, covering 16384 hash slots, with each directory containing 4096 hash slots. Next, FCM uses a static slot mapping algorithm to assign hash slot directories 0-5 to Redis cluster A, directories 6-10 to Redis cluster B, and directories 11-15 to Redis cluster C. A static routing method is used to construct the hash slot directory memory layout. Starting from Start, the first six 4-byte entries store the "Redis cluster A-ID", the middle... Five storage entries are designated "Redis Cluster B-ID", and the last five are designated "Redis Cluster C-ID", occupying a total of 64 bytes. The Redis cluster IDs allow for quick lookup of information such as the CP (IP 192.168.1.101) of Redis Cluster A, the CP (IP 192.168.1.102) of Redis Cluster B, and the CP (IP 192.168.1.103) of Redis Cluster C. Finally, FCM configures a static routing table, specifying the first mapping relationship (e.g., hash slot directory 0 corresponds to cluster A, directory 6 corresponds to cluster B), and the second mapping relationship (e.g., under directory 0 of cluster A, hash slots 0-4095 are routed to specific nodes such as the master node A1 (192.168.1.101:6379) and the slave node A2 (192.168.1.101:6380) of cluster A).

[0079] Step 103: When a client's operation request for the first node is received, the routing path of the first node in the Redis cluster federation is found based on the static routing table, and the operation request is forwarded to the first node according to the routing path so that the first node can perform the corresponding operation according to the operation request.

[0080] Here, after the execution entity constructs the static routing table, when it receives an operation request from a client for the first node, it can look up the routing path of the first node in the Redis cluster federation based on the static routing table, and forward the operation request to the first node according to the routing path, so that the first node can perform the corresponding operation according to the operation request.

[0081] In one possible implementation, such as Figure 5 As shown, finding the routing path of the first node in a Redis cluster federation based on the static routing table includes the following steps:

[0082] Step 1031: Perform a hash operation on the address information of the first node to obtain the first hash value.

[0083] Here, after constructing the static routing table, the executing entity can perform a hash operation on the address information of the first node to obtain the first hash value.

[0084] In one possible embodiment, the Redis cluster federation includes Redis cluster A and Redis cluster B. FCM has completed the construction of a static routing table. Redis cluster A is responsible for hash slot directory 0. Hash slots 0-1023 in this directory correspond to the master node of Redis cluster A as the first node, whose address information is "192.168.1.10:6379". FCM takes the address information of the first node "192.168.1.10:6379" as input and performs calculation using the CRC16 hash algorithm to obtain the first hash value 0x3A5F.

[0085] Step 1032: Perform bitwise operations on the first hash value to obtain the target hash slot directory number and the target offset.

[0086] Here, after performing a hash operation on the address information of the first node to obtain the first hash value, the executing entity can perform bitwise operations on the first hash value to obtain the target hash slot directory number and the target offset.

[0087] In one possible implementation, bitwise operations are performed on the first hash value to obtain the target hash slot directory number and the target offset, including the following steps:

[0088] The target offset is obtained by performing a modulo operation on the total number of hash slots contained in the Redis cluster federation based on the first hash value;

[0089] The target offset is shifted based on a preset value to obtain the target hash slot directory number.

[0090] Specifically, the executing entity calculates the target hash slot directory number by performing a first formula on the first hash value, where the first formula is H1=Hash(key)%(m+n)>>n, H1 is the target hash slot directory number, and Hash(key) is the first hash value; then, the executing entity calculates the target offset by performing a second formula on the first hash value, where the second formula is H2=Hash(key)%(m+n), H2 is the target offset, m+n is the total number of hash slots in the Redis cluster federation, and n is a preset value.

[0091] In one possible embodiment, for example, the total number of hash slots in the Redis cluster federation is known to be 16384, i.e., m+n=16384, with a preset value of n=10. After the execution entity FCM performs a hash operation on the address information of the first node, it obtains the first hash value Hash(key)=2560. FCM calculates the target hash slot directory number using the first formula H1=Hash(key)%(m+n)>>n. FCM substitutes Hash(key)=2560, m+n=16384, and n=10 into the first formula, first calculating 2560%16384=2560, then shifting the result 10 bits to the right, 2560>>10=2, to obtain the target hash slot directory number H1=2. Subsequently, FCM calculates the target offset using the second formula H2=Hash(key)%(m+n), and after substituting the value, obtains H2=2560%16384=2560, i.e., the target offset is 2560.

[0092] Step 1033: Determine the routing path in the static routing table based on the target hash slot directory number and the target offset.

[0093] Here, after performing bitwise operations on the first hash value to obtain the target hash slot directory number and the target offset, the executing entity can determine the routing path in the static routing table based on the target hash slot directory number and the target offset.

[0094] In one possible embodiment, determining the routing path in the static routing table based on the target hash slot directory number and the target offset includes the following steps:

[0095] Based on the first mapping relationship in the static routing table, find the Redis cluster associated with the target hash slot directory number to obtain the target Redis cluster;

[0096] Based on the second mapping relationship in the static routing table, the node associated with the target offset is found in the target Redis cluster to obtain the first node, and the routing path is determined based on the location of the first node.

[0097] Specifically, after performing bitwise operations on the first hash value to obtain the target hash slot directory number and the target offset, the executing entity first searches for the Redis cluster associated with the target hash slot directory number based on the first mapping relationship in the static routing table to obtain the target Redis cluster. Then, based on the second mapping relationship in the static routing table, the executing entity searches for the node associated with the target offset in the target Redis cluster to obtain the target node, and determines the routing path based on the location of the target node.

[0098] In one possible implementation, for example, FCM has constructed a static routing table, where the first mapping relationship clearly defines the association between hash slot directories and Redis clusters, such as "hash slot directory number 2 → Redis cluster A" and "hash slot directory number 5 → Redis cluster B". The second mapping relationship records the association between offsets and nodes, such as "under hash slot directory number 2 in Redis cluster A, offsets 2000-3000 correspond to master node A-M1". FCM has performed a hash operation on the key "order:202405" requested by the client to obtain the first hash value 2560, and then calculated the target hash slot directory number H1=2 using a formula. Given an offset H2=2560, FCM first uses the first mapping relationship in the static routing table to find the Redis cluster associated with the target hash slot directory number 2, determining that the target Redis cluster is cluster A. Next, based on the second mapping relationship in the static routing table, FCM searches for the node associated with the target offset 2560 (belonging to the 2000-3000 range) in the node routing records of Redis cluster A, obtaining the target node as the master node A-M1. Finally, FCM combines the CP address of cluster A recorded in the static routing table with the address of the first node to determine the complete routing path as "client → CP of cluster A → target node A-M1".

[0099] In this implementation, firstly, the executing entity performs a hash operation on the address information of the first node to obtain a first hash value; then, the executing entity performs bitwise operations on the first hash value to obtain the target hash slot directory number and the target offset; finally, the executing entity determines the routing path in the static routing table based on the target hash slot directory number and the target offset.

[0100] As described above, in this embodiment, the execution entity performs a hash operation on the address of the first node to obtain a unique hash value, which can provide an accurate initial basis for route positioning. This allows the execution entity to quickly obtain the target hash slot directory number and offset through efficient bit operations, significantly reducing the routing calculation time. Finally, based on the static routing table pre-built by FCM, the first node can be quickly locked and the routing path can be determined without dynamically calculating routing information. This reduces the consumption of cross-cluster communication resources, avoids the routing delay caused by the increase of traditional cluster nodes, and ensures the stability and accuracy of routing decisions, thereby effectively improving the data routing efficiency and overall reliability of the Redis cluster federation.

[0101] In one possible embodiment, after the executing entity finds the routing path of the first node in the Redis cluster federation based on the static routing table, it can forward operation requests to the first node according to the routing path, so that the first node can perform the corresponding operation according to the operation request. The operation request includes basic key-value pair read and write operations, such as GET to get the value corresponding to a specified key, SET to store or update key-value pairs, and also covers data creation, deletion, modification and query (CRUD) related operations, such as DEL to delete a specified key, EXISTS to check if a key exists, HGET / HSET to operate on hash type data, LPUSH / LPOP to operate on list type data, etc. In addition, the operation request also covers basic data status query operations, such as TTL to check the expiration time of a key, TYPE to query the data type corresponding to a key, etc. It should be noted that these operation requests require the first node to perform corresponding data processing or status feedback according to the instructions, and after being forwarded through the routing path determined based on the static routing table, they can accurately reach the target first node to complete the operation.

[0102] This disclosure provides a routing method, apparatus, device, medium, and product for Redis cluster federated data. In this embodiment, firstly, the hash slots of the Redis cluster federation are divided to obtain multiple hash slot directories; each hash slot directory contains multiple hash slots; a static routing table is constructed; the static routing table represents a first mapping relationship between each hash slot directory and each Redis cluster under the Redis cluster federation, and a second mapping relationship between each hash slot under each hash slot directory and a node in each Redis cluster; when a client's operation request for a first node is received, the routing path of the first node is found in the Redis cluster federation based on the static routing table, and the operation request is forwarded to the first node according to the routing path, so that the first node performs the corresponding operation according to the operation request.

[0103] As described above, the technical solution disclosed herein divides the hash slots of the Redis cluster federation into multiple hash slot directories, each containing multiple hash slots. Then, it constructs a static routing table containing a first mapping relationship between the hash slot directories and the Redis cluster, and a second mapping relationship between each hash slot in the hash slot directory and nodes in the Redis cluster. When a client's operation request for the first node is received, the routing path to the first node can be quickly found based on this static routing table, and the operation request can be forwarded. This technical solution significantly simplifies the routing decision-making process by leveraging a clear two-level mapping relationship, reducing the complexity and time consumption of route lookup, and significantly improving the response speed of client operation requests. Furthermore, this... The structured hash slot directory partitioning and static routing table design in the publicly available technical solution ensure a balanced distribution of data across Redis clusters and nodes in the Redis cluster federation, effectively avoiding overload of a single node or cluster and guaranteeing overall system performance stability. Furthermore, the fixed mapping of the static routing table not only reduces resource consumption from dynamic routing calculations but also lowers the operational complexity of routing management. It can also accurately forward operation requests to the target first node, avoiding data access anomalies caused by routing errors, further improving the reliability and accuracy of data access in the Redis cluster federation and providing strong support for efficient data routing in large-scale Redis cluster federation.

[0104] In one embodiment, such as Figure 6 As shown, after forwarding the operation request to the first node according to the routing path, the following steps are also included:

[0105] Step 601: In response to the data migration instruction for the second node, determine the first Redis cluster associated with the second node in the Redis cluster federation.

[0106] Here, after forwarding the operation request to the first node according to the routing path, the executing entity can respond to the data migration instruction for the second node and determine the first Redis cluster associated with the second node in the Redis cluster federation. The first Redis cluster includes the Redis cluster to which the second node belongs, and / or the Redis cluster to which the destination node of the second node belongs. The destination node is the endpoint for the data migration of the second node.

[0107] In one possible implementation, for example, when Slave node A2 of Redis cluster A receives a data migration instruction for the second node "migrate the hash slot directory 4 managed by A2 to the new node B1", FCM will first determine the first Redis cluster associated with the second node based on the cluster registration information and static routing table it maintains. For example, if FCM finds that A2 belongs to Redis cluster A through the node information of A2 in the static routing table, then the first Redis cluster is Redis cluster A. If FCM finds that A2 belongs to another Redis cluster B through the node information of A2 in the static routing table, then the first Redis cluster is Redis cluster B.

[0108] Step 602: Determine the network status metrics for the first Redis cluster.

[0109] Here, after the executing entity identifies the first Redis cluster associated with the second node in the Redis cluster federation, it can determine the network status indicators of the first Redis cluster. Among them, the network status indicators include at least one of the following: network latency, bandwidth, and packet loss rate.

[0110] In one possible embodiment, for example, when the executing entity determines the network status indicators of the first Redis cluster, regarding network latency, it needs to monitor the round-trip time (RTT) of communication between nodes within the first Redis cluster, such as between the second node and other nodes in the same cluster, and between the first Redis cluster and the Federation Cluster Manager (FCM) and Federation Agent (FP). Latency data is obtained by periodically sending probe packets to reflect the response efficiency of data transmission within the cluster and with external components. Regarding bandwidth, the executing entity needs to statistically analyze the data transmission between nodes within the first Redis cluster, the real-time bandwidth usage between the cluster and external components, and the remaining available bandwidth, using the number of bytes transmitted per unit time as the statistical dimension to ensure sufficient bandwidth resources during data transmission and avoid blocking of business requests due to bandwidth saturation. Regarding packet loss rate, the executing entity needs to monitor the packet loss ratio of relevant communication links of the first Redis cluster (including links between nodes within the cluster and interaction links between the cluster and the Federation Agent (FP) and FCM), and calculate the packet loss rate by comparing the total number of sent packets with the total number of received packets.

[0111] Step 603: Determine the migration strategy based on network status indicators.

[0112] Here, after determining the network status indicators of the first Redis cluster, the executing entity can determine the migration strategy based on the network status indicators. The migration strategy is used to indicate whether to implement data migration through the intermediary module, which includes the data transmission switch and data transmission router in the Redis cluster federation.

[0113] In one possible embodiment, such as Figure 7 As shown, determining the migration strategy based on network status indicators includes the following steps:

[0114] Step 6031: Evaluate the first Redis cluster based on network status indicators and obtain the evaluation results.

[0115] Here, after the executing entity determines the network status indicators of the first Redis cluster, it can evaluate the first Redis cluster based on the network status indicators and obtain the evaluation results.

[0116] In one possible embodiment, the first Redis cluster is evaluated based on network status metrics to obtain the evaluation results, including the following steps:

[0117] Assign weights to each network status indicator;

[0118] The evaluation results are obtained by weighting each network state index based on its weights.

[0119] Specifically, after determining the network status indicators of the first Redis cluster, the executing entity first assigns a weight to each network status indicator; then, the executing entity performs a weighted calculation on each network status indicator based on the weights to obtain the evaluation result.

[0120] In one possible implementation, FCM assigns weights to the network latency, bandwidth, and packet loss rate of the first Redis cluster. For example, considering that the packet loss rate directly affects the integrity of data migration, the weight of the packet loss rate is set to 40%. The bandwidth determines the transmission efficiency of data migration, so the weight of the bandwidth is set to 35%. The network latency affects the migration response speed, so the weight of the network latency is set to 25%. FCM converts the actual detected values ​​of each network status indicator into a quantitative score of 0-100 points. For example, a network latency of 50ms corresponds to 80 points, the remaining bandwidth meets the migration requirements by 1.2 times, which corresponds to 90 points, and a packet loss rate of 0.1% corresponds to 95 points. Then, a weighted calculation is performed based on the weights, that is, the evaluation result = (network latency score × 25%) + (bandwidth score × 35%) + (packet loss rate score × 40%). After substituting the example score, the evaluation result is (80 × 0.25) + (90 × 0.35) + (95 × 0.4) = 20 + 31.5 + 38 = 89.5 points.

[0121] Step 6032: Determine the migration strategy based on the evaluation results.

[0122] Here, the executing entity evaluates the first Redis cluster, and after obtaining the evaluation results, it can determine the migration strategy based on the evaluation results.

[0123] In one possible embodiment, determining a migration strategy based on the evaluation results includes the following steps:

[0124] If the evaluation result is greater than the preset threshold, the migration strategy will be determined as the first migration strategy.

[0125] If the evaluation result is less than or equal to the preset threshold, the migration strategy will be determined as the second migration strategy.

[0126] Specifically, after evaluating the first Redis cluster, if the evaluation result is greater than a preset threshold, the execution entity determines the migration strategy as the first migration strategy, which instructs the second node to directly migrate data to the destination node; if the evaluation result is less than or equal to the preset threshold, the migration strategy is determined as the second migration strategy, which instructs the data in the second node to be migrated to the destination node through an intermediary module.

[0127] In one possible implementation, assuming the Federated Cluster Manager (FCM) pre-sets a preset threshold of 80 points for evaluating the network status of the first Redis cluster, continuing the previous example, if the FCM calculates a weighted network status evaluation result of 89.5 points for the first Redis cluster, which is greater than the preset threshold of 80 points, then the FCM determines the migration strategy as the first migration strategy. This means that the second node directly transmits the hash slot data to be migrated and related metadata to the destination node without the need for an intermediary module. If the evaluation result calculated by the FCM is 72 points, which is less than the preset threshold of 80 points, then the FCM determines the migration strategy as the second migration strategy. This means that the second node first uploads the data to be migrated to the intermediary module for temporary storage and verification. After the intermediary module confirms the data is complete, it then pushes the data to the destination node. The buffering and fault tolerance capabilities of the intermediary module reduce the impact of network instability on the migration.

[0128] Step 604: Migrate the data in the second node to the destination node based on the migration strategy.

[0129] Here, after determining the migration strategy based on network status indicators, the executing entity can migrate the data in the second node to the destination node based on the migration strategy.

[0130] In one possible implementation, assuming the execution entity FCM obtains an evaluation result based on the weighted calculation of the network status indicators of the first Redis cluster and has determined the corresponding migration strategy, for example, if the determined migration strategy is the first migration strategy, i.e., the evaluation result is greater than a preset threshold, FCM will send a direct migration instruction to the second node. After receiving the instruction, the second node first performs integrity verification on its own data to be migrated, such as the key-value pairs under the hash slot directory. After the verification is passed, it directly establishes a data transmission connection with the destination node and synchronizes the data to the destination node in batches. After receiving the data, the destination node verifies it batch by batch. After confirming that there are no errors, it reports the completion of reception to FCM. FCM then updates the node mapping relationship corresponding to this part of the data in the static routing table. If the determined migration strategy is the second migration strategy, i.e., the evaluation result is less than or equal to the preset threshold, FCM will send migration instructions to the second node and the intermediary module at the same time. The second node first encrypts the data to be migrated and transmits it to the intermediary module. The intermediary module stores the data and completes the verification. Then, according to the FCM instruction, the intermediary module decrypts the data and forwards it to the destination node. After the destination node receives and verifies the data, it reports back to FCM. FCM updates the static routing table and completes the data migration from the second node to the destination node.

[0131] In one possible embodiment, if the migration strategy is a first migration strategy, migrating data from the second node to the destination node based on the migration strategy includes the following steps:

[0132] The target transmission path is determined in the network topology between the second node and the destination node; wherein, the network topology is determined based on the topological relationship between the nodes in the Redis cluster to which the second node belongs and the nodes in the Redis cluster to which the destination node belongs.

[0133] The data to be migrated in the second node is fragmented to obtain fragmented data, and the fragmented data is transmitted in parallel to the destination node through the target transmission path.

[0134] In one possible embodiment, for example, assume that the Redis cluster to which the second node belongs is cluster X, and cluster X contains nodes X1, X2, and X3, where X2 is the second node. The core network device of cluster X is switch SX, and the IP address of SX is 192.168.10.1. The Redis cluster to which the destination node belongs is cluster Y, and cluster Y contains nodes Y1, Y2, and Y3, where Y1 is the destination node. The core network device of cluster Y is switch SY, and the IP address of SY is 192.168.20.1. The network topology between the Redis cluster to which the second node belongs and the Redis cluster to which the destination node belongs clearly records the connection relationships between nodes and core devices, and between core devices. For example, all nodes in cluster X are directly connected... All nodes in cluster Y are directly connected to switch SY. There are two direct communication links between SX and SY, namely link A and link B. Since the migration strategy is the first migration strategy, the executing entity FCM first detects the real-time network status of the two links. The detection results show that the network latency of link A is 20 milliseconds, the available bandwidth is 1.2Gbps, and the packet loss rate is 0.02%. The network latency of link B is 65 milliseconds, the available bandwidth is 550Mbps, and the packet loss rate is 0.2%. Considering the requirements of transmission efficiency and stability for direct migration in the first migration strategy, the executing entity determines link A as the target transmission path. The specific route of this path is: second node X2 → switch SX → link A → switch SY → destination node Y1.

[0135] Subsequently, the executing entity sends a data sharding instruction to the second node X2. The second node X2 stores the data to be migrated; for example, the total data volume is 10GB, corresponding to all key-value pairs in hash slot directory 8 of the Redis cluster federation, including user order cache data and product information cache data. The second node X2 shards this data to be migrated according to a specification of 2GB per shard, ultimately obtaining 5 shards, named Shard 1, Shard 2, Shard 3, Shard 4, and Shard 5 respectively. A unique SHA256 checksum is generated for each shard for integrity verification. After sharding is completed, the second node X2 initiates 5 independent transmission processes through the target transmission path link A, each transmission process corresponding to one shard, synchronously sending data to the destination node Y1. During transmission, each process provides real-time feedback on the transmission progress to the executing entity, for example, "Shard 2 has been 60% transmitted." When the destination node Y1 receives fragmented data, it verifies the data integrity using the corresponding SHA256 checksum after receiving each fragment. After successful verification, it returns a confirmation message of "fragment received successfully" to the second node X2 through the target transmission path, and then continues to receive the next fragment. When all 5 fragments have been transmitted and all verifications have passed, the destination node Y1 sends a notification of "all fragments received successfully" to the execution entity. The FCM records the transmission results, completing the parallel transmission of fragmented data under the first migration strategy.

[0136] In this embodiment, firstly, in response to the data migration instruction for the second node, the executing entity determines the first Redis cluster associated with the second node in the Redis cluster federation; wherein the first Redis cluster includes the Redis cluster to which the second node belongs, and / or the Redis cluster to which the destination node of the second node belongs; then, the executing entity determines the network status indicators of the first Redis cluster; wherein the network status indicators include at least: network latency, bandwidth, and packet loss rate; subsequently, the executing entity determines a migration strategy based on the network status indicators; wherein the migration strategy is used to indicate whether to implement data migration through an intermediary module; finally, the executing entity migrates the data in the second node to the destination node based on the migration strategy.

[0137] As described above, in this embodiment, the executing entity first locks the first Redis cluster associated with the second node, clearly defining the cluster scope involved in the migration and avoiding cross-cluster coordination chaos, thus defining clear boundaries for subsequent operations. Then, by monitoring indicators such as network latency, bandwidth, and packet loss rate, it provides quantitative basis for the migration strategy, solving the adaptation problem caused by subjective decision-making. Next, the executing entity determines whether to use the migration strategy of the intermediary module based on the indicators, which can flexibly adapt to different network conditions. When the network is good, the intermediary module can be omitted to improve efficiency; when the network is poor, the intermediary module can be used to ensure stable transmission. Finally, the migration is completed according to the strategy, which can shorten business interruption time and ensure data integrity, thereby improving the overall flexibility and reliability of Redis cluster federated data migration. It can effectively support the stable operation of Redis clusters in scenarios such as expansion and load balancing, reducing the impact of migration on business services.

[0138] In one possible embodiment, such as Figure 8 As shown, the routing method for Redis cluster federated data also includes the following steps:

[0139] Step 801: During the process of migrating data from the second node to the destination node according to the migration strategy, obtain information on changes in network status indicators.

[0140] Here, during the process of migrating data from the second node to the destination node according to the migration strategy, the Redis cluster can obtain information on changes in network status metrics.

[0141] In one possible implementation, for example, the Redis cluster instructs the second node A2 to begin migrating data to the destination node B1 according to the first migration strategy. For example, the total amount of data to be migrated is 15GB, initially divided into 7 shards of 2GB each, and transmitted in parallel through the target transmission path (Cluster A switch → dedicated direct link → Cluster B switch). During the migration process, FCM starts a real-time monitoring mechanism, collecting network status indicators of the first Redis cluster every 5 seconds and recording changes: In the initial stage of the migration (the first 15 minutes), the network latency stabilizes at... Around 35ms, the available bandwidth remained at 1.2Gbps with a packet loss rate of 0.05%. Starting from the 16th minute, the network latency gradually increased, reaching 110ms at 16 minutes and 10 seconds, and then remained between 105-120ms. At the same time, the available bandwidth began to decrease, dropping to 480Mbps at 16 minutes and 15 seconds, and then maintaining at 450-490Mbps. The packet loss rate also increased from 0.05% to 1.2%, and then stabilized between 1.2%-1.5% after 16 minutes and 20 seconds. The executing entity obtained the changes in these network status indicators in real time.

[0142] Step 802: If it is determined that the changed information exceeds the preset change range, the migration strategy is switched from the first migration strategy to the second migration strategy, and the data that has not been migrated is migrated to the destination node in accordance with the second migration strategy.

[0143] Here, after obtaining information about changes in network status indicators, the executing entity can switch the migration strategy from the first migration strategy to the second migration strategy if it determines that the changes exceed the preset range. The entity then migrates the data that has not yet been migrated to the destination node according to the second migration strategy. The first migration strategy involves migrating data from the second node to the destination node, while the second migration strategy involves migrating data from the second node to the destination node through an intermediary module.

[0144] In one possible embodiment, continuing the previous example, the FCM acquires information on changes in network status indicators. For example, starting from 16 minutes and 20 seconds, the network latency has exceeded 100ms for 10 seconds, the available bandwidth has been below 500Mbps for 10 seconds, and the packet loss rate has been above 1% for 10 seconds. The FCM determines that these changes exceed a preset range. Subsequently, the FCM sends a policy switching instruction to the second node A2, the destination node B1, and the intermediary module, switching the migration policy from the first migration policy to the second migration policy, i.e., migrating data through the intermediary module. At this time, the second node A2 has completed the migration of fragments 1 to 4 (a total of 8GB of data), and the remaining fragment 5... If fragment 7 (7GB of data) is not migrated completely, according to the second migration strategy, the second node A2 stops directly transmitting data to the destination node B1. Instead, it encrypts fragments 5 through 7 sequentially and transmits them to the intermediary module. After receiving each fragment, the intermediary module performs an integrity check. If the check passes, the fragment is stored in the local cache. After all three unmigrated fragments have been received, the intermediary module decrypts fragments 5 through 7 and forwards them to the destination node B1 one by one. After receiving each fragment, the destination node B1 checks it again. If it is correct, it reports the reception status to the execution entity. This process continues until all unmigrated data is transmitted. The FCM records the migration result, completing the entire data migration process.

[0145] In this embodiment, during the process of migrating data from the second node to the destination node according to the migration strategy, information on changes in network status indicators is obtained. Then, if it is determined that the changes exceed a preset range, the migration strategy is switched from the first migration strategy to the second migration strategy, and the data that has not been migrated is migrated to the destination node according to the second migration strategy.

[0146] As described above, in this embodiment, the executing entity acquires real-time network status indicator changes during data migration, enabling it to promptly capture situations such as increased network latency, decreased bandwidth, and increased packet loss rate. This avoids data transmission interruptions, packet loss, or efficiency degradation caused by continuously using the first migration strategy adapted to the original network. When the executing entity determines that the changed information exceeds the preset change range, it proactively switches the migration strategy from the first to the second migration strategy. By leveraging the buffering and fault-tolerance capabilities of the intermediary module, it compensates for transmission defects caused by deteriorating network conditions, ensuring that unmigrated data continues to migrate along a relatively stable path. This guarantees the integrity and continuity of data migration. This dynamic switching mechanism breaks the limitations of the traditional single strategy throughout the entire process, allowing migration to flexibly adapt to network fluctuations. Simultaneously, unmigrated data is processed in an orderly manner after the strategy switch, reducing migration retries or retransmissions caused by network problems, shortening the overall migration time, and reducing the impact on the business services of the Redis cluster to which the second node and the destination node belong, further supporting the stable operation of the cluster federation.

[0147] In one possible embodiment, a binary communication protocol (RFCP protocol) is also provided. This binary communication protocol, as a core component of the Redis cluster federated system, can replace the traditional RESP protocol. Its implementation steps are as follows: First, clarify the protocol's positioning and core value, namely, providing a standard for data exchange between system components to support cross-cluster operations, ensuring data security, system reliability, and performance optimization, while improving compatibility, maintainability, and scalability through standardized definitions. Next, define a unified binary data format and structure. The protocol includes a fixed 16-byte message header and a message body that changes as needed. The message header includes metadata such as magic number, version, message type, message ID, message length, checksum, source / destination node type and ID, etc. Messages are also divided into different secondary types, including broadcast messages with type identifier 0x1 and cluster registration messages with type identifier 0x2, ensuring standardized data exchange between different components such as the federated agent and cluster agent, and between different versions of Redis instances, reducing the risk of parsing errors and data inconsistencies. Then, reduce bandwidth consumption and improve efficient parsing through binary compression. Leveraging the advantages of low CPU overhead, support for complex operations such as scaling up and down, and embedded encryption and verification to enhance security, a TCP-based protocol processing module is constructed. This module includes a binary parsing module responsible for decoding and instruction parsing, a data compression module responsible for reducing the amount of data transmitted, and a security verification module responsible for preventing data tampering and interception, ensuring the reliability and stability of data transmission. Finally, a general message processing module is implemented for components such as the federated cluster manager to use for protocol sending and receiving. This module first creates, binds, and listens to server sockets through the socket management module and the epoll event loop module, while managing the registration and handling of events such as readability, writability, and errors of client sockets. Then, the RFCP protocol parsing and processing module parses the received data, classifies and processes messages according to their type (e.g., node status updates, data synchronization requests), and executes corresponding operations. Subsequently, a response message is constructed according to the business logic and sent to the client. At the same time, the error handling and resource management module handles abnormal situations, ensures the correct release of resources, and guarantees system stability. Overall, it achieves efficient, secure, and reliable communication and management across clusters.

[0148] In one possible embodiment, a method for implementing scaling up and down modes of a Redis cluster federation is also provided. The specific steps are as follows: When scaling up within a Redis cluster, the Redis cluster federation automatically adds new nodes after sensing an increase in load. Data shards are re-partitioned, and an incremental migration strategy is used to migrate some data to the new nodes, while multiple shards are processed in parallel to improve efficiency. When scaling down within a Redis cluster, the Redis cluster federation automatically identifies idle or low-load nodes and reallocates the data shards they carry to the remaining nodes, reducing redundant resources and ensuring uninterrupted service. When scaling up between Redis clusters, the Redis cluster federation migrates data from the source cluster to the new target cluster, supports cross-cluster replication of data shards, and leverages the global view of the federation architecture to ensure data integrity. According to the distributed load balancing, after the expansion is completed, the new cluster is added to the federated network to support cross-cluster load balancing and data synchronization. When scaling down between Redis clusters, the Redis cluster federation migrates the data of the low-load cluster to other clusters through incremental synchronization technology to ensure data integrity and consistency. Throughout the expansion and scaling process, the Redis cluster federation reduces the amount and time of migration by capturing incremental changes in Redis cluster data. Shard migration uses multi-threaded processing to avoid performance bottlenecks. At the same time, it relies on data version control and cross-cluster multi-level verification, such as data hash value comparison and data version check, to ensure data consistency. It also supports online expansion and scaling. During the migration, the cluster continues to process read and write requests. After the migration is completed, it automatically switches to the new node or target cluster, realizing the transparency of expansion and scaling to external services.

[0149] As described above, the technical solution disclosed herein addresses the limitations of traditional Redis clusters in terms of scalability, network communication, data consistency, and management complexity. It provides enhanced scalability, more flexible data management, lower cross-region access latency, and higher system reliability. Furthermore, this technical solution optimizes the routing efficiency of the federated agent and cluster agent through a multi-level hash slot mechanism, further strengthening system reliability and availability. It also addresses challenges encountered during large-scale cluster scaling and can dynamically select between direct communication and path optimization transmission methods based on network conditions, significantly improving the efficiency and stability of data migration.

[0150] By dividing each functional module according to its corresponding function, this disclosure provides a routing device for Redis cluster federated data. This routing device for Redis cluster federated data can be a server or a chip applied to a server. Figure 9 A schematic block diagram of the functional modules of a routing apparatus for Redis cluster federated data provided as an exemplary embodiment of this disclosure. Figure 9 As shown, the routing mechanism for the federated data in this Redis cluster includes:

[0151] The first building module 901 is used to divide the hash slots of the Redis cluster federation to obtain multiple hash slot directories; wherein each hash slot directory contains multiple hash slots;

[0152] The second construction module 902 is used to construct a static routing table; the static routing table is used to represent the first mapping relationship between each hash slot directory and each Redis cluster under the Redis cluster federation, and the second mapping relationship between each hash slot under each hash slot directory and the node in each Redis cluster;

[0153] The forwarding module 903 is used to, when receiving an operation request from a client for the first node, look up the routing path of the first node in the Redis cluster federation based on the static routing table, and forward the operation request to the first node according to the routing path, so that the first node performs the corresponding operation according to the operation request.

[0154] In one embodiment, the forwarding module 903 includes:

[0155] The first computing unit is used to perform a hash operation on the address information of the first node to obtain a first hash value;

[0156] The second calculation unit is used to perform bitwise operations on the first hash value to obtain the target hash slot directory number and the target offset.

[0157] The first determining unit is used to determine the routing path in the static routing table based on the target hash slot directory number and the target offset.

[0158] In one embodiment, the forwarding module 903 includes:

[0159] The third calculation unit is used to perform a modulo operation on the total number of hash slots contained in the Redis cluster federation based on the first hash value to obtain the target offset;

[0160] The fourth calculation unit is used to shift the target offset based on a preset value to obtain the target hash slot directory number.

[0161] In one embodiment, the forwarding module 903 includes:

[0162] The first lookup unit is used to find the Redis cluster associated with the target hash slot directory number based on the first mapping relationship in the static routing table, and obtain the target Redis cluster;

[0163] The second lookup unit is used to find the node associated with the target offset in the target Redis cluster based on the second mapping relationship in the static routing table, obtain the target node, and determine the routing path based on the location of the target node.

[0164] In one embodiment, the apparatus further includes:

[0165] A response module is configured to, in response to a data migration instruction for the second node, determine a first Redis cluster associated with the second node in the Redis cluster federation; wherein the first Redis cluster includes the Redis cluster to which the second node belongs, and / or the Redis cluster to which the destination node of the second node belongs, the destination node being the endpoint for the data migration of the second node;

[0166] The first determining module is used to determine the network status indicators of the first Redis cluster; wherein the network status indicators include at least one of the following: network latency, bandwidth, and packet loss rate;

[0167] The second determining module is used to determine a migration strategy based on the network status indicators; wherein the migration strategy is used to indicate whether to implement data migration through an intermediary module, and the intermediary module includes a data transmission switch and a data transmission router in the Redis cluster federation;

[0168] The first migration module is used to migrate data from the second node to the destination node based on the migration strategy.

[0169] In one embodiment, the second determining module includes:

[0170] An evaluation unit is used to evaluate the first Redis cluster based on the network status indicators and obtain an evaluation result;

[0171] The second determining unit is used to determine the migration strategy based on the evaluation results.

[0172] In one embodiment, the second determining module includes:

[0173] An allocation unit is used to assign weights to each of the network status indicators;

[0174] The fifth calculation unit is used to perform weighted calculations on each of the network state indicators based on the weights to obtain the evaluation results.

[0175] In one embodiment, the second determining module includes:

[0176] The third determining unit is configured to determine the migration strategy as the first migration strategy if the evaluation result is greater than a preset threshold; wherein the first migration strategy is to instruct the second node to migrate data to the destination node;

[0177] The fourth determining unit is used to determine the migration strategy as the second migration strategy if the evaluation result is less than or equal to a preset threshold; wherein the second migration strategy is to indicate that the data in the second node is migrated to the destination node through the intermediary module.

[0178] In one embodiment, the first migration module includes:

[0179] The fifth determining unit is used to determine the target transmission path in the network topology between the second node and the destination node; wherein the network topology is determined based on the topological relationship between the nodes in the Redis cluster to which the second node belongs and the nodes in the Redis cluster to which the destination node belongs;

[0180] The transmission unit is used to fragment the data to be migrated in the second node to obtain fragmented data, and transmit the fragmented data to the destination node in parallel through the target transmission path.

[0181] In one embodiment, the apparatus further includes:

[0182] The acquisition module is used to acquire the change information of the network status indicators during the process of migrating the data in the second node to the destination node according to the migration strategy;

[0183] The second migration module is used to switch the migration strategy from the first migration strategy to the second migration strategy when it is determined that the changed information exceeds a preset change range, and to migrate the data that has not been migrated to the destination node according to the second migration strategy; the first migration strategy is to migrate data from the second node to the destination node; the second migration strategy is to migrate the data in the second node to the destination node through the intermediary module.

[0184] Figure 10 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this disclosure. For example... Figure 10 As shown, the electronic device 1000 includes at least one processor 1001 and a memory 1002 coupled to the processor 1001. The processor 1001 can perform the corresponding steps in the methods disclosed in the embodiments of this disclosure.

[0185] The processor 1001 described above can also be called a central processing unit (CPU), which can be an integrated circuit chip with signal processing capabilities. Each step in the method disclosed in this embodiment can be implemented by the integrated logic circuitry in the processor 1001 or by software instructions. The processor 1001 can be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this embodiment can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in the memory 1002, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor 1001 reads information from the memory 1002 and, in conjunction with its hardware, completes the steps of the method described above.

[0186] Furthermore, various operations / processes according to this disclosure, implemented via software and / or firmware, can be transmitted from a storage medium or network to a computer system with a dedicated hardware architecture, such as... Figure 11 The computer system 1100 shown is equipped with the programs that constitute the software. When various programs are installed, the computer system is able to perform various functions, including functions such as those described above. Figure 11 A block diagram of a computer system provided for an exemplary embodiment of this disclosure.

[0187] Computer system 1100 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0188] like Figure 11As shown, the computer system 1100 includes a computing unit 1101, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded into random access memory (RAM) 1103 from a storage unit 1108. The RAM 1103 may also store various programs and data required for the operation of the computer system 1100. The computing unit 1101, ROM 1102, and RAM 1103 are interconnected via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0189] Multiple components in computer system 1100 are connected to I / O interface 1105, including: input unit 1106, output unit 1107, storage unit 1108, and communication unit 1109. Input unit 1106 can be any type of device capable of inputting information into computer system 1100. Input unit 1106 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of the electronic device. Output unit 1107 can be any type of device capable of presenting information and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1108 may include, but is not limited to, hard disks and optical disks. Communication unit 1109 allows computer system 1100 to exchange information / data with other devices via a network such as the Internet, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0190] The computing unit 1101 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1101 performs the various methods and processes described above. For example, in some embodiments, the methods disclosed in this disclosure can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1000 via ROM 1102 and / or communication unit 1109. In some embodiments, the computing unit 1101 can be configured to perform the methods disclosed in this disclosure by any other suitable means (e.g., by means of firmware).

[0191] This disclosure also provides a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform the methods disclosed in this disclosure.

[0192] The computer-readable storage medium in this disclosure can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The aforementioned computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specifically, the aforementioned computer-readable storage medium may include electrical connections based on one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0193] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0194] Figure 12 A computer program product 1200 is provided as an exemplary embodiment of the present disclosure. The computer program product 1200 includes a computer program 1201, wherein the computer program 1201, when executed by a processor, implements the methods disclosed in the embodiments of the present disclosure.

[0195] In embodiments of this disclosure, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer.

[0196] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0197] The modules, components, or units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules, components, or units do not necessarily constitute a limitation on the module, component, or unit itself.

[0198] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0199] The above description is merely an embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0200] While specific embodiments of this disclosure have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. A method for routing data in a Redis cluster federation, wherein the Redis cluster federation includes multiple Redis clusters, and each Redis cluster includes multiple nodes, characterized in that, include: The hash slots of the Redis cluster federation are divided into multiple hash slot directories; each hash slot directory contains multiple hash slots. Construct a static routing table; the static routing table is used to represent the first mapping relationship between each hash slot directory and each Redis cluster under the Redis cluster federation, and the second mapping relationship between each hash slot under each hash slot directory and each node in each Redis cluster; When a client requests an operation on the first node, the system searches for the routing path of the first node in the Redis cluster federation based on the static routing table, and forwards the operation request to the first node according to the routing path, so that the first node performs the corresponding operation according to the operation request. The step of finding the routing path of the first node in the Redis cluster federation based on the static routing table includes: Perform a hash operation on the address information of the first node to obtain the first hash value; The target offset is obtained by performing a modulo operation on the total number of hash slots contained in the Redis cluster federation based on the first hash value; The target offset is shifted based on a preset value to obtain the target hash slot directory number; Based on the first mapping relationship in the static routing table, the Redis cluster associated with the target hash slot directory number is found to obtain the target Redis cluster; Based on the second mapping relationship in the static routing table, the node associated with the target offset is found in the target Redis cluster to obtain the target node, and the routing path is determined based on the location of the target node.

2. The method according to claim 1, characterized in that, After forwarding the operation request to the first node according to the routing path, the method further includes: In response to a data migration instruction for the second node, a first Redis cluster associated with the second node is determined in the Redis cluster federation; wherein the first Redis cluster includes the Redis cluster to which the second node belongs, and / or the Redis cluster to which the destination node of the second node belongs, the destination node being the endpoint for the data migration of the second node; Determine the network status metrics of the first Redis cluster; wherein the network status metrics include at least one of the following: network latency, bandwidth, and packet loss rate; The migration strategy is determined based on the network status indicators; wherein, the migration strategy is used to indicate whether to implement data migration through an intermediary module, the intermediary module including the data transmission switch and data transmission router in the Redis cluster federation; The data in the second node is migrated to the destination node based on the migration strategy.

3. The method according to claim 2, characterized in that, The process of determining the migration strategy based on the network state indicators includes: Based on the network status indicators, the first Redis cluster is evaluated to obtain the evaluation results; The migration strategy is determined based on the evaluation results.

4. The method according to claim 3, characterized in that, The evaluation of the first Redis cluster based on the network status indicators, and the resulting evaluation, include: Assign weights to each of the network state indicators; The evaluation result is obtained by weighting each of the network state indicators based on the weights.

5. The method according to claim 3, characterized in that, The process of determining the migration strategy based on the evaluation results includes: If the evaluation result is greater than a preset threshold, the migration strategy is determined as the first migration strategy; wherein, the first migration strategy is to instruct the second node to migrate data to the destination node; If the evaluation result is less than or equal to a preset threshold, the migration strategy is determined as the second migration strategy; wherein, the second migration strategy is to indicate that the data in the second node is migrated to the destination node through the intermediary module.

6. The method according to claim 5, characterized in that, The migration strategy is a first migration strategy, and the migration of data from the second node to the destination node based on the migration strategy includes: A target transmission path is determined in the network topology between the second node and the destination node; wherein, the network topology is determined based on the topological relationship between the nodes in the Redis cluster to which the second node belongs and the nodes in the Redis cluster to which the destination node belongs. The data to be migrated in the second node is fragmented to obtain fragmented data, and the fragmented data is transmitted in parallel to the destination node through the target transmission path.

7. The method according to claim 3, characterized in that, The method further includes: During the process of migrating data from the second node to the destination node according to the migration strategy, the change information of the network status indicators is obtained; If the change information is determined to exceed a preset change range, the migration strategy is switched from the first migration strategy to the second migration strategy, and the data that has not been migrated is migrated to the destination node according to the second migration strategy; the first migration strategy is to migrate data from the second node to the destination node; the second migration strategy is to migrate data from the second node to the destination node through the intermediary module.

8. A routing device for federated data in a Redis cluster, characterized in that, include: The first building module is used to divide the hash slots of the Redis cluster federation into multiple hash slot directories; wherein each hash slot directory contains multiple hash slots; The second construction module is used to construct a static routing table; the static routing table is used to represent the first mapping relationship between each hash slot directory and each Redis cluster under the Redis cluster federation, and the second mapping relationship between each hash slot under each hash slot directory and the node in each Redis cluster; The forwarding module is used to, when receiving an operation request from a client for the first node, look up the routing path of the first node in the Redis cluster federation based on the static routing table, and forward the operation request to the first node according to the routing path, so that the first node can perform the corresponding operation according to the operation request; The forwarding module is further configured to: Perform a hash operation on the address information of the first node to obtain the first hash value; The target offset is obtained by performing a modulo operation on the total number of hash slots contained in the Redis cluster federation based on the first hash value; The target offset is shifted based on a preset value to obtain the target hash slot directory number; Based on the first mapping relationship in the static routing table, the Redis cluster associated with the target hash slot directory number is found to obtain the target Redis cluster; Based on the second mapping relationship in the static routing table, the node associated with the target offset is found in the target Redis cluster to obtain the target node, and the routing path is determined based on the location of the target node.

9. An electronic device, characterized in that, include: At least one processor; Memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the method as described in any one of claims 1-7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images