A Redis Cluster-based topology synchronization method

Abstract

The application provides a Redis Cluster-based topology synchronization method, comprising the following steps: S1, obtaining virtual machine online related information and preparing related configuration parameters; S2, obtaining the network card data of the online CVK under the VPC and performing incremental supplement; S3, calculating the changed version number based on the VPC and splicing the version number in the unique identifier of Redis; S4, calculating the changed version number based on the CVK and splicing the version number to be refreshed in the unique identifier of Redis; S5, calculating the Redis Slot belonging to the available area; S6, refreshing the version number recorded in Redis based on the Redis lua script in steps S3 and S4. The application has the beneficial effects that the Redis Cluster-based topology synchronization method and the Redis Cluster-based OVN configuration issuing method can quickly perform data fragmentation, data replication and fault reporting, improve the throughput of VPC configuration issuing, reduce the configuration data volume, improve the fault switching capability and ensure that the VPC configuration can be sequentially issued to various devices.

Description

Technical Field

[0001] This invention belongs to the field of cloud computing, and in particular relates to a topology synchronization method based on Redis Cluster. Background Technology

[0002] VPCs, as the foundation of cloud computing networks, provide users with custom private networks, logically and completely isolated from each other. VPC configurations, as fundamental to cloud computing communication, need to be accurately, efficiently, systematically, and persistently distributed to designated devices. Traditional VPC configurations require distribution via physical devices such as firewalls, leaf switches, and border switches. This reliance on physical devices leads to poor scalability and high costs. Therefore, OVN was introduced to facilitate VPC configuration distribution. OVN extends OVS, overcoming its limitations such as single-machine storage and poor cluster scalability. It utilizes a multi-data source storage approach, running OVN Northbound and OVN Southbound databases on a central node. The Northbound database stores logical network data; the Southbound database converts the logical network data format from the Northbound database to the physical network data format and stores it. The distribution of network L2 and L3 layer configuration data is accomplished through data storage comparison, forwarding, and storage. While introducing OVN's powerful control capabilities, it also brings data storage pressure. Since OVN's north-south database resides independently on CVK devices, its data volume is relatively small compared to the control layer data. As the number of devices and virtual machines within the availability zone increases, the data volume in the network control layer grows exponentially. When a node's Redis is used for data storage, it impacts OVN's data synchronization capabilities, resulting in longer latency and slower implementation times for network communication configurations. Horizontally scaling up the Redis cluster involves introducing Redis Cluster and dividing resources into multiple blocks based on availability zones. This reduces the storage size of each block and improves data retrieval capabilities. Simultaneously, the high availability introduced by Redis Cluster ensures that network communication service unavailability is avoided due to single-node failures in multi-availability zone scenarios. This improves service availability, ensuring a smooth switchover of network services after failures, thus enhancing overall availability. Summary of the Invention

[0003] In view of this, the present invention aims to propose a topology synchronization method based on Redis Cluster to at least solve one of the problems in the background art.

[0004] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0005] A topology synchronization method based on Redis Cluster includes the following steps:

[0006] S1. Obtain information related to the virtual machine going online and prepare relevant configuration parameters;

[0007] S2. Obtain the network card data that is currently online on the CVK under the VPC and perform incremental supplementation;

[0008] S3. Calculate the change version number based on VPC and concatenate the version number with the unique identifier in Redis;

[0009] S4. Calculate the change version number based on CVK and concatenate it with the unique identifier of the version number to be refreshed in Redis;

[0010] S5. Calculate the Redis Slot to which the availability zone belongs;

[0011] S6. Based on the Redis Lua script, refresh the version number recorded in Redis in steps S3 and S4;

[0012] S7. Push the network card information calculated in step S2;

[0013] S8. Start AOF file listening and synchronize the AOF files of the Redis master slot and replica slot;

[0014] S9. Start listening to CVK report alarms and determine whether data needs to be refreshed synchronously. If so, refresh the information of the current VPC and the SLOT to which the CVK belongs.

[0015] S10. Send an HTTP request to distribute the configuration information corresponding to the currently online network card and record the operation log;

[0016] S11. Listen for Redis script execution callbacks and record operation logs.

[0017] Furthermore, the unique identifier format in step S3 is as follows:

[0018] VPC:HOST:ZONE_{Availability Zone ID}:{l3vni}.

[0019] Furthermore, the unique identifier format in step S4 is as follows:

[0020] VPC:CVK:ZONE_{Availability Zone ID}:{l3vni}.

[0021] Furthermore, this solution discloses an electronic device, including a processor and a memory communicatively connected to the processor and used to store executable instructions of the processor, wherein the processor is used to execute a topology synchronization method based on Redis Cluster.

[0022] Furthermore, this solution discloses a server including at least one processor and a memory communicatively connected to the processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the processor to cause the at least one processor to perform a Redis Cluster-based topology synchronization method.

[0023] Furthermore, this solution discloses a computer-readable storage medium storing a computer program, which is executed by a processor as a Redis Cluster-based topology synchronization method.

[0024] Compared with existing technologies, the Redis Cluster-based topology synchronization method described in this invention has the following advantages:

[0025] The present invention discloses a topology synchronization method based on Redis Cluster. The OVN configuration distribution method based on Redis Cluster can quickly perform data sharding, data replication, and fault reporting, thereby improving the throughput of VPC configuration distribution, reducing the amount of configuration data, improving fault switching capability, and ensuring that VPC configurations can be distributed to various devices in an orderly manner. Attached Figure Description

[0026] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0027] Figure 1 This is a schematic diagram of a topology synchronization method based on Redis Cluster according to an embodiment of the present invention. Detailed Implementation

[0028] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0029] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0030] This solution ensures ordered partitioning of CVK configurations within a VPC: Data is partitioned using Redis Cluster Hash Tags, guaranteeing that data within the same availability zone of the VPC resides in a fixed Redis slot. Distributed locks and single-threaded execution are employed to ensure the ordered nature of data received by the OVN central control node within the availability zone and concurrent execution within the region, thereby guaranteeing the ordered and efficient distribution of configurations to devices.

[0031] This solution splits data from multiple dimensions to reduce data fetch volume: Data is split and sharded for storage based on VPC, CVK, and availability zones, reducing data fetch volume, improving the efficiency of configuration distribution data transmission, and increasing the throughput of VPC configuration distribution.

[0032] This solution introduces the concept of Redis replica slots to ensure reasonable fault migration: by backing up data into replica slots, high availability of network services is ensured when the primary Redis node experiences data anomalies.

[0033] This solution leverages Redis's persistent data file feature to ensure the persistence of VPC configurations: Redis data is persisted to the server's disk as files, and is split into multiple smaller files, which indirectly ensures the persistence of VPC-related device configurations.

[0034] This solution improves data transmission efficiency through binary data compression: by performing binary compression on VPC configuration data, the data packets are reduced in size, improving transmission efficiency and saving network bandwidth;

[0035] This solution, based on Redis Cluster's OVN configuration distribution method, can quickly perform data sharding, data replication, and fault reporting, thereby improving the throughput of VPC configuration distribution, reducing the amount of configuration data, enhancing failover capabilities, and ensuring that VPC configurations can be distributed to various devices in an orderly manner.

[0036] 1. Data sharding improves cluster scalability and reduces the cost of large-scale cluster data transmission;

[0037] 2. By persisting logs, configurations that failed to be distributed can be traced back and re-distributed.

[0038] 3. Introduce Redis Slot replicas, using Redis AOF file synchronization, to prevent configuration errors caused by partitioned Redis data anomalies, thereby improving service high availability;

[0039] 4. Introduce a fault reporting mechanism to automatically reissue configurations and improve operation and maintenance capabilities;

[0040] The topology synchronization method based on Redis Cluster is as follows:

[0041] 1. Obtain information related to the virtual machine going online and prepare relevant configuration parameters;

[0042] 2. Obtain the network interface card (NIC) data currently online on the CVK under the VPC and perform incremental supplementation;

[0043] 3. Calculate the change version number (timestamp) based on VPC, and concatenate it with the unique identifier of the version number in Redis, in the following format:

[0044] VPC:HOST:ZONE_{Availability Zone ID}:{l3vni};

[0045] 4. Calculate the change version number (timestamp) based on CVK, and concatenate it with the unique identifier of the version number to be refreshed in Redis;

[0046] The format is as follows: VPC:CVK:ZONE_{Availability Zone ID}:{l3vni};

[0047] 5. Calculate the Redis Slot to which the availability zone belongs;

[0048] 6. Using a Redis Lua script, refresh the version number recorded in Redis in steps 3 and 4;

[0049] 7. Push the network card information calculated in step 2;

[0050] 8. Start AOF file listening and synchronize the AOF files of the Redis master slot and replica slot;

[0051] 9. Start the listener to monitor CVK reports and alarms, determine whether data needs to be refreshed synchronously, and if so, refresh the current VPC completely;

[0052] And information about the SLOT to which CVK belongs;

[0053] 10. Send an HTTP request to distribute the configuration information corresponding to the currently online network card and record the operation log;

[0054] 11. Listen for Redis script execution callbacks and record operation logs. The flowchart for the Canal-based network topology synchronization method is as follows: Figure 1 As shown.

[0055] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0056] In the several embodiments provided in this application, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the division of units described above is merely a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The aforementioned units may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention according to actual needs.

[0057] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

[0058] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A topology synchronization method based on Redis Cluster, characterized in that, Includes the following steps: S1. Obtain information related to the virtual machine going online and prepare relevant configuration parameters; S2. Obtain the network interface card data that is currently online on the CVK under the VPC and perform incremental supplementation; S3. Calculate the change version number based on VPC and concatenate the version number with the unique identifier in Redis; S4. Calculate the change version number based on CVK and concatenate it with the unique identifier of the version number to be refreshed in Redis; S5. Calculate the Redis Slot to which the availability zone belongs; S6. Based on the Redis Lua script, refresh the version number recorded in Redis in steps S3 and S4; S7. Push the network card information calculated in step S2; S8. Start AOF file listening and synchronize the AOF files of the Redis master slot and replica slot; S9. Start listening to CVK report alarms and determine whether data needs to be refreshed synchronously. If so, refresh all information of the current VPC and the slot to which the CVK belongs. S10. Send an HTTP request to distribute the configuration information corresponding to the currently online network card and record the operation log; S11. Listen for Redis Lua script execution callbacks and record operation logs; The unique identifier format in step S3 is as follows: VPC:HOST:ZONE_{Availability Zone ID}:{L3vni}; The unique identifier format in step S4 is as follows: VPC:CVK:ZONE_{Availability Zone ID}:{L3vni}.

2. An electronic device, comprising a processor and a memory communicatively connected to the processor and used for storing processor-executable instructions, characterized in that: The processor is used to execute the RedisCluster-based topology synchronization method as described in claim 1.

3. A server, characterized in that: It includes at least one processor and a memory communicatively connected to the processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the processor to cause the at least one processor to perform a Redis Cluster-based topology synchronization method as described in claim 1.

4. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, it implements the topology synchronization method based on Redis Cluster as described in claim 1.

Images