Redis master-slave instance switching method and device, storage medium
By combining monitoring programs and hook programs, the heartbeat changes of Redis instances are monitored in real time and the master-slave roles are switched, which solves the problem of unstable access addresses of Redis instances in Kubernetes clusters and ensures the stability of the access entry point.
Patent Information
- Application Number
- CN202310060406.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-01-18
- Publication Date
- 2026-02-17
- Estimated Expiration
- 2043-01-18
AI Technical Summary
In a Kubernetes cluster, the access addresses of Redis master and slave instances are susceptible to cluster jitter, resulting in a lack of stable access points.
The system monitors the heartbeat changes of Redis instances in real time using a monitoring program, switches the master and slave instance roles in case of anomalies using preset election rules, and updates Pod tags through a hook program to ensure the stability of the access point.
It enables timely failover when the master instance fails, prevents Pod restarts, maintains the stability of the Redis master-slave instance access addresses, and provides a stable access point.
Smart Images

Figure CN116473311B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer, in particular to a Redis master-slave instance switching method and device, storage medium and computer equipment. BACKGROUND
[0002] Redis(Remote Dictionary Server) mainly runs on Kubernete cluster, and is automatically deployed by Operator to deploy Redis related Kubernete resources, Service responsible for request forwarding and load balancing, and program resources triggered by sentinel switching master-slave label event driver, mainly including StatefulSet controller of Redis master-slave instance, Deployment controller of Redis sentinel, and ConfigMap related to Redis instance and Redis sentinel configuration information.
[0003] In the cloud environment managed by Kubernete, the master instance, slave instance and sentinel instance of Redis are regarded as the smallest unit Pod in the Kubernete system. Unstable influence caused by cluster environment jitter has a great probability to cause Pod restart. After restart, Pod drift occurs, and the access address of the master instance and slave instance of Redis also changes, resulting in lack of a stable access entry to the outside. SUMMARY
[0004] The purpose of the present application is to at least solve one of the above technical defects, in particular, the technical defect that the unstable cluster environment jitter causes the access address of the master instance and slave instance of Redis to change, resulting in lack of a stable access entry to the outside.
[0005] The present application provides a Redis master-slave instance switching method, which comprises:
[0006] obtaining a monitoring program and a hook program, and using the monitoring program to listen to the heartbeat change of each instance in Redis in real time;
[0007] When the heartbeat exception of any instance in all instances is monitored, the instance with the heartbeat exception is taken as a first instance, and whether the first instance is the master instance of the Redis is judged;
[0008] If the first instance is the master instance of the Redis, a slave instance is elected as a second instance from all slave instances associated with the first instance according to a preset election rule, and the monitoring program is triggered to switch the role, the role of the second instance is switched to the master instance of the Redis, and the role of the first instance is switched to the slave instance of the second instance.
[0009] acquire, by the monitoring program, a first parameter corresponding to the first instance and a second parameter corresponding to the second instance, and pass the first parameter and the second parameter to the hook program;
[0010] switch, by the hook program, the Pod labels of the first instance and the second instance based on the first parameter and the second parameter.
[0011] Optionally, the monitoring program and the hook program comprise:
[0012] determining a custom resource definition (CRD) comprising configurations of creating the monitoring program and the hook program;
[0013] generating the monitoring program and the hook program based on the CRD.
[0014] Optionally, the monitoring program listens to the heartbeat changes of each instance in Redis, comprising:
[0015] sending, by the monitoring program, a heartbeat detection command to each instance in Redis at a preset frequency;
[0016] acquiring, based on the heartbeat detection command, a heartbeat detection result of each instance;
[0017] listening to the heartbeat changes of each instance according to the heartbeat detection result of each instance.
[0018] Optionally, the monitoring program listens to the heartbeat changes of each instance in Redis, comprising:
[0019] If it is found that the heartbeat interval time of any instance exceeds a preset heartbeat timeout time, it is determined that the heartbeat of the instance is abnormal.
[0020] Optionally, the method further comprises:
[0021] acquiring a first instance ID and first configuration information corresponding to the first instance and a second instance ID and second configuration information corresponding to the second instance;
[0022] replacing the first instance ID and the second instance ID, and modifying the first configuration information corresponding to the first instance and the second configuration information corresponding to the second instance according to the first instance ID and the second instance ID, so as to realize the role switching of the first instance and the second instance.
[0023] Optionally, the method further comprises:
[0024] Real-time detection is performed on the Pod labels of all master instances in Redis. When it is detected that the role of any master instance does not conform to the Pod label, a slave instance corresponding to the Pod label is determined, and the Pod labels of the master instance and the slave instance are switched.
[0025] Optionally, the method further comprises:
[0026] If the first instance is not a master instance of the Redis, the connection between the first instance and a master instance associated with the first instance is disconnected.
[0027] The application further provides a Redis master-slave instance switching device, comprising:
[0028] A program acquisition module is configured to acquire a monitoring program and a hook program, and listen to the heartbeat changes of each instance in Redis by using the monitoring program.
[0029] An instance judgment module is configured to, when it is detected that there is a heartbeat exception of any instance in all instances, take the instance with the heartbeat exception as a first instance, and judge whether the first instance is a master instance of the Redis.
[0030] An instance role switching module is configured to, according to a preset election rule, elect a slave instance as a second instance from all slave instances associated with the first instance, trigger the monitoring program to perform role switching, switch the role of the second instance to a master instance of the Redis, and switch the role of the first instance to a slave instance of the second instance.
[0031] A parameter determination module is configured to determine a first parameter corresponding to the first instance and a second parameter corresponding to the second instance by using the monitoring program, switch the role of the second instance to a master instance of the Redis, and switch the role of the first instance to a slave instance of the second instance.
[0032] An instance label switching module is configured to switch the Pod labels of the first instance and the second instance by using the hook program based on the first parameter and the second parameter.
[0033] The application further provides a storage medium, wherein the storage medium stores computer readable instructions. When the computer readable instructions are executed by one or more processors, the one or more processors perform the steps of the Redis master-slave instance switching method according to any one of the above embodiments.
[0034] The application further provides a computer device, comprising one or more processors and a memory.
[0035] The memory stores computer readable instructions which, when executed by the one or more processors, perform the steps of the Redis master-slave instance switching method of any of the above embodiments.
[0036] From the above technical solutions, the embodiments of the present application have the following advantages:
[0037] The Redis master-slave instance switching method, device, storage medium and computer device provided by the present application can obtain a monitoring program and a hook program before switching the Redis master-slave instances, so as to switch the roles and Pod labels of the master-slave instances when the master instance abnormally, and use the monitoring program to listen to the heartbeat changes of each instance in Redis in real time. When it is found that there is a heartbeat abnormality of the master instance in all instances, a slave instance can be elected from all slave instances associated with the master instance according to a preset election rule, and the monitoring program is triggered to switch the roles of the master-slave instances. The role of the elected slave instance is switched to a new master instance of Redis, and the role of the abnormal master instance is switched to a slave instance of the new master instance. This prevents the Pod from being restarted due to the long-term abnormality of the master instance, thereby changing the access address of the Redis master-slave instances. Then, the corresponding parameters of the new master-slave instances after the role switching can be obtained and transmitted to the hook program, so as to trigger the hook program to switch the Pod labels. The hook program is used to switch the Pod labels of the new master instance and the new slave instance, complete the whole switching process of the Redis master-slave instances, and maintain a stable access entry to the outside. BRIEF DESCRIPTION OF DRAWINGS
[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.
[0039] Figure 1 A flowchart of a Redis master-slave instance switching method provided by an embodiment of the present application is shown in the figure.
[0040] Figure 2 A Redis failover flowchart of a Redis master-slave instance switching method provided by an embodiment of the present application is shown in the figure.
[0041] Figure 3 A structure diagram of a Redis master-slave instance switching device provided by an embodiment of the present application is shown in the figure.
[0042] Figure 4An internal structure schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0043] The technical solutions in the embodiments of the present application will be clearly and completely described in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.
[0044] In a Kubernete-managed cloud environment, the master instance, the slave instance and the sentinel instance of Redis are regarded as the minimum unit of Pod in the Kubernete system. Unstable influence caused by jitter of the cluster environment has a great probability to cause the Pod to restart. After the restart, the Pod will drift, and the access addresses of the master instance and the slave instance of Redis will change, resulting in a lack of a stable access entry to the outside.
[0045] Based on this, the present application provides the following technical solutions, please see the following:
[0046] In an embodiment, as shown in Figure 1 , Figure 1 A flowchart of a Redis master-slave instance switching method provided in an embodiment of the present application; the present application provides a Redis master-slave instance switching method, which can include:
[0047] S110: Obtain the monitoring program and the hook program, and use the monitoring program to listen to the heartbeat change of each instance in Redis in real time.
[0048] In the embodiment, in order to listen to the running state of Redis in the cluster and timely transfer when the Redis fails, the monitoring program and the hook program can be configured in the cluster, and the monitoring program is used to listen to the heartbeat change of each instance in Redis in real time, so as to timely trigger the monitoring program and the hook program to switch the master-slave instance when the heartbeat of the master instance of Redis is abnormal, and timely transfer the failure.
[0049] It can be understood that Redis is a key-value storage system, which is mainly run on a Kubernete cluster. Redis supports master-slave synchronization, and data can be synchronized from the master instance to any number of slave instances. On the Kubernete cluster, the master instance can perform read and write operations, and when the read and write operations cause data changes, the data is automatically synchronized to the slave instance. The data stream is unidirectional, and can only be synchronized from the master instance to the slave instance. The slave instance is generally read-only and receives data synchronized from the master instance. One master instance can have multiple slave instances, but one slave instance can only correspond to one master instance.
[0050] Further, in the cluster, when the master instance is down, a slave instance needs to be switched to a master instance, which will cause the cluster to be unavailable for a period of time. Therefore, the Redis cluster mode mostly adopts the sentinel mode. The sentinel is an independent monitoring program that can run independently. The running state of the master instance and the slave instance in Redis is monitored. When the sentinel detects that the Redis master instance is down, the slave instance is automatically switched to the master instance, and the master-slave switching process is converted from manual to automatic. At the same time, it also reduces the duration of cluster unavailability caused by the master instance being down.
[0051] S120: When it is detected that there is a heartbeat exception of any instance in all instances, the instance with the heartbeat exception is taken as a first instance, and whether the first instance is a master instance of the Redis is judged.
[0052] In this step, after the monitoring program is obtained through S110, the monitoring program can be used to monitor all instances in Redis. When it is detected that there is a heartbeat exception of any instance in all instances, the instance with the heartbeat exception is taken as a first instance, and whether the first instance is a master instance of the Redis is judged.
[0053] Specifically, in the cluster, information exchange is needed between the master instance and the slave instance. The heartbeat mechanism can be used for maintenance to keep both connections online. After the monitoring program is configured, the monitoring program can establish a connection to all master instances and slave instances on Redis, monitor the heartbeat of all instances, so as to respond in time and transfer the fault when the master instance fails, so as to maintain the normal operation of the cluster.
[0054] Further, in the cluster, to reduce the misjudgment of the monitoring program, multiple monitoring programs are generally configured. When judging whether the master instance has a heartbeat exception, it is not determined by one monitoring program. Only when most of the monitoring programs think that the master instance has gone offline, the master instance will be marked as offline. This judgment mechanism is majority over minority. When the master instance is marked as offline, the master-slave switching mode of the monitoring program is triggered.
[0055] For example, in a Kubernetes cluster deployment, there is a set of one master and two slaves of three sentinels of Redis instances, and there is a heartbeat detection mechanism among the three instances, which is used to keep the connection between the master and slave instances online, and at the same time, the sentinel can be used to monitor the heartbeat of the three instances in the Kubernetes cluster. When all sentinels are monitoring the three instances, if all instances respond to two or more sentinels within a specified time, it is determined that all instances are in a live state, and if there is an instance that does not respond or only responds to one sentinel within a specified time, the sentinel will determine that the instance heartbeat is abnormal.
[0056] S130: According to the preset election rule, a slave instance is elected as a second instance from all slave instances associated with the first instance, and a monitoring program is triggered to switch the roles of the first and second instances, the role of the second instance is switched to the master instance of Redis, and the role of the first instance is switched to the slave instance of the second instance.
[0057] In this step, after the monitoring program detects the heartbeat abnormality of the first instance and determines that the first instance has been offline, a slave instance can be elected as a second instance from all slave instances associated with the first instance according to the preset election rule, and the monitoring program is triggered to switch the roles of the first and second instances, the role of the second instance is switched to the master instance of Redis, and the role of the first instance is switched to the slave instance of the second instance, preventing the master instance from being restarted for a long time and changing the access address of the Redis master-slave instance.
[0058] Specifically, when the master instance in the cluster is abnormal, it will cause the entire cluster to be unavailable, so a slave instance needs to be elected from all slave instances associated with the first instance and switched to the master instance. The data in all slave instances associated with the first instance is synchronized with the data of the first instance. According to the preset election rule, a slave instance is elected as a second instance, and the monitoring program can be triggered to switch the roles of the first and second instances, thereby restoring the normal operation of the entire cluster.
[0059] Further, in a cluster deployed with a master and multiple slave Redis instances, when a new master instance is elected from all slave instances associated with the first instance after the heartbeat of the master instance is abnormal, the preset election rule can be used to elect all slave instances. The election process can be a combination of screening and sorting. The monitoring program can screen out slave instances that do not meet the requirements according to the preset election rule, and then sort the slave instances that meet the requirements, thereby generating a new master instance.
[0060] For example, when Redis is re-elected, in the screening process, all offline or disconnected slave instances can be screened out first to ensure that the remaining slave instances are all online normally, then all slave instances that do not respond to the INFO command of the sentinel within a specified time can be screened out to ensure that the remaining slave instances are all recently successfully communicated, and finally all slave instances that are disconnected from the offline master instance for more than 10 milliseconds of down-after-milliseconds can be screened out to ensure that the data saved by the remaining slave instances is relatively new. Then, the screened slave instances can be sorted, the slave instance with the highest priority is selected, if there are multiple slave instances with the highest priority, the slave instance with the largest replication offset is selected, when there are multiple slave instances with the highest priority and the largest replication offset, the running ID is compared to select the slave instance with the smallest running ID as the new master instance. Other ways to select the new master instance in the application can be used as the preferred scheme of the application, which is not limited here.
[0061] S140: The monitoring program obtains the first parameter corresponding to the first instance and the second parameter corresponding to the second instance, and transmits the first parameter and the second parameter to the hook program.
[0062] In this embodiment, after the roles of the first instance and the second instance are switched through step S130, the monitoring program can be used to obtain the first parameter corresponding to the first instance and the second parameter corresponding to the second instance, and the first parameter and the second parameter are transmitted to the hook program to trigger the master-slave Pod label switching of the hook program.
[0063] It should be noted that the hook program of the application can be triggered after the master-slave instance role switching event of the monitoring program, and is used to switch the Pod labels of the master-slave instance. The first parameter corresponding to the first instance and the second parameter corresponding to the second instance are the latest parameters modified by the first instance and the second instance after the role switching, which are used as the basis for verifying the role of the instance.
[0064] S150: Based on the first parameter and the second parameter, the hook program switches the Pod labels of the first instance and the second instance.
[0065] In this embodiment, after the hook program receives the first parameter corresponding to the first instance and the second parameter corresponding to the second instance through step S140, the first parameter and the second parameter are analyzed, and the roles corresponding to the first instance and the second instance are determined. After it is judged that the roles are inconsistent with the Pod labels carried by the instances, the Pod labels of the first instance and the second instance can be switched.
[0066] It can be understood that when the application in the cluster needs to access Redis, the server is associated with the Pod with the master instance Pod label, and then the Redis instance in the Pod is accessed, so that the role of the Redis instance in the cluster and the Pod label are consistent, and a stable access entry can be provided to the outside. Among them, the Pod is the smallest atomic scheduling unit of the master instance, the slave instance and the sentinel instance of Redis in the cluster.
[0067] In the above embodiment, before switching the Redis master-slave instance, the monitoring program and the hook program can be obtained first, so as to switch the role and the Pod label of the master-slave instance when the master instance abnormally occurs, and the monitoring program is used to listen to the heartbeat change of each instance in Redis in real time. When it is found that there is a master instance heartbeat exception in all instances, a slave instance can be elected from all slave instances associated with the master instance according to the preset election rule, and the monitoring program is triggered to switch the role of the master-slave instance. The role of the elected slave instance is switched to the new master instance of Redis, and the role of the abnormal master instance is switched to the slave instance of the new master instance, so as to prevent the master instance from being abnormal for a long time, causing the Pod to restart and changing the access address of the Redis master-slave instance. Then, the corresponding parameters of the new master-slave instance after switching the role can be obtained and transmitted to the hook program, so as to trigger the hook program to switch the Pod label. The Pod label of the new master instance and the new slave instance is switched by the hook program, and the whole switching process of the Redis master-slave instance is completed, so that a stable access entry is maintained to the outside.
[0068] In one embodiment, the step of obtaining the monitoring program and the hook program in step S110 can include:
[0069] S111: Determine a custom resource definition CRD, and the CRD includes a configuration for creating the monitoring program and the hook program.
[0070] S112: Generate the monitoring program and the hook program based on the CRD.
[0071] In this embodiment, in order to listen to the running state of Redis in the cluster and transfer in time when Redis fails, the CRD (CustomResource Definition, custom resource definition) for customizing the configuration for creating the monitoring program and the hook program can be created in the cluster. Then, the monitoring program and the hook program can be generated according to the CRD, so as to switch the abnormal master instance in time when Redis fails.
[0072] It can be understood that the custom resource is a common extension in the Kubernetes cluster, that is, the self-defined resource can be added to the Kubernetes cluster to extend the function of the Kubernetes cluster, and the resource can be used like the built-in resource of the Kubernetes. In the application, the custom resource can be realized by the CRD to create a monitoring program and a hook program, so as to realize the switching of the Redis master-slave instance.
[0073] In one embodiment, the step of monitoring the change of the heartbeat of each instance in Redis by the monitoring program in step S110 can include:
[0074] S113: sending a heartbeat detection command to each instance of Redis by the monitoring program according to a preset frequency.
[0075] S114: obtaining the heartbeat detection result of each instance based on the heartbeat detection command.
[0076] S115: monitoring the change of the heartbeat of each instance according to the heartbeat detection result of each instance.
[0077] In the embodiment, in the process of monitoring the change of the heartbeat of each instance in Redis, the heartbeat detection command can be sent to each instance of Redis by the monitoring program according to a preset frequency, each instance returns the heartbeat detection result to the monitoring program after receiving the heartbeat detection command, and the monitoring program judges the heartbeat detection of each instance, so as to obtain the change of the heartbeat of each instance, and further determine whether an instance is offline.
[0078] For example, in a Kubernetes cluster in which a set of one master, two slaves and three sentinels of Redis instances are deployed, each sentinel can send an INFO command to all instances in Redis at a frequency of once every 10 seconds, which can be determined by repl-ping-slave-period, and the default is generally 10 seconds. Further, if the master instance does not respond to two or more sentinels and is marked as objectively offline, the sentinel changes the frequency of sending the INFO command to all slave instances to once every 1 second, determines the final state of the master instance according to the return result of all instances, and uses it as the basis for the election when the Redis instance is offline after the master instance.
[0079] In one embodiment, the step of monitoring the change of the heartbeat of each instance in Redis by the monitoring program in step S110 can include:
[0080] S121: if it is monitored that the heartbeat interval time of an instance exceeds the preset heartbeat timeout time, it is determined that the heartbeat of the instance is abnormal.
[0081] In this embodiment, during the monitoring of the heartbeat changes of each instance in Redis, the heartbeat detection results of each instance are analyzed based on the heartbeat detection results received by the monitoring program. If the heartbeat interval of an instance exceeds the preset heartbeat timeout time, it can be determined that the heartbeat of that instance is abnormal.
[0082] For example, in a Kubernetes cluster, the heartbeat timeout can be specified in the cluster-node-timeout configuration file. Here, the default setting is 15 seconds. By analyzing the heartbeat detection results returned by all instances in the Kubernetes cluster, if there is a Redis instance that has not returned a heartbeat detection result for more than 15 seconds, it can be determined that the heartbeat of that instance is abnormal.
[0083] In one embodiment, the step of switching the role of the second instance to the master instance of Redis and the step of switching the role of the first instance to the slave instance of the second instance in step S130 may include:
[0084] S131: Obtain the first instance ID and first configuration information corresponding to the first instance, and the second instance ID and second configuration information corresponding to the second instance.
[0085] S132: Replace the first instance ID and the second instance ID, and modify the first configuration information corresponding to the first instance and the second configuration information corresponding to the second instance according to the first instance ID and the second instance ID, so as to realize the role switching between the first instance and the second instance.
[0086] In this embodiment, when switching the master-slave role between the first instance and the second instance, the first instance ID and first configuration information corresponding to the first instance, as well as the second instance ID and second configuration information corresponding to the second instance, can be obtained first. Then, the obtained first instance ID and second instance ID can be replaced. After the replacement is completed, the first instance and the second instance can modify the configuration information according to the new instance ID, thereby realizing the role switching between the first instance and the second instance and ensuring that the access address of the master instance will not change due to long-term disconnection.
[0087] It is understandable that the instance ID here provides a unique ID for each instance in the cluster, which is used to uniquely identify and track the instance. This application utilizes the replacement of the master and slave instance IDs, and then modifies the configuration information of the master and slave instances according to the instance IDs to complete the master and slave instance role switching event. The configuration information of the master and slave instances may include IP address, port number, protection mode, background daemon process, etc.
[0088] In one embodiment, the method can further include:
[0089] S160: Real-time detection is performed on the Pod labels of all master instances in Redis. When it is detected that the role of any master instance does not conform to the Pod label, a slave instance corresponding to the Pod label is determined, and the Pod labels of the master instance and the slave instance are switched.
[0090] In this embodiment, the Operator on the Redis cluster can listen to the Pod label of the master instance of Redis through a coroutine. When it is found that the actual role of the master instance of Redis does not conform to the Pod label, the slave instance corresponding to the Pod label is determined, and the Pod labels of the master instance and the slave instance are switched in time to ensure that the application in the cluster can determine the master instance through the Pod label when accessing Redis, so as to obtain relevant data.
[0091] Specifically, the present application uses both Operator and monitoring program for automatic inspection, which double guarantees the master-slave switching of Redis and the consistency of the Pod label and the role of the Redis instance. When the sentinel abnormally fails to trigger the hook program to switch the master-slave Pod label, the automatic inspection of the Operator can timely discover and switch.
[0092] Further, if the monitoring program abnormally fails to work, the Redis failover and master-slave switching can be performed by the Operator, which simultaneously switches the role and the Pod label of the master-slave instance. Through the double guarantee of the Operator and the monitoring program, the Redis failure can be automatically transferred without human intervention to repair the failed Redis, which reduces the cost and time of the operation and maintenance personnel.
[0093] In one embodiment, the method can further include:
[0094] S170: If the first instance is not the master instance of the Redis, the connection between the first instance and the master instance associated with the first instance is disconnected.
[0095] In this embodiment, when the monitoring program listens to all instances of Redis and detects that the first instance with abnormal heartbeat is not the master instance of the Redis, the connection between the first instance and the master instance associated with the first instance is disconnected. After the first instance recovers to normal, the first instance is connected to the master instance again.
[0096] For example, in the Kubernetes cluster, when the sentinel detects that the slave instance does not respond within a specified time, the sentinel marks it as "subjectively online", because the offline of the slave instance generally has little effect, and the external service of the Kubernetes cluster will not be interrupted. After the slave instance goes offline, the connection with the master instance is disconnected for restart. If the slave instance resumes normal after restart, it will join the cluster according to the Kubernetes cluster information. First, it checks whether the master instance is online. If it is online, it automatically reconnects. If the master instance is not online, it becomes a slave instance of the newly elected master instance.
[0097] In order to better explain the Redis master-slave instance switching method of the present application, the following will be further described by taking Figure 2 as an example. As shown in Figure 2 , Figure 2 a Redis failover process schematic diagram of a Redis master-slave instance switching method provided by an embodiment of the present application.
[0098] Figure 2 In the present application, before switching the Redis master-slave instance, the Operator can add a monitoring program and a hook program through a custom resource definition CRD, and listen to the heartbeat of the master instance and the slave instance in the Redis by using the monitoring program. When the monitoring program detects that the heartbeat of the master instance is abnormal, causing the Redis to fail, the monitoring program can re-select a master instance from the running normal slave instances, and perform role switching on the elected slave instance. After the monitoring program triggers the master selection event, the hook program can switch the labels of the master-slave instance after role switching, so as to realize Redis failover. While the monitoring program listens to the heartbeat of the master instance and the slave instance in the Redis, the Operator can continuously detect the roles and labels of the master instance and the slave instance in the Redis, so as to prevent the inconsistency of the roles and labels of the redis instance Pod from causing the applications in the cluster to be unable to access the master instance Pod label through association, thereby providing stable services to the outside.
[0099] The Redis master-slave instance switching device provided by the present application will be described below. The Redis master-slave instance switching device described below can be correspondingly referred to the Redis master-slave instance switching method described above.
[0100] In one embodiment, as shown in Figure 3 , Figure 3A structural schematic diagram of a Redis master-slave instance switching device provided in an embodiment of the present application; the present application also provides a Redis master-slave instance switching device, which comprises a program acquisition module 210, an instance judgment module 220, an instance role switching module 230, a parameter determination module 240, an instance label switching module 250, and specifically comprises the following:
[0101] The program acquisition module 210 is used for acquiring a monitoring program and a hook program, and listening to the heartbeat change of each instance in Redis by using the monitoring program.
[0102] The instance judgment module 220 is used for taking an instance with a heartbeat exception as a first instance when it is found that there is any instance with a heartbeat exception in all instances, and judging whether the first instance is a master instance of the Redis.
[0103] The instance role switching module 230 is used for electing a slave instance as a second instance from all slave instances associated with the first instance according to a preset election rule, and triggering the monitoring program to switch the role of the second instance to the master instance of the Redis and switch the role of the first instance to the slave instance of the second instance.
[0104] The parameter determination module 240 is used for acquiring a first parameter corresponding to the first instance and a second parameter corresponding to the second instance by using the monitoring program, and transmitting the first parameter and the second parameter to the hook program.
[0105] The instance label switching module 250 is used for switching the Pod label of the first instance and the second instance by using the hook program based on the first parameter and the second parameter.
[0106] In the above embodiment, before the Redis master-slave instance is switched, the monitoring program and the hook program can be acquired first, so that the role and the Pod label of the master-slave instance are switched respectively when the master instance is abnormal, and the monitoring program is used to listen to the heartbeat change of each instance in Redis in real time. When it is found that there is a master instance with a heartbeat exception in all instances, a slave instance can be elected from all slave instances associated with the master instance according to a preset election rule, and the monitoring program is triggered to switch the role of the master-slave instance. The role of the elected slave instance is switched to the new master instance of the Redis, and the role of the abnormal master instance is switched to the slave instance of the new master instance, so as to prevent the Pod from being restarted and the access address of the Redis master-slave instance from being changed due to the long-time abnormality of the master instance. Then, the corresponding parameters of the new master-slave instance after the role is switched can be acquired and transmitted to the hook program, so as to trigger the hook program to switch the Pod label. The Pod label of the new master instance and the new slave instance is switched by using the hook program, and the whole switching process of the Redis master-slave instance is completed, so that a stable access entrance is maintained to the outside.
[0107] In an embodiment, the program obtaining module 210 can include:
[0108] a CRD defining submodule for determining a custom resource definition (CRD), the CRD including configurations for creating the monitoring program and the hook program;
[0109] a program generating submodule for generating the monitoring program and the hook program based on the CRD.
[0110] In an embodiment, the monitoring program obtaining module 210 can further include:
[0111] a command sending submodule for sending a heartbeat detection command to each instance of Redis using the monitoring program at a preset frequency.
[0112] a result obtaining submodule for obtaining a heartbeat detection result of each instance based on the heartbeat detection command.
[0113] a heartbeat listening submodule for listening to a heartbeat change of each instance according to the heartbeat detection result of each instance.
[0114] In an embodiment, the instance determining module 220 can include:
[0115] an abnormality determining submodule for determining that a heartbeat of an instance is abnormal if it is found that a heartbeat interval time of the instance exceeds a preset heartbeat timeout time.
[0116] In an embodiment, the instance role switching module 230 can include:
[0117] an information obtaining submodule for obtaining a first instance ID and first configuration information corresponding to the first instance and a second instance ID and second configuration information corresponding to the second instance.
[0118] an instance role switching submodule for replacing the first instance ID and the second instance ID, and modifying the first configuration information corresponding to the first instance and the second configuration information corresponding to the second instance according to the first instance ID and the second instance ID, so as to switch the roles of the first instance and the second instance.
[0119] In an embodiment, the apparatus can further include:
[0120] a label inspection module for performing real-time detection on Pod labels of all master instances in Redis, and determining a slave instance corresponding to a Pod label when it is found that a role of any master instance does not conform to the Pod label, and switching the Pod labels of the master instance and the slave instance.
[0121] In one embodiment, the apparatus can further comprise:
[0122] a master-slave instance connection module, configured to disconnect the first instance from a master instance associated with the first instance if the first instance is not the master instance of the Redis.
[0123] In one embodiment, the present application further provides a storage medium having computer readable instructions stored therein, which, when executed by one or more processors, cause the one or more processors to perform the steps of the Redis master-slave instance switching method according to any one of the above embodiments.
[0124] In one embodiment, the present application further provides a computer device having computer readable instructions stored therein, which, when executed by one or more processors, cause the one or more processors to perform the steps of the Redis master-slave instance switching method according to any one of the above embodiments.
[0125] As shown in Figure 4 , Figure 4 Fig. 3 is a schematic diagram of an internal structure of a computer device according to an embodiment of the present application. The computer device 300 can be provided as a server. As shown in Figure 4 , the computer device 300 includes a processing assembly 302, which further includes one or more processors, and a memory resource represented by a memory 301 for storing instructions, such as application programs, executable by the processing assembly 302. The application programs stored in the memory 301 can include one or more than one module each corresponding to a set of instructions. In addition, the processing assembly 302 is configured to execute the instructions to perform the Redis master-slave instance switching method according to any one of the above embodiments.
[0126] The computer device 300 can further include a power supply assembly 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input output (I / O) interface 305. The computer device 300 can operate based on an operating system stored in the memory 301, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM, or the like.
[0127] Those skilled in the art can understand that Figure 4The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0128] Finally, it should be noted that in this document, the terms such as first and second are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between such entities or operations. Also, the terms "comprises", "comprising", or any other variations thereof are intended to cover non-exclusive inclusions, so that a process, method, article, or apparatus that comprises a list of elements does not only include those elements, but also includes other elements not expressly listed, or other elements inherent in such process, method, article, or apparatus. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.
[0129] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The various embodiments can be combined as needed, and the same and similar parts refer to each other.
[0130] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A Redis master-slave instance switching method, characterized in that, The method comprises: obtaining a monitoring program and a hook program, and using the monitoring program to listen to the heartbeat change of each instance in Redis in real time; when it is found that any instance in all instances has heartbeat exception, taking the instance with heartbeat exception as a first instance, and judging whether the first instance is a master instance of the Redis; if the first instance is the master instance of the Redis, according to a preset election rule, selecting a slave instance from all slave instances associated with the first instance as a second instance, triggering the monitoring program to perform role switching, switching the role of the second instance to the master instance of the Redis, and switching the role of the first instance to the slave instance of the second instance; during the election process, according to the preset election rule, filtering out slave instances that do not meet the requirements, and sorting the slave instances that meet the requirements to obtain the second instance as a new master instance, wherein the slave instances that do not meet the requirements include slave instances in offline or disconnected state, slave instances that do not respond to the INFO command of the sentinel within a specified time, and slave instances that are disconnected from the offline master instance for more than 10 milliseconds of down-after-milliseconds; obtaining first parameters corresponding to the first instance and second parameters corresponding to the second instance by using the monitoring program, and passing the first parameters and the second parameters to the hook program; based on the first parameters and the second parameters, using the hook program to switch the Pod labels of the first instance and the second instance; the hook program is triggered after the master-slave instance role switching event of the monitoring program; while the monitoring program is listening to the heartbeat change of each instance in Redis in real time, it also includes real-time detection of the Pod labels of all master instances in Redis, and when it is found that any master instance has a role that does not meet the Pod label, determining a slave instance corresponding to the Pod label, and switching the Pod labels of the master instance and the slave instance.
2. The Redis master-slave instance switching method according to claim 1, characterized in that, The method comprises: determining a custom resource definition CRD, the CRD including configuration for creating a monitoring program and a hook program; based on the CRD, generating a monitoring program and a hook program.
3. The Redis master-slave instance switching method according to claim 1, characterized in that, The method comprises: sending a heartbeat detection command to each instance in Redis at a preset frequency by using the monitoring program; based on the heartbeat detection command, obtaining the heartbeat detection result of each instance; according to the heartbeat detection result of each instance, listening to the heartbeat change of each instance.
4. The Redis master-slave instance switching method according to claim 3, characterized in that, If it is found that the heartbeat interval time of any instance in all instances exceeds a preset heartbeat timeout time, it is judged that the instance has heartbeat exception. The method comprises:
5. The Redis master-slave instance switching method according to claim 1, characterized in that, obtain a first instance ID and first configuration information corresponding to the first instance and a second instance ID and second configuration information corresponding to the second instance; replace the first instance ID and the second instance ID, and modify the first configuration information corresponding to the first instance and the second configuration information corresponding to the second instance according to the first instance ID and the second instance ID, to realize role switching of the first instance and the second instance.
6. The Redis master-slave instance switching method according to claim 1, characterized in that, The method further comprises: if the first instance is not the master instance of the Redis, disconnect the first instance and the master instance associated with the first instance.
7. A Redis master-slave instance switching device, characterized in that, Comprise: a monitoring program acquisition module, configured to obtain a monitoring program and a hook program, and listen to heartbeat changes of each instance in the Redis by using the monitoring program; an instance judgment module, configured to, when it is found that there is any instance with heartbeat exception among all instances, take the instance with heartbeat exception as a first instance, and judge whether the first instance is the master instance of the Redis; an instance role switching module, configured to, according to a preset election rule, select one slave instance from all slave instances associated with the first instance as a second instance, trigger the monitoring program to perform role switching, switch the role of the second instance to the master instance of the Redis, and switch the role of the first instance to the slave instance of the second instance; in the election process, filter out slave instances that do not meet the requirements according to the preset election rule, and sort the slave instances that meet the requirements to obtain the second instance as a new master instance, wherein the slave instances that do not meet the requirements include slave instances in an offline or disconnected state, slave instances that do not respond to an INFO command of a sentinel within a specified time, and slave instances whose connection with the offline master instance is disconnected for more than down-after-milliseconds*10 milliseconds; a parameter determination module, configured to obtain a first parameter corresponding to the first instance and a second parameter corresponding to the second instance by using the monitoring program, and pass the first parameter and the second parameter to the hook program; an instance label switching module, configured to switch Pod labels of the first instance and the second instance by using the hook program based on the first parameter and the second parameter; the hook program is triggered after the monitoring program performs master-slave instance role switching; while the monitoring program is listening to heartbeat changes of each instance in the Redis in real time, the monitoring program further comprises real-time detection of Pod labels of all master instances in the Redis, and when it is found that there is any master instance with a role that does not meet the Pod label, a slave instance corresponding to the Pod label is determined, and the Pod labels of the master instance and the slave instance are switched.
8. A storage medium characterized by: The storage medium stores computer readable instructions, and the computer readable instructions are executed by one or more processors to perform the steps of the Redis master-slave instance switching method according to any one of claims 1 to 6.
9. A computer device, comprising: Comprise: one or more processors, and a memory; The memory stores computer readable instructions which, when executed by the one or more processors, perform the steps of the Redis master-slave instance switching method of any one of claims 1 to 6.
Citation Information
Patent Citations
Prometheus management and control method, device and equipment, and storage medium
CN112994935A
System and method for monitoring database in container
CN113467897A