Disaster recovery switching method for proxy simulation app voting

Through the automatic disaster recovery switching method based on proxy simulation app voting, the business impact problem caused by manual switching in system disaster recovery is solved, and fast and reliable disaster recovery switching and fault analysis is achieved, reducing the user's downtime.

CN118945040APending Publication Date: 2024-11-12HANGZHOU YIKE INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410980283.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-02-03
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In the system disaster recovery system, manual switching of the production environment to the disaster recovery environment will lead to short-term business impacts, and the existing technology is difficult to quickly and reliably realize automatic disaster recovery switching, affecting database stability and user service downtime.

Method used

The automatic disaster recovery switching method based on proxy simulation app voting is adopted. By deploying multiple proxy nodes and simulation apps, voting debugging and error analysis are performed, traffic is automatically switched to the disaster recovery cluster to ensure the switching speed and reliability.

Benefits of technology

It minimizes the user's downtime, improves system reliability and automation of disaster recovery switching, quickly analyzes the causes of cluster failures, and reduces the need for manual intervention.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118945040A_ABST
    Figure CN118945040A_ABST
Patent Text Reader

Abstract

The invention discloses a disaster recovery backup switching method for proxy simulation app voting. The disaster recovery backup switching method for proxy simulation app voting specifically comprises the following steps: 101) a proxy node deployment step, 102) a preprocessing step, 103) a voting step and 104) a switching step, and the disaster recovery backup switching method for proxy simulation app voting can realize automatic disaster recovery backup switching and upload error reasons to facilitate rapid analysis of cluster fault reasons.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This proposal is a division of the application number 2023100539497 entitled "Automatic disaster recovery switching method based on proxy simulation app voting" Technical Field

[0002] The present invention relates to the technical field of server side, and more specifically, to a disaster recovery switching method for proxy simulation app voting. Background Art

[0003] When designing a system disaster recovery system, you need to consider the RTO indicator (Recovery Time Objective) to minimize the downtime of the production environment. In actual applications, database stability is the main consideration for the continuous provision of services in the production environment. When the production database is damaged, you need to manually switch the application route of the production environment to the disaster recovery environment. In the disaster recovery system, this manual switch will cause a certain business impact in a short period of time. Summary of the invention

[0004] The present invention overcomes the deficiencies of the prior art and provides a disaster recovery switching method for proxy simulation app voting.

[0005] The technical solution of the present invention is as follows: An automatic disaster recovery switching method based on proxy simulation app voting specifically includes the following steps: 101) Steps for deploying proxy nodes: deploy multiple proxy nodes across the country, and deploy multiple simulation apps on each node for voting debugging; 102) Preprocessing step: The proxy program of the simulated app is started, and the voting information of each cluster is obtained from each node; 103) Voting steps: When simulating app voting, the check_health interface is called to simulate the business on the actual server to perform read and write operations; when an error occurs, in addition to voting, the simulation app will also upload the error results for cause analysis; Each node first collects and obtains the configuration of all clusters, then loops by cluster, checks each cluster in turn, and obtains an address for access after taking the remainder of the current number of addresses for each cluster. The inspection result is reported to the collection node. If the cluster inspection is good, wait appropriately according to the overall agreed time. 104) Switching steps: Based on the voting results, if there is a bad result in the voting of a cluster and it exceeds the threshold, the disaster recovery switching procedure automatically switches the traffic.

[0006] Furthermore, there are at least 8 server locations across the country, mainly in public cloud services in Guangzhou, Shanghai and Beijing.

[0007] Furthermore, after the cluster configuration information is read in step 102), the check starts, and all clusters are checked once every certain period of time, and each cluster calls the check_health interface in turn according to the set order.

[0008] Furthermore, each certain time interval is within 15 seconds.

[0009] Furthermore, the voting results of the simulated app are stored in the database of the collection node. The switching program periodically checks the voting results. If more than 70% of the agents in a cluster report errors, it is considered that there is a problem with this cluster and the backend service needs to be switched to the disaster recovery cluster on the other end.

[0010] Furthermore, the switching procedure includes an error switching function. When there are fewer than 3 reporting agents, only an alarm is issued without switching. The expected number of agents is n, and n is greater than 5. When the actual number of reports is insufficient, it is determined that the network is not good and a manual disaster recovery switching suggestion is issued.

[0011] Furthermore, the periodic check is performed once every 5 seconds.

[0012] Furthermore, it also includes 105) recovery step: after the switching action of the disaster recovery switching program occurs, a switching flag is set and relevant personnel are notified, and manual switching back is performed during off-peak hours.

[0013] Compared with the prior art, the present invention has the following advantages: the present invention simulates the actual network situation of users, and multiple agents ensure the reliability of voting. The switching speed is much faster than manual, and the user downtime is reduced as much as possible. The collection program and the switching program are independent and will not interfere with each other, thereby improving the reliability of the system. The reason for the proxy node uploading error helps to quickly analyze the cause of the cluster failure. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the present invention. DETAILED DESCRIPTION

[0015] The embodiments of the present invention are described in detail below, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and cannot be used as a limitation of the present invention.

[0016] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as generally understood by those skilled in the art in the field to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0017] The numbers of the steps mentioned in each embodiment are only for the convenience of description and have no substantial sequence relationship. Different steps in each specific embodiment can be combined in different sequences to achieve the purpose of the invention.

[0018] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0019] like Figure 1 As shown, an automatic disaster recovery switching method based on proxy simulation app voting specifically includes the following steps: 101) Steps for deploying proxy nodes: deploy multiple proxy nodes across the country, and deploy multiple simulation apps on each node for voting debugging; among them, there are at least 8 server points across the country, mainly distributed on public cloud services in Guangzhou, Shanghai and Beijing.

[0020] 102) Preprocessing step: The proxy program of the simulated app is started, and the voting information of each cluster is obtained from each node; after reading the cluster configuration information, the check is started, and all clusters are checked once every certain interval, and each cluster calls the check_health interface in the set order. Each certain interval is generally within 15 seconds.

[0021] 103) Voting steps: When simulating app voting, the check_health interface is called to simulate the business on the actual server to perform read and write operations; when an error occurs, in addition to voting, the simulation app will also upload the error results for cause analysis; The voting results of the simulated app are stored in the database of the collection node. The switching program regularly checks the voting results. If more than 70% of the agents in a cluster report errors, it is considered that there is a problem with this cluster and the backend service needs to be switched to the disaster recovery cluster on the other end. The switching program includes an error switching function. When there are fewer than 3 reporting agents, only an alarm is issued without switching. The expected number of agents is n, and n is greater than 5. When the actual number of reports is insufficient, it is determined that the network is not good and a manual disaster recovery switching suggestion is issued. Regular checks are performed every 5 seconds, or less, to speed up the time to query problems and reduce business impact.

[0022] Each node first collects and obtains the configuration of all clusters, then loops by cluster, checks each cluster in turn, and obtains an address for access after taking the remainder of the current number of addresses for each cluster. The inspection result is reported to the collection node. If the cluster inspection is good, wait appropriately according to the overall agreed time. 104) Switching steps: According to the voting results, if there is a bad result in the voting of a cluster and it exceeds the threshold, the disaster recovery switching program automatically switches the traffic. The collection program and the switching program are independent of each other and will not interfere with each other, thereby improving system reliability.

[0023] It also includes 105) recovery steps: after the disaster recovery switching program switching action occurs, set the switching flag and notify relevant personnel. Generally, it is manually switched back during off-peak hours. Naturally, it can also be manually switched during peak hours according to actual needs. It depends on the actual situation.

[0024] The specific method called by the proxy node is ClusterAgent, which implements the judgment and detection of the cluster. First, obtain the configuration of all clusters from the collection node, loop by cluster, and check each cluster in turn. For each cluster, multiple addresses are obtained by taking the remainder of the current number of times to access an address, and the inspection result is reported to the collection node. If there is an error, the error log will also be reported. If all clusters are good, wait appropriately according to the overall agreed time.

[0025] class ClusterAgent(ClusterHealthBase): def chk_all(self): self.set_dict_c_url() i=0 # Total number of runs max1 = 15 # Overall running time, seconds sleep1 = 0.01 # sleep for a number of seconds after a cluster check is completed exit = False while (exit != True): count_ko = 0 # Number of unhealthy clusters start = time.time() for (c_code, urls) in self.dict_c_url.items(): try: chk_ok = self.chk_cluster(c_code, urls, i) self.report(c_code, chk_ok) if chk_ok == False: count_ko += 1 logger.info(c_code+'='+str(chk_ok)+',count_ko='+str(count_ko)) time.sleep(sleep1) except SystemExit: logger.info('exit') exit = True end = time.time() i += 1 # All cluster checks are good, wait for the agreed minimum time logger.info('wait,max1='+str(max1)+' seconds') while end-start <max1: end = time.time() time.sleep(1) Among them, the collection node obtains the configuration of all clusters, and obtains json similar to {"Cluster A": ["https: / / domain name 1: port 1 / slh / checkCluster.do?kind=checkHealth&serverApplicationName=stsServer", "http: / / ip2: port 2 / slh / checkCluster.do?kind=checkHealth&serverApplicationName=stsServer"],"Cluster B": ["https: / / domain name 1: port 1 / slh / check.do?svc=checkHealth&db=true&redis=true&thread=true", "https: / / ip2: port 2 / slh / check.do?svc=checkHealth&db=true&redis=true&thread=true", "https: / / ip3: port 3 / slh / check.do?svc=checkHealth&db=true&redis=true&thread=true"],...}. Each cluster has multiple access addresses to prevent false alarms of single line failures.

[0026] serverApplicationName=stsServer means that dubbo calls need to be checked; db=true means that database read and write need to be checked; redis=true means that cache read and write need to be checked; thread=true means that threads need to be checked for exceeding the limit.

[0027] The switching program first obtains a list of all cluster codes that need to be checked from the collection node database, then checks each cluster in turn, and then checks whether this cluster needs to start disaster recovery. If more than 70% of the agents in a cluster report errors, it is considered that this cluster has problems and needs to switch the backend service to the disaster recovery cluster on the other end. In order to prevent accidental switching, the switching program will do some checks. For example, if there are fewer than 3 reporting agents, only an alarm will be issued without switching. For example, if the expected number of agents is 8, but the actual number of reports is insufficient, it means that the network is not good and cannot be reported. Manual switching is recommended. Then, if a cluster starts disaster recovery, all clusters will start a new round of checks, and the already started clusters will be automatically skipped. The results of the disaster recovery will be notified to the corresponding group for subsequent manual processing. The specific method used is ClusterGuard: class ClusterGuard(ClusterHealthBase): def chk_all(self, mod_result): # self.set_dict_c_url() mod = 3 max1 = 10 dao = DeployDAO() exit = False while (exit != True): start119 = False list_agent_err = [] start = time.time() codes = dao.get_cluster119_need_chk_codes(mod, mod_result) for c_code in codes: try: start119 = self.chk_cluster(c_code, dao, list_agent_err) time.sleep(0.1) if start119: # If a cluster starts disaster recovery, exit immediately break except SystemExit: print('exit') exit = True self.ding_all(list_agent_err) time.sleep(3) In summary, this solution simulates the actual network conditions of users, and multiple proxies ensure the reliability of voting. The switching speed is much faster than manual, which minimizes the user downtime. The collection program and the switching program are independent and will not interfere with each other, which improves the reliability of the system. The reason for the proxy node uploading error helps to quickly analyze the cause of the cluster failure.

[0028] The above is only a preferred embodiment of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and modifications without departing from the concept of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A disaster recovery switching method for proxy simulation app voting, characterized by: The specific steps include: 101) Steps for deploying proxy nodes: deploy multiple proxy nodes across the country, and deploy multiple simulation apps on each node for voting debugging; 102) Preprocessing step: The proxy program of the simulated app is started, and the voting information of each cluster is obtained from each node; After reading the cluster configuration information, the check starts and all clusters are checked once every certain period of time. Each cluster calls the check_health interface in the set order. 103) Voting steps: When simulating app voting, the check_health interface is called to simulate the business on the actual server to perform read and write operations; when an error occurs, in addition to voting, the simulation app will also upload the error results for cause analysis; Each node first collects and obtains the configuration of all clusters, then loops by cluster, checks each cluster in turn, and obtains an address for access after taking the remainder of the current number of addresses for each cluster. The inspection result is reported to the collection node. If the cluster inspection is good, wait appropriately according to the overall agreed time. The method called by the proxy node is ClusterAgent, which implements cluster judgment and detection. First, it obtains the configuration of all clusters from the collection node, loops by cluster, and checks each cluster in turn. For each cluster, it obtains an address after taking the remainder of the current number of addresses for access, and reports the inspection result to the collection node. If there is an error, it will also report the error log. If all clusters are good, it will wait for the agreed time as a whole. 104) Switching steps: According to the voting results, if there is a bad result in the voting of a cluster and it exceeds the threshold, the disaster recovery switching program automatically switches the traffic; The voting results of the simulated app are stored in the database of the collection node. The switching program periodically checks the voting results. If more than 70% of the agents in a cluster report errors, it is considered that there is a problem with the cluster and the backend service needs to be switched to the disaster recovery cluster on the other end. The switching procedure includes an error switching function. When there are fewer than 3 alarm agents, only an alarm will be issued without switching. The expected number of agents is n, and n is greater than 5. When the actual number of reports is insufficient, it is determined that the network is not good and a manual disaster recovery switching suggestion is issued.

2. The disaster recovery switching method for proxy simulation app voting according to claim 1 is characterized by: There are at least 8 server locations across the country, mainly in public cloud services in Guangzhou, Shanghai and Beijing.

3. According to claim 1, a disaster recovery switching method for proxy simulation app voting is characterized by: Each interval is within 15 seconds.

4. According to claim 1, a disaster recovery switching method for proxy simulation app voting is characterized by: Check periodically every 5 seconds.