Disaster recovery method and device and disaster recovery platform

Abstract

The invention discloses a disaster recovery method and device and a disaster recovery platform, and relates to the technical field of computers. The control node automatically determines the state of the service system according to the index data of the service system, and can timely and accurately determine the state of the service system under the condition that the index data of the service system changes. And the control node automatically determines the state transition rule matched with the state, so that the state transition rule can be timely and accurately determined under the condition that the index data of the business system changes, the business system can timely migrate the state, and a guarantee is provided for the business system to normally provide external services.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a disaster recovery method, device and disaster recovery platform. Background Art

[0002] Disaster recovery refers to the establishment of multiple systems with the same functions in different locations. These systems support mutual health status monitoring and function switching. When a system stops working, another system can replace the stopped system so that the entire system can provide services to the outside world. Usually, when a system in the entire system fails, the operator manually determines the failure scenario and determines the disaster recovery switching plan based on the failure scenario to obtain a system to replace the failed system, so that the entire system can provide services to the outside world.

[0003] In the above process, the operator manually determines the fault scenario and disaster recovery switching plan. When the fault changes, the operator may not be able to determine the fault scenario and disaster recovery switching plan in a timely and accurate manner, resulting in the entire system being unable to provide normal services to the outside world. Summary of the invention

[0004] The present application provides a disaster recovery method, device and disaster recovery platform for solving the problem that the fault scenario and disaster recovery switching plan cannot be determined timely and accurately when the fault changes, resulting in the business system being unable to provide normal services to the outside world.

[0005] In a first aspect, the present application provides a disaster recovery method. The method is applied to a disaster recovery platform, which includes a control node and a business system. The method can be executed by a control node, which can be a single computing device, a computing device cluster including multiple computing devices, or a component of a computing device, such as a processor, a chip, a chip system, etc. of a computing device. The control node can also be a logic module or software that can realize all or part of the functions of the computing device. The disaster recovery method provided in the first aspect includes: the control node obtains indicator data of the business system. The control node determines that the business system is in a first state according to the indicator data. The first state is that there is at least one abnormal service node in the business system that cannot provide services normally. The control node determines a first state migration rule that matches the first state. Among them, the first state migration rule is used to indicate: migrate the business system from the first state to the second state, and the second state is to migrate the service of at least one abnormal service node to at least one normal service node that can provide services normally. And the control node sends a first request to the business system. The first request is used to instruct the business system to migrate from the first state to the second state.

[0006] In the first aspect, the control node automatically determines the state of the business system based on the indicator data of the business system, and can timely and accurately determine the state of the business system when the indicator data of the business system changes. And the control node automatically determines the state migration rule that matches the state, and can timely and accurately determine the state migration rule when the indicator data of the business system changes, so that the business system can migrate the state in time, which provides a guarantee for the business system to provide normal services to the outside world.

[0007] In a possible implementation, the control node determines a first state transition rule that matches the first state, including: the control node determines the first state transition rule that matches the first state according to a state transition diagram. The state transition diagram includes multiple states and state transition rules corresponding to each of the multiple states, and the multiple states include the first state. In this way, the state transition diagram includes each state and the state transition rule corresponding to each state, and the state transition rule corresponding to the state can be intuitively determined according to the state, thereby improving the reliability of the determined state transition diagram and improving the friendliness of the interaction.

[0008] In another possible implementation, the abnormal service node is a region or an available partition. In this way, different state migration rules can be determined according to different situations of the abnormal service node, thereby improving the accuracy of the determined state migration rules.

[0009] In another possible implementation, the first state migration rule is used to migrate the service of at least one abnormal service node to at least one normal service node, thereby ensuring that the business system can provide services normally.

[0010] In another possible implementation, after the control node sends the first request to the business system, the method also includes: the control node determines an abnormal service node that can provide services normally among at least one abnormal service node. The control node adds the abnormal service node that can provide services normally to at least one normal service node. The control node obtains a second state migration rule that matches the second state. The second state migration rule is used to indicate that the business system is migrated from the second state to the third state, and the third state is that the business system uses normal service nodes to provide services. And the control node sends a second request to the business system. The second request is used to indicate that the business system migrates from the second state to the third state. In this way, when at least one abnormal service node is able to provide normal services, using it to provide services can improve the utilization rate of service nodes, shorten the time required to provide services, and improve the efficiency of providing services.

[0011] In another possible implementation, before the control node determines the first state transition rule that matches the first state, the method also includes: the control node obtains multiple states of the business system. The multiple states include a first state, a second state, and a third state. The control node obtains multiple state transition rules. Among them, the multiple state transition rules include a first state transition rule and a second state transition rule, and each state transition rule in the multiple state transition rules is used to indicate: if the business system is in an initial state, an action matching the initial state is performed on the business system to put the business system in an end state, the initial state is one of the multiple states, and the end state is one of the multiple states, and the initial state and the end state are different. And the control node determines the matching relationship between the multiple states and the multiple state transition rules based on the multiple states and the multiple state transition rules. In this way, the control node establishes a matching relationship between the multiple states and the multiple state transition rules, which improves the efficiency of obtaining the matching relationship compared to manually establishing the matching relationship.

[0012] In another possible implementation, the method further includes: the control node establishes a state transition diagram based on the matching relationship between the multiple states and the multiple state transition rules. The state transition diagram includes: multiple initial states, actions corresponding to each of the multiple initial states, and end states corresponding to the actions. In this way, the state transition diagram can intuitively present the association relationship between the initial state, the action corresponding to the initial state, and the end state, thereby improving the reliability of the determined state transition rules and the friendliness of the interaction.

[0013] In another possible implementation, the control node obtains multiple states, including: the control node provides a state configuration interface. The control node obtains multiple state information in response to the operation of the state configuration interface. Among them, each state information in the multiple state information includes one or more of the following: state name, fault scenario, first state condition, the state name is used to indicate: the name of the configured state, the fault scenario is used to indicate: the fault scenario applicable to the configured state, and the first state condition is used to indicate: indicator data for determining that the business system is in the configured state. And the control node obtains multiple states based on the multiple state information. In this way, the state can be configured according to the needs of the actual application, the usage scenario is expanded, and the interactive friendliness is improved.

[0014] In another possible implementation, the fault scenario includes: disaster recovery mode, middleware type, and application technology stack type. The disaster recovery mode is used to indicate the disaster recovery and load balancing strategies, the middleware type is used to indicate the database system that processes data in the business system, and the application technology stack type is used to indicate the software development language of the business system. In this way, the fault scenario is subdivided according to the disaster recovery mode, middleware type, and application technology stack type, which provides a guarantee for the matching of the determined state migration rules with the faults.

[0015] In another possible implementation, the control node obtains multiple state migration rules, including: the control node provides a state migration rule configuration interface. The control node obtains multiple rule information in response to the operation of the state migration rule configuration interface. Among them, each rule information in the multiple rule information includes one or more of the following: a state migration rule name, a second state condition, the state migration rule name is used to indicate: the name of the configured state migration rule, and the second state condition is used to indicate: indicator data for determining that the business system is in an initial state, the action performed on the business system in the initial state, and the end state of the business system after the action is performed. And the control node obtains multiple state migration rules based on the multiple rule information. In this way, the state migration rules can be configured according to the needs of actual applications, which expands the usage scenarios and improves the interactive friendliness.

[0016] In another possible implementation, the disaster recovery method further includes: the control node provides alarm information. The alarm information is used to indicate that there is at least one abnormal service node in the business system. In this way, the abnormal service node can be discovered in time, which provides a guarantee for the business system to provide normal services to the outside.

[0017] In the second aspect, the present application provides a disaster recovery method. The method is applied to a disaster recovery platform, which includes a control node and a business system. The method is executed by a business system, and the method includes: the business system receives a first request. Among them, the first request is used to instruct the business system to migrate from a first state to a second state, the first state is that there is at least one abnormal service node in the business system that cannot provide services normally, and the second state is to migrate the services of at least one abnormal service node to at least one normal service node. And the business system responds to the first request and migrates from the first state to the second state.

[0018] In the second aspect, the business system automatically migrates its state based on a request sent by the control node to indicate the state of the business system to be migrated. In the case where there is at least one abnormal service node in the business system, the state can be migrated and the abnormal service node is not used to provide services, which provides a guarantee for the business system to provide normal services to the outside world.

[0019] In another possible implementation, the method further includes: the business system receives a second request. The second request is used to instruct the business system to migrate from the second state to the third state, and the third state is that the business system uses at least one abnormal service node and at least one normal service node to provide services. And the business system responds to the second request and migrates from the second state to the third state. In this way, when at least one abnormal service node can provide services normally, the at least one service node is added to the normal service node, and the normal service node is used to provide services. The number of service nodes used to provide services is increased, the time for providing services is shortened, and the efficiency of providing services is improved.

[0020] In a third aspect, the present application provides a disaster recovery platform. The disaster recovery platform includes a control node and a business system. The control node is used to execute the operation steps of the disaster recovery method in the first aspect or any possible implementation of the first aspect, and the business system is used to execute the operation steps of the disaster recovery method in the second aspect or any possible implementation of the second aspect.

[0021] In a fourth aspect, the present application provides a disaster recovery device, which includes modules for executing the disaster recovery method in the first aspect or any possible design of the first aspect.

[0022] In a fifth aspect, the present application provides a processor, wherein the processor is used to execute the operation steps of the disaster recovery method in the first aspect or any possible design of the first aspect.

[0023] In a sixth aspect, the present application provides a computing device cluster. The computing device cluster includes at least one computing device, each computing device including a processor and a memory. The processor of at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the operation steps of the disaster recovery method in the first aspect or any possible design of the first aspect, or performs the operation steps of the disaster recovery method in the second aspect or any possible implementation of the second aspect.

[0024] In a seventh aspect, the present application provides a computer-readable storage medium, including: computer software instructions; when the computer software instructions are executed in a computing device cluster, the computing device cluster executes the operation steps of the method described in the first aspect or any possible implementation of the first aspect, or the computing device cluster executes the operation steps of the method described in the second aspect or any possible implementation of the second aspect.

[0025] In an eighth aspect, the present application provides a computer program product. When the computer program product is run on a computing device cluster, the computing device cluster is caused to perform the operation steps of the method described in the first aspect or any possible implementation of the first aspect, or the computing device cluster is caused to perform the operation steps of the method described in the second aspect or any possible implementation of the second aspect.

[0026] The beneficial effects of the third to eighth aspects above can be referred to the description of any implementation in the first or second aspect, and will not be repeated here. Based on the implementation provided by the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A structural diagram of a disaster recovery platform provided for this application;

[0028] Figure 2 A schematic diagram of the structure of a computing device provided for this application;

[0029] Figure 3 A schematic diagram of the structure of a chip provided in this application;

[0030] Figure 4 A flowchart of a disaster recovery method provided for this application;

[0031] Figure 5 A flowchart of a disaster recovery method provided in this application;

[0032] Figure 6 A flowchart of a method for establishing a matching relationship provided in this application;

[0033] Figure 7 An example diagram of an acquisition state provided for this application;

[0034] Figure 8 An example diagram of an acquisition state transition rule provided by this application;

[0035] Fig. 9 An example diagram of the process of instantiating states and state transition rules provided for this application;

[0036] Fig.10 An example diagram of a state transition diagram provided for this application;

[0037] Fig.11 A schematic diagram of the structure of a disaster recovery device provided in this application;

[0038] Fig.12 A schematic diagram of the structure of a computing device cluster provided for this application;

[0039] Fig.13 A schematic diagram of the connection between computing devices provided for this application. DETAILED DESCRIPTION

[0040] The present application provides a disaster recovery method, in which: the control node automatically determines the state of the business system based on the indicator data of the business system, and can timely and accurately determine the state of the business system when the indicator data of the business system changes. And the control node automatically determines the state migration rule that matches the state, and can timely and accurately determine the state migration rule when the indicator data of the business system changes, so that the business system can migrate the state in time, which provides a guarantee for the business system to provide normal services to the outside.

[0041] In order to make the description of the following embodiments clear and concise, a brief introduction of the terms is first given.

[0042] (1) Disaster recovery system.

[0043] A disaster recovery system refers to the establishment of two or more IT systems with the same functions in different geographical locations, which can monitor each other's health status and switch functions. When a system stops working due to an accident (such as fire, earthquake, etc.), the entire application system can be switched to another location so that the system functions can continue to work normally.

[0044] (2) Region.

[0045] Region is a concept used to describe the location of data centers in cloud computing. Generally, regions can be divided according to geographical location and network latency. When regions are divided according to geographical location, different regions are located in different geographical locations.

[0046] (3) Availability zone.

[0047] An availability zone (AZ) is a collection of one or more physical data centers with independent wind, fire, water, and electricity facilities. A region may include multiple availability zones. An availability zone may deploy multiple clusters including at least one computing device. In some possible scenarios, an availability zone may also be called an availability partition, a partition, and so on.

[0048] The above briefly introduces the terms that may be involved in this application, and the following briefly introduces the related technologies that may be involved in this application.

[0049] Disaster recovery refers to the establishment of multiple systems with the same functions in different locations. These systems support mutual health status monitoring and function switching. When a system stops working, a system in another location can replace the stopped system so that the entire system can provide services to the outside world. Usually, when a system stops working, the operator will determine the fault scenario of the entire system based on the current status of the stopped system, and select the disaster recovery switching plan corresponding to the fault scenario, and manually execute the disaster recovery switching plan so that another system replaces the stopped system, providing guarantees for the entire system to provide services to the outside world, thereby enabling the entire system to meet the service quality objectives (Service Level Objective, SLO).

[0050] In the above process, the operator determines the fault scenario, selects the disaster recovery switching plan corresponding to the fault scenario, and manually executes the disaster recovery switching plan. In the case of a change in the system fault, the operator may not be able to promptly and accurately determine the fault scenario, the disaster recovery switching plan corresponding to the fault scenario, and execute the disaster recovery switching plan, which may cause the entire system to be unable to provide normal services to the outside world. In addition, due to the limitations of the operator's professional knowledge, the determined fault scenario and disaster recovery switching plan may not match the fault that has occurred, which may affect the ability of the entire system to provide services to the outside world.

[0051] Based on this, the present application provides a disaster recovery method, which can be applied to a disaster recovery platform. Figure 1 A structural diagram of a disaster recovery platform provided for this application, such as Figure 1 As shown, the disaster recovery platform 100 includes: a control node 110 and a business system 120. The control node 110 and the business system 120 can communicate with each other by wired mode or wireless mode. The wired communication mode can be: Ethernet, optical fiber, and various peripheral component interconnect express (PCIe) buses arranged inside the disaster recovery platform 100 for connecting the control node 110 and the business system 120. The wireless communication mode can be: Internet, wireless fidelity (WIFI) and ultra wide band (UWB) technology, etc.

[0052] The control node 110 obtains the indicator data of the business system, and sends a request to the business system based on the obtained indicator data, so that the business system changes its state.

[0053] The business system 120 receives the request of the control node 110, changes its state based on the received request, and provides services to the outside based on the changed state. The business system 120 includes a distribution node, a middleware, and multiple business nodes. The distribution node receives the business request and distributes the business request to the business node based on the state of the business system 120. The middleware processes the data of the business system 120, and the middleware may include a database (such as MySQL, mongodb, cassandra, etc.), a remote dictionary service (Remote Dictionary Server, redis), a data stream processing platform kafka, etc. The business node changes its working state according to the request of the control node. The business node receives and processes the business request of the distribution node to obtain the processing result, and feeds back the processing result.

[0054] The multiple business nodes included in the business system 120 can be deployed in multiple regions. A region can include one or more availability zones, and an availability zone of a region can include some or all business nodes of the region. An availability zone can include one or more clusters. A cluster of an availability zone can include some or all business nodes of the availability zone. According to the needs of actual applications, one or more middleware can be deployed in a region to process the data of the region.

[0055] According to the needs of actual applications, one or more microservices can be deployed on a business node in a region, or one microservice can be deployed on multiple nodes in a region. And the data of the microservices deployed on the business nodes in the region are processed by the middleware of a region. For example, the business system 120 includes region 1. Region 1 includes business node 1 and middleware 1. Microservices 1 to 3 are deployed on business node 1. In this case, middleware 1 can process the data of microservices 1 to 3.

[0056] In some possible scenarios, the multiple regions included in the business system 120 may have different data processing capabilities, such as some regions in the multiple regions may have the ability to write data and the ability to read data, and another part of the regions in the multiple regions may only have the ability to read data. In this case, the region with the ability to write data and the ability to read data may be referred to as the primary region, and the region with only the ability to read data may be referred to as the multi-active region. The business nodes in the primary region may be used to process business requests that require writing data and / or reading data, and the business nodes in the multi-active region may be used to process business requests that require reading data, but cannot be used to process business requests that require writing data. For example, the business system 120 includes regions 1 to 3. The business nodes in region 1 have the ability to read data and the ability to write data, and the business nodes in regions 2 and 3 have the ability to read data, but not the ability to write data. In this case, region 1 may be referred to as the primary region, and regions 2 and 3 may be referred to as multi-active regions.

[0057] The control node 110, the distribution node of the business system 120, and the business node of the business system 120 can all be implemented by using a computing device. The computing device can be a common computing device, such as a personal computer, a desktop computer, a mobile phone, and the like. Figure 2 A schematic diagram of a computing device provided in this application, such as Figure 2 As shown, the computing device 200 includes: a communication interface 214, a processor 211, and a memory 213. The communication interface 214 is used to communicate with a device located outside the computing device 200. For example, the computing device 200 obtains the indicator data of the business system through the communication interface 214, the computing device 200 obtains the processing result (such as the first request, etc.) based on the obtained indicator data, and feeds back the processing result through the communication interface 214. The communication interface 214 can be an input / output (I / O) interface.

[0058] The processor 211 is the computing core and control core of the computing device 200, which may include: a central processing unit (CPU), a specific integrated circuit, other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. In practical applications, the computing device 200 may also include multiple processors. The processor 211 may include one or more processor cores. An operating system and other software programs are installed in the processor 211, so that the processor 211 can access the memory 213 and various peripheral component interconnect express (PCIe) devices.

[0059] The processor 211 is connected to the memory 213 via a bus 216. The bus 216 may be a double data rate (DDR) bus or other types of buses. The memory 213 is the main memory of the computing device 200. The memory 213 is usually used to store various running software in the operating system. In order to improve the access speed of the processor 211, the memory 213 needs to have the advantage of fast access speed. In traditional computer devices, dynamic random access memory (DRAM) is usually used as the memory 213. In addition to DRAM, the memory 213 may also be other random access memories, such as static random access memory (SRAM). In addition, the memory 213 may also be a read-only memory (ROM). For example, the read-only memory may be a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), etc. This embodiment does not limit the number and type of the memory 213.

[0060] For example, Figure 2 The processor 211 in the embodiment may be implemented by a chip, such as Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of a chip provided in the present application. By way of example, the chip 300 includes a core 301 , a CPU 302 , a system buffer 303 , an input / output (I / O) device 305 and a DDR 306 .

[0061] Among them, CPU 302 is used to accept tasks (such as compression tasks, decompression tasks, and disaster recovery tasks) and call core 301 to perform the tasks. In the case where chip 300 has multiple cores 301, CPU 302 is also used to undertake scheduling tasks. For example, CPU 302 can be implemented by an ARM processor, which is small in size, low in power consumption, uses a 34-bit reduced instruction set, and has simple and flexible addressing. Of course, in some embodiments, CPU 302 can also be implemented by other processors.

[0062] The core 301 is used to provide the computing power required for disaster recovery tasks, etc. In an optional scenario, the core 301 includes a load / store unit (LSU), a cube computing unit, a scalar computing unit, a vector computing unit, and a buffer. Among them, the LSU is used to load the data to be processed and store the processed data, and can also be used for the read and write management of the internal data in the core between different buffers, and to complete some format conversion operations. The cube computing unit is used to provide the core computing power of matrix multiplication. The scalar computing unit is a single instruction single data (SISD) processor, which only processes one piece of data (usually an integer or a floating point number) at the same time. The vector computing unit, also known as an array processor, is a processor that can directly operate a group of arrays or vectors for calculation. The number of buffers may be one or more, such as the buffer mainly refers to the level 1 buffer (L1 buffer), the buffer is used to temporarily store some data that the core 301 needs to use repeatedly to reduce reading and writing from the bus. In addition, the implementation of certain data format conversion functions also requires that the source data is located in the buffer. The system buffer 303 mainly refers to the secondary cache, which is used to temporarily store input data, intermediate results or final results passing through the chip.

[0063] DDR 306 is an off-chip memory, which can also be replaced by high bandwidth memory (HBM) or other off-chip memory. DDR 306 is located between the chip and the external memory, overcoming the access speed limitation when computing resources share memory reading and writing.

[0064] The I / O device 305 included in the chip 300 refers to the hardware for data transmission, and can also be understood as a device connected to the I / O interface. Common I / O devices 305 include network cards, printers, keyboards, mice, etc. All external memories can also be used as I / O devices 305, such as hard disks, floppy disks, optical disks, etc.

[0065] The core 301, the CPU 302, the system buffer 303, the I / O device 305, and the DDR 306 are connected via a bus. The bus may include a path for transmitting information between the above components (such as the CPU 302 and the system buffer 303). In addition to the data bus, the bus may also include a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus may be a PCIe bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a computer express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. For example, the core 301 can access these I / O devices 305 via the PCIe bus. The core 301 is connected to the system buffer 303 via the DDR bus. Here, different system buffers 303 may use different data buses to communicate with the core 301, so the DDR bus may also be replaced by other types of data buses, and the embodiment of the present application does not limit the bus type.

[0066] For example, when the CPU 302 loads the data to be processed by the disaster recovery task (such as the indicator data of the business system) into the DDR 306, the LSU in the core 301 reads (loads) the data from the DDR 306, and processes the data to obtain the processing result (such as the first request). After obtaining the processing result, the LSU loads (stores) the processing result into the DDR 306, and the network interface card feeds back the processing result.

[0067] It is to be understood that the structures illustrated in the above embodiments do not constitute specific limitations on the disaster recovery platform, computing device, and chip. In other embodiments, the disaster recovery platform may include more or fewer components, and the computing device and chip may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0068] Combination of the above Figures 1 to 3The disaster recovery platform used in the disaster recovery method, as well as the computing devices and chips that can be used to implement control nodes and business nodes, are described below. Figures 1 to 3 The content shown provides a detailed description of the disaster recovery method provided by this application.

[0069] Figure 4 The present application provides a flow chart of a disaster recovery method, which can be applied to a disaster recovery platform, which can include a control node and a business system. Figure 1 For more information about the disaster recovery platform, see the above article. Figures 1 to 3 The relevant description will not be repeated here.

[0070] Here, the disaster recovery platform has Figure 1 The described architecture is used as an example to describe the disaster recovery method provided by this application. Figure 4 As shown, the disaster recovery method may include the following S410 to S480.

[0071] S410, the control node obtains indicator data of the business system.

[0072] According to the needs of actual applications, the control node can obtain indicator data of various indicators of the business system. The various indicators include but are not limited to: rate, bandwidth, throughput, latency, interface error rate, synchronization status of the middleware, recovery point objective (RPO), the number of microservices associated with the middleware, etc. The number of microservices associated with the middleware may refer to the number of microservices that use the middleware to process data.

[0073] In some possible scenarios, after obtaining the indicator data of multiple indicators of the business system, the control node can determine the failure scenario of the business system and the status of the business system according to the indicator data of some or all of the multiple indicators. For example, the business system includes regions 1 to 3. Region 1 is the main region, and regions 2 and 3 are multi-active regions. The middleware of regions 1 to 3 is the database, and the middleware of regions 1 to 3 are databases 1 to database 3 respectively. In this case, the control node can determine whether there is a fault in the middleware of the multi-active region of the business system and the status of the business system according to the obtained RPO1 between database 2 and database 1, the RPO2 between database 3 and database 1, and the number of microservices associated with each database in database 2 and database 3.

[0074] Generally, the control node can obtain the RPO according to the synchronization time between the database of the tested area and the database of the target area. For example, the control node obtains the synchronization time between database 2 of area 2 and database 1 of area 1, and obtains the RPO between database 2 and database 1. In some possible scenarios, the RPO between database 2 of area 2 and database 1 of area 1 can become the RPO of database 2 of area 2.

[0075] Based on the description of the indicator data above, the following describes the method of obtaining the indicator data. The control node can obtain the indicator data in a variety of ways, and examples of two possible methods are given below.

[0076] Example 1: The control node actively obtains indicator data of the business system.

[0077] In this case, the control node can send an indicator data acquisition request to the business node, middleware, etc. of the business system, and based on the indicator data sent by the business node, middleware, etc., the control node can periodically send an indicator data acquisition request to the business node, middleware, etc., or send an indicator data acquisition request to the business node, middleware, etc. according to the needs of the actual application, which is not limited in this application.

[0078] Example 2: The control node passively receives indicator data of the business system.

[0079] In this case, commands can be preset in the business nodes and middleware of the business system so that the business nodes and middleware of the business system actively send indicator data to the control node. And the control node receives the indicator data sent by the business nodes and middleware of the business system. The business nodes and middleware of the business system can periodically send indicator data to the control node, or send indicator data to the control node when the data sending conditions are met, which is not limited in this application. The data sending conditions may include but are not limited to: greater than or equal to the RPO threshold, less than or equal to the number of microservices associated with the middleware, etc. For example, in the case where the RPO of the middleware in area 2 of the multi-active area in the business system is greater than the RPO threshold, and the number of microservices associated with the middleware is zero, the business system can send the middleware RPO of area 2 of the business system, the number of microservices associated with the middleware, etc. to the control node. According to actual needs, the data sending conditions can also be other conditions, which are not limited in this application.

[0080] The above gives two possible examples of the control node obtaining the indicator data. According to the actual application requirements, the control node can also use other methods to obtain the indicator data of the business system, which is not limited in this application. After obtaining the indicator data, the control node can execute S420 to determine the state of the business system.

[0081] S420: The control node determines that the business system is in a first state according to the indicator data.

[0082] The first state is that the business system has at least one abnormal service node that cannot provide services normally. The abnormal service node is a region or an available partition.

[0083] Depending on the number and type of abnormal service nodes, the business system may be in the following states.

[0084] Case 1: The number of abnormal service nodes is 1 and it is a region.

[0085] In this case, the abnormal service node can be the main area of ​​the business system or the multi-active area of ​​the business system.

[0086] Case 2: The number of abnormal service nodes is greater than 1 and all are regions.

[0087] In this case, the abnormal service node can be the main area and the multi-active area of ​​the business system, or the multi-active area of ​​the business system.

[0088] Case 3: The number of abnormal service nodes is 1 and it is an available partition.

[0089] In this case, the abnormal service node can be an available partition in the main area of ​​the business system, or an available partition in the multi-active area of ​​the business system.

[0090] Case 4: The number of abnormal service nodes is greater than 1 and all are available partitions.

[0091] The number of abnormal service nodes is M, where M is greater than 1 and is an integer. In this case, the abnormal service nodes may be M available partitions in the main area of ​​the business system, or M available partitions in the multi-active area of ​​the business system, or N available partitions in the main area of ​​the business system and MN available partitions in the multi-active area. Where M>N>1, and N is an integer.

[0092] Case 5: The number of abnormal service nodes is greater than 1 and includes regions and availability zones.

[0093] In this case, the area included in the abnormal service node can be the main area or the multi-active area. The available partition included in the abnormal service node can be the available partition of the main area or the available partition of the multi-active area. For example, the abnormal service node includes the available partitions of the main area and the multi-active area. For another example, the abnormal service node includes the available partitions of the multi-active area and the main area. For another example, the abnormal service node includes the available partitions 31 in area 2 of the multi-active area and area 3 of the multi-active area, and so on. Depending on the different indicator data of the business system obtained in the actual application process, the abnormal service node can also have other combinations, which are not limited in this application.

[0094] The above text gives several possible examples of the status of the business system based on the number of abnormal service nodes and the types of abnormal service nodes. In actual application, the business system may be in other states depending on the indicator data of the business system obtained by the control node. In other words, the first state may also be other states in addition to the states given in situations 1 to 5, and this application is not limited to this.

[0095] Based on the above description of the first state, the following describes a process in which the control node determines that the business system is in the first state according to the indicator data.

[0096] The control node can determine the state of the business system based on the multiple states and the indicator data corresponding to each state in the multiple states, according to the acquired indicator data. The multiple states and the indicator data corresponding to each state in the multiple states can be preset, or can be set by the user according to the needs of the actual application, and this application is not limited to this. For the process of the user setting multiple states and the indicator data corresponding to each state in the multiple states according to the needs of the actual application, please refer to the following Figure 6 The relevant description will not be repeated here. According to the needs of actual applications, multiple states and the indicator data corresponding to each state in the multiple states can be recorded in a variety of ways, such as state lists, graphs, etc., and this application is not limited to this. The following takes the control node using a state list to record multiple states and the indicator data corresponding to each state in the multiple states as an example to illustrate the process of the control node determining the state of the business system.

[0097] In some possible situations, the state list includes the first state and the indicator data corresponding to the first state. When the indicator data acquired by the control node matches the indicator data corresponding to the first state, the control node determines that the business system is in the first state.

[0098] Exemplarily, the business system includes regions 1 to 3. Region 1 is the main region, and regions 2 and 3 are multi-active regions. The middleware of region 1 and region 2 are database 1 and database 2, respectively. The control node obtains the RPO1 of data synchronization between database 1 and database 2 as 12000 milliseconds (ms), and there is 1 microservice associated with database 2. The state list includes the first state (such as the multi-active region database RPO is too large) and the indicator data corresponding to the first state. The indicator data corresponding to the first state includes: RPO ≥ 10000ms and the number of associated microservices is greater than 0. In this case, the control node can determine that the business system is in the first state, that is, the multi-active region database RPO is too large, and the first state is that the business system has region 2 that cannot provide services normally, based on the obtained indicator data of database 2 (such as RPO1 is 12000ms, the number of associated microservices is 2), and the indicator data corresponding to the multi-active region database RPO is too large recorded in the state list (RPO ≥ 10000ms and the number of associated microservices is greater than 0).

[0099] The above text takes the abnormal service node as an example to describe the process of the control node determining the state of the business system based on the indicator data. In the case where the abnormal service node is an available partition, the control node can also determine the state of the business system in the manner described above, which will not be repeated here. After the control node determines the state of the business system, and the state indicates that there is at least one abnormal service node in the business system that cannot provide services normally, the control node can also determine the state migration rule that matches the state, so that the business system after executing the state migration rule can provide normal services to the outside world, providing guarantees for meeting the SLO. Regarding the process of the control node determining the state migration rule that matches the state, please refer to S430 below.

[0100] S430: The control node determines a first state transition rule that matches the first state.

[0101] The first state migration rule is used to indicate: migrate the business system from the first state to the second state. The first state migration rule is used to migrate the service of at least one abnormal service node to at least one normal service node. The second state is to migrate the service of at least one abnormal service node to at least one normal service node that can provide services normally.

[0102] The state migration rule may refer to a rule for adjusting the state of the business system. According to the needs of the actual application, one or more actions may be adopted to adjust the state of the business system. In this case, the state migration rule may include an initial state, an action corresponding to the initial state, and an end state. For example, the first state migration rule may include: the RPO between the multi-active region 2 database and the primary region database is too large (referred to as the multi-active region 2 database RPO is too large), the traffic of the multi-active region 2 is removed, and services are provided in other regions of the business system except the multi-active region 2. Removing the traffic of the multi-active region 2 may refer to migrating the services provided by the multi-active region 2 to other regions.

[0103] The state transition rule is described above. The following describes a process in which the control node determines the first state transition rule that matches the first state.

[0104] In some possible situations, the control node may determine the first state transition rule that matches the first state according to the matching relationship. The matching relationship may include multiple states and state transition rules corresponding to each of the multiple states. The matching relationship may be preset or set according to the needs of the application scenario, which is not limited in this application. When the matching relationship is set according to the needs of the application scenario, the following may be used Figure 6 The control node can record the matching relationship in a variety of ways, such as a state transition list, a state transition diagram, etc.

[0105] The following takes the control node using the state transition diagram to record the matching relationship as an example to illustrate the process of the control node determining the first state rule matching the first state. Specifically: the control node determines the state transition rule corresponding to the first state of the state transition diagram, and determines the state transition rule as the first state transition rule.

[0106] According to the needs of actual applications, the control node may instruct the service system to migrate from the first state to the second state in a variety of ways. Examples of two possible ways are given below.

[0107] Example 1: When the control node detects that the service system is in the first state, the control node instructs the service system to migrate from the first state to the second state.

[0108] In this case, when the control node detects that the business system is in the first state, it executes S430 to determine the first state migration rule corresponding to the first state. And the control node generates and sends a first request to the business system according to the first state migration rule. The first request is used to instruct the business system to migrate from the first state to the second state. In this way, it is guaranteed that the business system can quickly recover to provide normal services to the outside world.

[0109] Example 2: When the control node detects that the service system is in the first state and satisfies a migration condition, the control node instructs the service system to migrate from the first state to the second state.

[0110] Migration conditions may include but are not limited to: the number of times the business system is detected to be in the first state within a set time is greater than or equal to a number threshold, etc. In this case, when the control node detects that the business system is in the first state, it can obtain the indicator data of the business system immediately or after a waiting period, and determine the state of the business system based on the obtained indicator data, and determine whether to generate and send the first request to the business system according to the reconfirmed state of the business system. In this way, the impact of misjudgment caused by inaccurate state detection is reduced, and the resources consumed by system recovery caused by misjudgment are saved.

[0111] S440: The control node sends a first request to the service system.

[0112] The first request is used to instruct the business system to migrate from the first state to the second state.

[0113] In some possible situations, the control node may also provide warning information when sending the first request to the business system. The warning information is used to indicate that there is at least one abnormal service node in the business system. In this way, the operator can determine the state of the business system and improve the friendliness of the interaction.

[0114] The control node can provide alarm information in a variety of ways. Examples of several possible ways are given below.

[0115] Example 1: The control node displays an alarm interface and displays alarm information in the alarm interface.

[0116] The alarm information is used to indicate that there is at least one abnormal service node in the business system. The alarm information may include but is not limited to: text, image, etc. For example, the control node directly displays "multi-active region database RPO is too large" in the alarm interface.

[0117] Example 2: The control node emits an alarm sound.

[0118] The control node can send out an alarm sound in a preset manner. For example, if the business system is in a multi-active region and the RPO of the database is too large, the control node sends out the first alarm sound (such as a long beep, etc.). According to different preset methods, the control node sends out an alarm sound corresponding to the state of the business system.

[0119] Two possible examples of alarm information are given above. In some possible situations, other methods may be used to provide alarm information, such as combining an alarm interface with an alarm sound, etc. This application is not limited to this.

[0120] After the control node executes S440 to send the first request to the business system, the corresponding business system may execute the following S450 to receive the first request.

[0121] S450: The business system receives a first request.

[0122] S460: The business system migrates from the first state to the second state in response to the first request.

[0123] In this case, the business system, in response to the first request, migrates the service provided by at least one abnormal service node indicated by the first state to be provided by at least one normal service node. And when the business system receives other requests for requesting services, the business system uses at least one normal service node to process other requests, and uses at least one normal service node to provide the services requested by other requests. In some possible situations, the state used to indicate that the business system has an abnormal service node can also be called a transient state (such as the first state, the second state), and the state used to indicate that the business system does not have an abnormal service node can also be called a steady state (such as the third state).

[0124] For example, the business system includes regions 1 to 3. Region 2 is a multi-active region, and the control node determines that region 2 in the multi-active region cannot provide normal services to the outside world, that is, region 2 is an abnormal service node. The control node sends request 1 to the business node. Request 2 is used to indicate that the business system is to be migrated from using region 2 to provide service 1 to not using region 2. In this case, the business system receives request 1 and migrates service 1 provided by region 2 to that provided by region 1 and region 3. And when the business system receives request 2 for requesting service 2, the business system uses region 1 and region 3 to provide service 2.

[0125] The above describes the process of migrating the business system to the second state. After the business system migrates to the second state, in order to improve the efficiency of the business system in providing services, the control node may send a new request (such as a second request) to the business system when at least one abnormal service node can provide services normally, so that the business system migrates from the second state to the third state. Specifically, the control node may execute the following S470 to send a second request to the business system. The business system may execute the following S480 to migrate the business system from the second state to the third state.

[0126] S470: The control node determines that at least one abnormal service node indicated by the first status can provide services normally, and the control node sends a second request to the service system.

[0127] The control node may adopt the following (1) to (4) to send the second request to the service system.

[0128] (1) The control node determines an abnormal service node among at least one abnormal service node that can provide services normally.

[0129] After sending the first request to the business system, the control node may also obtain the indicator data of at least one abnormal service node, and determine whether the at least one abnormal service node can provide services normally according to the obtained indicator data. The control node may obtain the indicator data of at least one abnormal service node in a variety of ways.

[0130] For example, the control node may periodically obtain indicator data of at least one abnormal service node.

[0131] For another example, the control node obtains the indicator data of at least one abnormal service node based on the received command. The command may be initiated by the user according to the needs of the actual application.

[0132] For example, when the control node detects that the acquisition condition is met, the control node acquires the indicator data of at least one abnormal service node. The acquisition condition may include but is not limited to: the number of service nodes supporting normal service provision is less than or equal to the normal service quantity threshold, the number of abnormal service nodes is greater than or equal to the abnormal service quantity threshold, etc.

[0133] (2) The control node adds the abnormal service node that can provide services normally to at least one normal service node.

[0134] The abnormal service nodes that can provide normal services may be all of the at least one abnormal service node or part of the at least one abnormal service node. In this case, the control node may add the abnormal service nodes that support providing normal services in the at least one abnormal service node to the at least one normal service node.

[0135] (3) The control node obtains a second state transition rule that matches the second state.

[0136] Among them, the second state migration rule is used to indicate that the business system is migrated from the second state to the third state. The third state is that the business system uses the normal service node to provide services. Similarly, the control node can use the above S430 method to determine the second state migration rule that matches the second state. Please refer to the above for related descriptions, which will not be repeated here.

[0137] (4) The control node sends a second request to the business system.

[0138] The second request is used to instruct the business system to migrate from the second state to the third state. Similarly, the control node can generate and send the second request to the business system in the manner of S440 above. Please refer to the above for related descriptions, which will not be repeated here.

[0139] Exemplarily, the business system includes: service nodes 1 to service nodes n. At least one abnormal service node includes: service node 1 and service node 10. State 2 is that the business system uses service nodes 11 to service nodes n to provide services. The control node determines that service nodes 1 to service nodes 6 can provide services normally based on the re-acquired indicator data. In this case, the control node adds service nodes 1 to service nodes 6 to the service nodes that can provide services normally. And the service nodes that can provide services normally in the business system include: service nodes 1 to service nodes 6, service nodes 11 to service nodes n. The control node determines a state migration rule 2 that matches state 2. State migration rule 2 is used to indicate that the business system is migrated from state 2 (i.e., using service nodes 11 to service nodes n to provide services) to state 3 (i.e., using service nodes 1 to service nodes 6, service nodes 11 to service nodes n) to provide services. The control node generates and sends request 2 based on state migration rule 2. Request 2 is used to instruct the business system to migrate from state 2 to state 3.

[0140] Accordingly, the business system may execute S480A. S480A includes the business system receiving a second request.

[0141] S480: The business system migrates from the second state to the third state in response to the second request.

[0142] Similarly, the business system can migrate from the second state to the third state by using the method of S460 above. Please refer to the above for related descriptions, which will not be repeated here.

[0143] In some possible situations, the control node may use multiple modules to implement the above disaster recovery method. Figure 5 A flow chart of a disaster recovery method provided for this application is shown as follows: Figure 5 As shown, the control node implements the disaster recovery method provided by the present application based on the system observable service, message queue, disaster recovery multi-active service and data storage service. According to the needs of the actual application, the distributed cache can be used to provide data storage service. Specifically, the process can include the following (1) to (8).

[0144] (1) The observability service obtains the indicator data of the business system, determines the faults in the business system, and sends the indicator data and the faults in the business system to the message queue.

[0145] In this case, the system observable service can execute S410 to obtain the indicator data of the business system. After obtaining the indicator data of the business system, the system observable service can also determine the faults existing in the business system. After determining the faults existing in the business system, the system observable service can also send the indicator data of the business system and the faults existing in the business system to the message queue.

[0146] (2) The message queue receives the indicator data of the business system and the faults existing in the business system.

[0147] (3) The message queue receives a data acquisition request sent by the disaster recovery multi-active service, and sends the indicator data of the business system and the faults existing in the business system to the disaster recovery multi-active service in response to the data acquisition request.

[0148] Data acquisition requests are used to obtain: indicator data of the business system and requests for faults in the business system.

[0149] (4) After obtaining the indicator data of the business system and the faults existing in the business system, the disaster recovery multi-active service can also save the indicator data of the business system to the distributed cache.

[0150] According to the needs of actual applications, the disaster recovery multi-active service can also use the distributed cache to obtain indicator data of other related indicators. Other related indicators may refer to indicators other than the indicators that have been obtained and are required to determine the state of the business system. For example, the indicator data required to determine the business system includes indicator data 1 to indicator data 10 of indicators 1 to 10. The indicator data obtained by the disaster recovery multi-active service from the message queue includes indicator data 1 to indicator data 8. In this case, the indicator data of other related indicators are indicator data 9 and indicator data 10, and the disaster recovery multi-active service can obtain indicator data 9 and indicator data 10 from the distributed cache.

[0151] (5) The disaster recovery multi-active service uses the obtained indicator data of the business system to determine the status of the business system.

[0152] In this case, the disaster recovery multi-active service may execute S420 to determine the state of the business system.

[0153] In some possible situations, the disaster recovery multi-active service can also save the status of the business system to the data storage system.

[0154] (6) After determining the state of the business system, the disaster recovery multi-active service can also determine the state migration rules corresponding to the state.

[0155] In this case, the disaster recovery multi-active service may execute S430 to determine a state migration rule.

[0156] (7) The disaster recovery multi-active service can send requests to the business system based on the determined state migration rules.

[0157] This request is used to instruct the business system to execute state transition rules to change the state of the business system.

[0158] In some possible situations, the disaster recovery multi-active service can also execute (8) to prompt the operator that there is a fault in the business system.

[0159] (8) Disaster recovery multi-active service provides alarm information.

[0160] Alarm information is used to indicate that there is a fault in the business system.

[0161] In some possible scenarios, the above (1) to (4) may be referred to as abnormality perception, and (5) to (8) may be referred to as abnormality handling or abnormality processing, etc.

[0162] Combination of the above Figures 1 to 5 The present application describes a disaster recovery method. In the method, a control node determines the state of a business system based on the acquired indicator data of the business system, determines the corresponding state migration rule based on the state of the business system, and sends a request to the business system. The business system changes the state of the business system in response to the request, uses a normal service node to provide services, and provides a guarantee for the business system to provide services normally to the outside.

[0163] According to the above description, the business system includes multiple regions and multiple available zones, which may become abnormal service nodes. Depending on the distribution and number of regions and available zones that become abnormal service nodes, the business system may have multiple fault scenarios. Business systems in different fault scenarios may be in different states, and different states often correspond to different state migration rules. The operator determines the state migration rules that match each state in multiple states. Due to the huge number of states, it may be impossible to accurately determine the state migration rules corresponding to each state, and the operator has a lot of work to determine the state migration rules corresponding to the state. And when the matching rules between the state and the state migration rules change, the operator needs to re-determine the state migration rules corresponding to each state, which is inefficient and labor-intensive.

[0164] The present application also provides a method for establishing a matching relationship, in which an operator can input a construction rule for a matching relationship between a state and a state transition rule, and a computing device (such as a control node) constructs a matching relationship between each state in a plurality of states and each state transition rule in a plurality of state transition rules according to the construction rule, and obtains a matching relationship between a state and a state transition rule corresponding to the state. The following takes the construction of a matching relationship by a control node as an example to exemplify the matching relationship construction method provided in the present application. According to the needs of actual applications, other computing devices can also be used to construct a matching relationship, and the control node can obtain a matching relationship from other computing devices. The present application does not limit the device for constructing a matching relationship.

[0165] Figure 6 A flowchart of a method for establishing a matching relationship provided in this application is shown in FIG. Figure 6 As shown, the method may include the following (1) to (4).

[0166] (1) The control node obtains multiple states of the business system.

[0167] The multiple states include a first state, a second state, and a third state.

[0168] In some possible situations, the control node may adopt multiple methods to obtain multiple states of the business system. Examples of two possible methods are given below.

[0169] Example 1: A control node receives multiple states sent by other devices.

[0170] In this case, the multiple states sent by other devices and received by the control node may be presented in a variety of ways, including but not limited to: a state list, a graph, and the like.

[0171] Example 2: The control node receives multiple states configured by the user.

[0172] In this case, the control node may provide a status configuration interface to prompt the user to input status information, and the control node obtains the status based on the status information input by the user. Specifically, the process may include the following ① to ③.

[0173] ①The control node provides a status configuration interface.

[0174] In this case, the control node may use a display device to display a status configuration interface.

[0175] According to the needs of actual applications, the control node can prompt the user to input status information in a variety of ways. For example, the control node can display prompt information in the status configuration interface to prompt the user to input status information. For example, the control node displays "Name" in the status configuration interface to prompt the user to enter the status name, the control node displays "Target Type" in the status configuration interface to prompt the user to enter the fault scenario, and the control node displays "Status Condition Definition" in the status configuration interface to prompt the user to enter indicator data to determine whether the business system is in the currently defined state. According to the needs of actual applications, the control node can also display more or less prompt information to prompt the user to enter status information, and this application is not limited to this. For example, the control node can also display "Description" in the status configuration interface to prompt the user to enter a description of the status.

[0176] ② The control node obtains multiple status information in response to the operation on the status configuration interface.

[0177] Each of the multiple state information includes one or more of the following: state name, fault scenario, and first state condition. The state name is used to indicate: the name of the configured state. The fault scenario is used to indicate: the fault scenario applicable to the configured state. The first state condition is used to indicate: indicator data for determining that the business system is in the configured state. And the control node can obtain the state name from the name input area, obtain the fault scenario from the fault scenario input area, and obtain the first state condition from the state condition definition area.

[0178] In some possible situations, the failure scenarios include: disaster recovery mode, middleware type, and application technology stack type. The disaster recovery mode is used to indicate the disaster recovery and load balancing strategies. The disaster recovery mode may include but is not limited to: multi-read in different locations, multi-active in the same city, multi-active in different locations, etc. The middleware type is used to indicate the database system that processes data in the business system. The middleware types include but are not limited to: database, redis, kafka, etc. The database includes but is not limited to: relational database management system MySQL, database mongodb based on distributed file storage, and open source distributed key-value (Key-Value, KV) storage system cassandra. The application technology stack type is used to indicate the software development language of the business system. The application technology stack types include but are not limited to: open source programming language go, object-oriented programming language java, object-oriented dynamic type language python.

[0179] In this case, the user may input one or more pieces of status information based on the status configuration interface provided by the control node, and the control node acquires one or more pieces of status information in response to the user's input operation on the status configuration interface.

[0180] ③ The control node obtains multiple states according to multiple state information.

[0181] In this case, the control node can obtain the state name, fault scenario, and first state condition of each state information in multiple state information, and determine a state based on the state name, fault scenario, and first state condition included in the obtained state information.

[0182] In some possible situations, the control node implements the above ① to ③ using pseudo code to obtain multiple states.

[0183] Figure 7 An example diagram of an acquisition status provided for this application, such as Figure 7 As shown, the control node displays the status configuration interface, and the control node uses the above method based on the user input in the status configuration interface. Figure 7The information shown generates the status "Multi-active region database RPO is too large". That is, when the RPO of the multi-active region database of the business system is greater than 10000ms and the number of microservices associated with the database is greater than 0, it is confirmed that the business system is in the status "Multi-active region database RPO is too large" and the fault scenario corresponding to this status is middleware service. The control node can use the pseudo code in Table 1 to implement the above status.

[0184] Table 1 An example of obtaining status

[0185]

[0186]

[0187] (2) The control node obtains multiple state transition rules.

[0188] The multiple state transition rules include a first state transition rule and a second state transition rule. Each state transition rule in the multiple state transition rules is used to indicate that if the business system is in an initial state, an action matching the initial state is executed on the business system to put the business system in an end state. The initial state is one of the multiple states, and the end state is one of the multiple states, and the initial state and the end state are different.

[0189] In some possible situations, the control node may adopt multiple methods to obtain multiple state transition rules. Examples of two possible methods are given below.

[0190] Example 1: A control node receives multiple state transition rules sent by other devices.

[0191] In this case, the multiple state transition rules sent by other devices and received by the control node may be presented in a variety of ways, including but not limited to: a state list, a diagram, and the like.

[0192] Example 2: The control node receives multiple state transition rules configured by the user.

[0193] Similar to the control node obtaining multiple states, the control node can provide a state migration rule configuration interface to prompt the user to input rule information, and the control node obtains the state migration rule based on the rule information input by the user. Specifically, the process can include the following ① to ③.

[0194] ①The control node provides a state migration rule configuration interface.

[0195] Similarly, the control node may use a display device to display a state transition rule configuration interface.

[0196] According to the needs of actual applications, the control node can prompt the user to input rule information in a variety of ways. For example, the control node can display prompt information in the state migration rule configuration interface to prompt the user to input rule information. For example, the control node displays "Name" in the state migration rule configuration interface to prompt the user to enter the state migration rule name, and the control node displays "State Condition Definition" in the state migration rule configuration interface to prompt the user to enter the indicator data of the business system in the initial state, the actions performed on the business system in the initial state, and the end state of the business system after the action is performed. According to the needs of actual applications, the control node can also display more or less prompt information to prompt the user to enter rule information, and the present application is not limited to this. For example, the control node can also display "Description" in the state migration rule configuration interface to prompt the user to enter a description of the state migration rule.

[0197] ② The control node obtains multiple rule information in response to the operation on the state transition rule configuration interface.

[0198] Among them, each rule information in the multiple rule information includes one or more of the following: a state transition rule name and a second state condition. The state transition rule name is used to indicate: the name of the configuration state transition rule; the second state condition is used to indicate: indicator data for determining that the business system is in an initial state, the action performed on the business system in the initial state, and the end state of the business system after the action is performed.

[0199] Similar to the control node acquiring multiple states, the user can input one or more rule information based on the state transition rule configuration interface provided by the control node, and the control node acquires one or more rule information in response to the user's input operation in the state transition rule configuration interface.

[0200] ③ The control node obtains multiple state migration rules based on multiple rule information.

[0201] Similar to the control node acquiring multiple states, the control node can acquire the state transition rule name and the second state condition of each rule information in multiple rule information, and determine a state transition rule based on the state transition rule name and the second state condition included in the acquired rule information.

[0202] Figure 8 An example diagram of an acquisition state transition rule provided for this application, such as Figure 8 As shown, the control node displays the state transition rule configuration interface, and the control node uses the above method based on the user input in the state transition rule configuration interface. Figure 8The information shown generates the state transition rule "When the RPO of the multi-active region database is too large, remove the multi-active region traffic". That is, when the RPO of the multi-active region database of the business system is high and the main region Grid is normal (i.e., the initial state), execute the workflow "remove the multi-active region traffic" (the action performed on the business system in the initial state) to make the "multi-active region no traffic" of the business system (i.e., the end state). The control node can use the pseudo code in Table 2 to implement the above state transition rule.

[0203] Table 2 An example of obtaining state transition rules

[0204]

[0205]

[0206] (3) The control node determines a matching relationship between the multiple states and the multiple state transition rules according to the multiple states and the multiple state transition rules.

[0207] In this case, the control node can determine the matching relationship between the multiple states and the multiple state transition rules according to the state names of the multiple states and the initial states of the multiple state transition rules. Specifically, the control node can obtain the state name of each state in the multiple states (such as the first state), and determine the state transition rule whose initial state is the first state from the multiple state transition rules (such as the first state transition rule), and determine the matching relationship between the first state and the first state transition rule.

[0208] For example, multiple states include a first state. Multiple state transition rules include a first state transition rule. The first state transition rule includes: a first state (i.e., an initial state), a first action, and a second state (i.e., an end state). Among them, the first action is an action performed when the business system is in the first state, and the second state is an expected state after the first action is performed on the business system in the first state. In this case, the control node can determine the matching relationship between the first state and the first state transition rule based on the first state and the initial state of the first state transition rule.

[0209] In some possible situations, after the control node establishes the matching relationship using the above (1) to (3), in order to improve the friendliness of the interaction, the control node can also perform (4) to establish a state transition diagram. And the operator can quickly locate the state of the business system (i.e., the initial state) based on the state transition diagram, and the action performed in the state, and the expected state of the business system after the action is performed (i.e., the end state), thereby ensuring the predictability of the business system state, improving the reliability of the disaster recovery method, and the friendliness of the interaction.

[0210] (4) The control node establishes a state transition diagram based on the matching relationship between multiple states and multiple state transition rules.

[0211] The state transition diagram includes: multiple initial states, actions corresponding to each of the multiple initial states, and end states corresponding to the actions. One initial state corresponds to one action, and one action corresponds to one end state.

[0212] The disaster recovery platform can use the acquired state template and state transition rule template to instantiate the state and state transition rule. The disaster recovery platform can obtain the instantiated state and state transition rule in a variety of ways. For example, the disaster recovery platform receives the state template and state transition rule template input by the user, and instantiates the state and state transition rule according to the acquired business system information. Fig. 9 An example flow diagram of an instantiation state and state transition rules provided for this application, such as Fig. 9 As shown, the process includes the following (1) to (6).

[0213] (1) The disaster recovery platform obtains the status template and status migration rule template, and saves the obtained templates.

[0214] In this case, the disaster recovery platform may receive the template input by the disaster recovery platform administrator and save the received template.

[0215] (2) The observability service obtains the business system’s region, available partitions, business nodes, middleware, application technology stack type, etc., and saves the obtained information.

[0216] (3) The disaster recovery platform obtains the disaster recovery model and manages the business system.

[0217] In this case, the disaster recovery platform can receive the disaster recovery mode and managed business systems input by the business administrator.

[0218] (4) The disaster recovery platform obtains the middleware and application technology stack types of the business system from the observable services.

[0219] (5) The disaster recovery platform determines the state template and state migration rule template based on the obtained middleware and application technology stack types.

[0220] (6) The disaster recovery platform instantiates the state and state migration rule based on the acquired state template and state migration rule template.

[0221] In some possible situations, the fault scenario can be broken down into multiple atomic fault modes. Each atomic fault mode has both the function of isolating the fault when a fault occurs (i.e., there is at least one abnormal service node) to put the business system into a safe state (i.e., not using at least one abnormal service node to provide services, and using at least one normal service node to provide services), and the function of migrating the business system from the safe state to the normal state after the fault is recovered (i.e., all service nodes of the business system provide normal services to the outside world). Specifically, the control node can construct multiple atomic fault modes based on multiple states and multiple state migration rules, and determine the atomic fault mode that matches the fault scenario (i.e., disaster recovery mode, middleware type, application technology stack type), and use the multiple matching atomic fault modes to construct a state migration diagram.

[0222] Fig.10 An example diagram of a state transition diagram provided for this application, such as Fig.10 As shown, the control node constructs atomic failure modes 1 to 3 respectively according to multiple states and multiple state migration rules. Atomic failure mode 1 is used to indicate that the primary region database is unavailable (multiple reads in different locations, multiple active applications in different locations). Atomic failure mode 2 is used to indicate that a single available partition service is unavailable (multiple active in the same city, multiple active in different locations). Atomic failure mode 3 is used to indicate that the RPO of the multiple active region database is too long (multiple reads in different locations). And the control node uses atomic failure modes 1 to 3 to construct a state migration diagram according to the initial state, action, and end state included in each state migration diagram in the multiple states and multiple state migration diagrams.

[0223] In this application, the control node automatically determines the state of the business system based on the indicator data of the business system. When the indicator data of the business system changes, the state of the business system can be determined in a timely and accurate manner. The control node also automatically determines the state migration rules that match the state. When the indicator data of the business system changes, the state migration rules can be determined in a timely and accurate manner, so that the business system can migrate its state in a timely manner, which provides a guarantee for the business system to provide normal services to the outside world. The control node is used to determine the state of the business system and the state matching rules corresponding to the state, which reduces the requirements for the professional knowledge of the staff and reduces the workload.

[0224] It is understandable that in order to implement the functions in the above embodiments, the control node and / or the business system may include hardware structures and / or software modules corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0225] Combined with the above Figures 1 to 10 The disaster recovery method provided by this embodiment is described in detail. Fig.11 , describing the disaster recovery device provided in this embodiment, which can be used to implement the function of the control node in the above method embodiment.

[0226] Fig.11 A schematic diagram of the structure of a disaster recovery device provided in this application is shown in FIG. Fig.11 As shown, the disaster recovery device 1100 includes: a transceiver module 1110 and a processing module 1120. The transceiver module 1110 is used to: obtain indicator data of the business system. The processing module 1120 is used to: determine that the business system is in a first state based on the indicator data. The first state is that there is at least one abnormal service node in the business system that cannot provide services normally. The processing module 1120 is also used to: determine a first state migration rule that matches the first state. The first state migration rule is used to indicate: migrate the business system from the first state to the second state, and the second state is to migrate the service of at least one abnormal service node to at least one normal service node that can provide services normally. The transceiver module 1110 is also used to: send a first request to the business system. The first request is used to instruct the business system to migrate from the first state to the second state.

[0227] In some possible situations, the processing module 1120 is specifically used to: determine a first state transition rule matching the first state according to the state transition diagram. The state transition diagram includes multiple states and state transition rules corresponding to each of the multiple states, and the multiple states include the first state.

[0228] In some possible scenarios, the abnormal service node is a region or an availability zone.

[0229] In some possible situations, the first state migration rule is used to migrate the service of at least one abnormal service node to at least one normal service node.

[0230] In some possible situations, the processing module 1120 is further used to: determine an abnormal service node that can provide services normally among at least one abnormal service node. The processing module 1120 is further used to: add the abnormal service node that can provide services normally to at least one normal service node. And the processing module 1120 is further used to: obtain a second state migration rule that matches the second state. The second state migration rule is used to instruct the business system to migrate from the second state to the third state, and the third state is that the business system uses the normal service node to provide services. The transceiver module 1110 is further used to: send a second request to the business system. The second request is used to instruct the business system to migrate from the second state to the third state.

[0231] In some possible situations, the processing module 1120 is also used to: obtain multiple states of the business system. The multiple states include a first state, a second state, and a third state. The processing module 1120 is also used to: obtain multiple state transition rules. Among them, the multiple state transition rules include a first state transition rule and a second state transition rule, and each state transition rule in the multiple state transition rules is used to indicate: if the business system is in an initial state, then an action matching the initial state is performed on the business system to put the business system in an end state, the initial state is one of the multiple states, the end state is one of the multiple states, and the initial state and the end state are different. And the processing module 1120 is also used to: determine the matching relationship between multiple states and multiple state transition rules based on multiple states and multiple state transition rules.

[0232] In some possible situations, the processing module 1120 is further used to: establish a state transition diagram based on the matching relationship between multiple states and multiple state transition rules. The state transition diagram includes: multiple initial states, actions corresponding to each of the multiple initial states, and end states corresponding to the actions.

[0233] In some possible situations, the disaster recovery device 1100 further includes a display module 1130. The display module 1130 is used to provide a status configuration interface. The processing module 1120 is further used to obtain multiple status information in response to an operation on the status configuration interface. Each of the multiple status information includes one or more of the following: a status name, a fault scenario, and a first status condition. The status name is used to indicate the name of the configured status. The fault scenario is used to indicate the fault scenario applicable to the configured status. The first status condition is used to indicate the indicator data for determining that the business system is in the configured status. And the processing module 1120 is further used to obtain multiple statuses according to the multiple status information. .

[0234] In some possible situations, the failure scenarios include: disaster recovery mode, middleware type, and application technology stack type. The disaster recovery mode is used to indicate disaster recovery and load balancing strategies, the middleware type is used to indicate the database system that processes data in the business system, and the application technology stack type is used to indicate the software development language of the business system.

[0235] In some possible situations, the display module 1130 is also used to provide a state transition rule configuration interface. The processing module 1120 is also used to obtain multiple rule information in response to the operation of the state transition rule configuration interface. Each rule information in the multiple rule information includes one or more of the following: a state transition rule name, a second state condition, the state transition rule name is used to indicate: the name of the configured state transition rule, and the second state condition is used to indicate: indicator data for determining that the business system is in an initial state, the action performed on the business system in the initial state, and the end state of the business system after the action is performed. And the processing module 1120 is also used to obtain multiple state transition rules based on multiple rule information.

[0236] In some possible situations, the display module 1130 is further used to provide warning information. The warning information is used to indicate that there is at least one abnormal service node in the business system.

[0237] Specifically, for more descriptions about the transceiver module 1110, the processing module 1120 and the display module 1130, please refer to the above descriptions about the disaster recovery method, which will not be repeated here.

[0238] When the disaster recovery device 1100 corresponds to executing the steps performed by the control node in the disaster recovery method described in the embodiment of the present application, the above and other operations and / or functions of each module in the disaster recovery device 1100 are respectively for implementing the method flow performed by the control node in the aforementioned figures.

[0239] It is worth noting that if the above-mentioned disaster recovery devices 1100 are implemented through software modules, for example, the software module can be provided to users through a cloud service subscription model, and users can choose different subscription levels according to their needs; for example, the software module can also provide enterprise-level customized services with professional domain customization, interface personalization and extended functions according to the needs of users or enterprises.

[0240] In addition, the disaster recovery device 1100 provided in the present application can also be provided to users as a value-added service, which is not limited in the present application.

[0241] The disaster recovery device of the embodiment of the present application can also be implemented by hardware. For example, the hardware refers to a computing device, a chip or a processor. For the specific implementation of the computing device, please refer to Figure 2 For details about the implementation of chips and processors, please refer to Figure 3 The description is not repeated here.

[0242] The method steps in this embodiment can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a computing device. Of course, the processor and the storage medium can also exist as discrete components in a network device or a terminal device.

[0243] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. In some embodiments, the computing device may also be a terminal device such as a desktop, a laptop or a smart phone. The computing device cluster can be used to implement the function of the control node in the above method embodiment, and can also be used to implement the function of the business system in the above method embodiment, and can also be used to implement the function of the disaster recovery platform in the above method embodiment. The following uses the computing device cluster to implement the function of the control node in the implementation of the above method as an example to exemplify the computing device cluster provided in the present application. Similarly, the way in which the computing device cluster implements the control node function can be referred to so that the computing device cluster implements the function of the business system or the disaster recovery platform in the above method embodiment, which will not be repeated here.

[0244] like Fig.12 As shown, Fig.12 This is a schematic diagram of the structure of a computing device cluster provided by the present application, and the computing device cluster includes at least one computing device 200. The memory 213 in one or more computing devices 200 in the computing device cluster may store the same instructions for executing the disaster recovery method.

[0245] In some possible implementations, the memory 213 of one or more computing devices 200 in the computing device cluster may also store partial instructions for executing the disaster recovery method. In other words, the combination of one or more computing devices 200 may jointly execute instructions for executing the disaster recovery method.

[0246] It should be noted that the memory 213 in different computing devices 200 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the control node. That is, the instructions stored in the memory 213 in different computing devices 200 can implement the functions of one or more modules in the transceiver module 1110, the processing module 1120 and the display module 1130.

[0247] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.13 A possible implementation is shown. Fig.13 As shown, Fig.13 A schematic diagram of a connection between computing devices provided in the present application, wherein two computing devices 200A and 200B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the instructions stored in the memory 213 in the computing device 200A can implement the functions implemented by the transceiver module 1110 and the display module 1130. At the same time, the instructions stored in the memory 213 in the computing device 200B can implement the functions implemented by the processing module 1120.

[0248] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes a disaster recovery method.

[0249] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the disaster recovery method.

[0250] The present application also provides a chip. The chip includes an interface circuit and a control circuit. The interface circuit is used to obtain indicator data of the business system, and the control circuit is used to implement the function of the control node in the disaster recovery method. Alternatively, the interface circuit is used to receive the first request and the second request, and the control circuit is used to implement the function of the business system in the disaster recovery method.

[0251] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD).

[0252] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A disaster recovery method, characterized in that: The method is applied to a disaster recovery platform, the disaster recovery platform includes: a control node and a business system, the method is executed by the control node, and the method includes: Obtaining indicator data of the business system; Determining, according to the indicator data, that the business system is in a first state; the first state is that the business system has at least one abnormal service node that cannot provide services normally; Determine a first state migration rule matching the first state; the first state migration rule is used to indicate: migrate the business system from the first state to a second state, the second state is to migrate the service of the at least one abnormal service node to at least one normal service node that can provide services normally; A first request is sent to the business system; the first request is used to instruct the business system to migrate from the first state to the second state.

2. The method according to claim 1, characterized in that The determining a first state transition rule matching the first state includes: A first state transition rule matching the first state is determined according to a state transition diagram; the state transition diagram includes a plurality of states and a state transition rule corresponding to each of the plurality of states, and the plurality of states includes the first state.

3. The method according to claim 1 or 2, characterized in that: The abnormal service node is a region or an available partition.

4. The method according to any one of claims 1 to 3, characterized in that The first state migration rule is used to migrate the service of the at least one abnormal service node to the at least one normal service node.

5. The method according to any one of claims 1 to 4, characterized in that After sending the first request to the business system, the method further includes: Determining an abnormal service node among the at least one abnormal service node that can provide services normally; adding the abnormal service node capable of providing services normally to the at least one normal service node; Acquire a second state migration rule matching the second state; the second state migration rule is used to instruct the business system to migrate from the second state to a third state, the third state being that the business system uses a normal service node to provide services; A second request is sent to the business system; the second request is used to instruct the business system to migrate from the second state to the third state.

6. The method according to any one of claims 2 to 5, characterized in that: Before determining a first state transition rule matching the first state, the method further includes: Acquire multiple states of the business system; the multiple states include the first state, the second state, and the third state; Acquire multiple state transition rules; wherein the multiple state transition rules include the first state transition rule and the second state transition rule, each of the multiple state transition rules is used to indicate: if the business system is in an initial state, then perform an action matching the initial state on the business system to put the business system in an end state, the initial state is one of the multiple states, the end state is one of the multiple states, and the initial state and the end state are different; According to the multiple states and the multiple state transition rules, a matching relationship between the multiple states and the multiple state transition rules is determined.

7. The method according to claim 6, characterized in that The method further comprises: Based on the matching relationship between the multiple states and the multiple state transition rules, the state transition diagram is established; wherein the state transition diagram includes: multiple initial states, actions corresponding to each of the multiple initial states, and end states corresponding to the actions.

8. The method according to claim 6 or 7, characterized in that: The obtaining of multiple states includes: Provides a status configuration interface; In response to an operation on the status configuration interface, obtaining a plurality of status information; Each of the multiple state information includes one or more of the following: a state name, a fault scenario, and a first state condition, wherein the state name is used to indicate: a name of a configured state, the fault scenario is used to indicate: a fault scenario applicable to the configured state, and the first state condition is used to indicate: indicator data for determining that the business system is in the configured state; The multiple states are acquired according to the multiple state information.

9. The method according to claim 8, characterized in that The failure scenarios include: disaster recovery mode, middleware type, and application technology stack type. The disaster recovery mode is used to indicate disaster recovery and load balancing strategies, the middleware type is used to indicate the database system that processes data in the business system, and the application technology stack type is used to indicate the software development language of the business system.

10. The method according to any one of claims 6 to 9, characterized in that: The obtaining of multiple state transition rules includes: Provides a state migration rule configuration interface; In response to an operation on the state transition rule configuration interface, obtaining a plurality of rule information; Each of the plurality of rule information includes one or more of the following: a state transition rule name and a second state condition, wherein the state transition rule name is used to indicate: a name of a configuration state transition rule, and the second state condition is used to indicate: indicator data for determining that the business system is in an initial state, an action performed on the business system in the initial state, and an end state of the business system after performing the action; The multiple state transition rules are acquired according to the multiple rule information.

11. The method according to any one of claims 1 to 10, characterized in that The method further comprises: Providing alarm information; the alarm information is used to indicate that: there is at least one abnormal service node in the business system.

12. A disaster recovery method, characterized in that: The method is applied to a disaster recovery platform, the disaster recovery platform includes: a control node and a business system, the method is executed by the business system, and the method includes: Receive a first request; wherein the first request is used to instruct the business system to migrate from a first state to a second state, the first state is that there is at least one abnormal service node in the business system that cannot provide services normally, and the second state is to migrate the services of the at least one abnormal service node to the at least one normal service node; In response to the first request, the first state is transitioned to the second state.

13. The method according to claim 12, characterized in that The method further comprises: receiving a second request; wherein the second request is used to instruct the business system to migrate from the second state to a third state, and the third state is that the business system uses the at least one abnormal service node and the at least one normal service node to provide services; In response to the second request, the second state is transitioned to the third state.

14. A disaster recovery platform, characterized in that: The disaster recovery platform includes: a control node and a business system, the control node executes the method according to any one of claims 1 to 11, and the business system executes the method according to claim 12 or 13.

15. A disaster recovery device, characterized in that: The device comprises: The transceiver module is used to obtain the indicator data of the business system; A processing module is used to: determine that the business system is in a first state according to the indicator data; the first state is that the business system has at least one abnormal service node that cannot provide services normally; The processing module is further used to: determine a first state migration rule matching the first state; the first state migration rule is used to instruct: migrating the business system from the first state to a second state, and the second state is to migrate the service of the at least one abnormal service node to at least one normal service node that can provide services normally; The transceiver module is further used to: send a first request to the business system; the first request is used to instruct the business system to migrate from the first state to the second state.

16. A processor, characterized in that: The processor includes an interface circuit and a control circuit; the interface circuit is used to obtain indicator data and cooperate with the control circuit to execute the method described in any one of claims 1-11.

17. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, each computing device includes a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method as described in any one of claims 1-11, or so that the computing device cluster executes the method as described in claim 12 or 13.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes computer instructions; when the computer instructions are executed in a computing device cluster, the computing device cluster executes the method of any one of claims 1-11, or the computing device cluster executes the method of claim 12 or 13.

19. A computer program product, characterized in that When the computer program product is run in a computing device cluster, the computing device cluster executes the method of any one of claims 1 to 11, or the computing device cluster executes the method of claim 12 or 13.

Images