Business service recovery method and device, computer device

Abstract

The application relates to a service recovery method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: a physical node creates a new logical node with the same service attribute as a target logical node in a failed physical node when receiving a service recovery instruction for the target logical node sent by a management node; reading target to-be-written data of each logical volume contained in the target logical node in a logical volume unit, and sending the target to-be-written data to a storage pool; and when determining that the target to-be-written data of any target logical volume corresponding to the target logical node is completed, receiving a data writing request for the target logical volume through the new logical node, and completing a service processing service for the target logical volume. The method can improve service recovery efficiency.

Description

Technical Field

[0001] This application relates to the field of distributed storage technology, and in particular to a business service recovery method, apparatus, and computer equipment. Background Technology

[0002] In large-scale distributed storage systems, there are many physical nodes, each of which hosts at least one logical node. Each logical node handles a large number of business requests. When the physical node hosting a logical node fails, all data on that logical node must be fully recovered before service can resume, resulting in service interruption.

[0003] Specifically, after the logical nodes deployed on the failed physical node are migrated to other normal physical nodes, the data that was not written to disk during the failure (i.e. all cached data to be written) needs to be sent to the storage pool before the logical nodes can continue to receive write requests.

[0004] The above-mentioned business service recovery methods are slow, resulting in long service outage times and slow business service recovery efficiency. Summary of the Invention

[0005] Therefore, it is necessary to provide a business service recovery method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve business recovery efficiency in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a method for restoring business services. The method is applied to physical nodes in a distributed system, which further includes a management node and other physical nodes. Each physical node deploys at least one logical node, and each logical node corresponds to at least one logical volume. The method includes:

[0007] Upon receiving a service recovery instruction from the management node for a target logical node in a faulty physical node, a new logical node with the same service attributes as the target logical node is created.

[0008] Read the target data to be written from each logical volume contained in the target logical node, and send the target data to be written to the storage pool, on a logical volume basis.

[0009] If the target data to be written to any target logical volume corresponding to the target logical node is completed, the new logical node receives the data write request for the target logical volume and completes the business processing service for the target logical volume.

[0010] Logical volumes were created based on logical nodes, and data recovery was performed using logical volumes. Since the business services were completed first for the logical volumes whose data was being recovered, instead of waiting for all logical volumes to be recovered before resuming business services, the business interruption time was reduced.

[0011] In one embodiment, upon receiving a service recovery instruction from the management node for a target logical node among the failed physical nodes, creating a new logical node with the same service attributes as the target logical node includes:

[0012] Upon receiving a service recovery instruction from the management node for a target logical node among the faulty physical nodes, the resource configuration information and connection configuration information of the target logical node are determined.

[0013] A new logical node is constructed, and the new logical node is configured according to the resource configuration information and the connection configuration information.

[0014] After receiving a service recovery instruction, the physical node creates a logical node with the same attributes as the target logical node to take over the services of the target logical node.

[0015] In one embodiment, the step of reading the target data to be written from each logical volume contained in the target logical node, and sending the target data to be written to the storage pool, on a logical volume basis, includes:

[0016] The system iterates through each logical volume contained in the target logical volume, reads the target data to be written corresponding to each logical volume in the cache in a preset order, and sends the target data to be written to the storage pool.

[0017] Read the data to be written in the cache order to prevent data errors caused by incorrect writing order.

[0018] In one embodiment, the method further includes:

[0019] After receiving a write data request through a logical node, the data to be written and the logical volume identifier of the logical volume to be written are determined in the write data request.

[0020] The data to be written is cached in the cache space corresponding to the logical volume identifier through the logical node, and the data to be written and the logical volume identifier are synchronized to other physical nodes;

[0021] The system reads the data to be written from the cache space corresponding to the logical volume identifier through the logical node, sends the data to be written to the storage pool, and synchronizes the sent data to be written to the other physical nodes.

[0022] Each time a physical node stores a piece of data, it synchronizes the data to other physical nodes so that, in the event of a failure of the physical node, the other physical nodes can restore the services handled by that physical node based on the synchronized data.

[0023] Secondly, this application also provides a distributed system, the system including a management node and multiple physical nodes, wherein each physical node deploys at least one logical node, and the logical node corresponds to at least one logical volume, wherein:

[0024] The management node is used to determine a target physical node that has backed up the cached data of the failed physical node when it is determined that any of the physical nodes has failed.

[0025] The target physical node is configured to, upon receiving a service recovery instruction from the management node for a target logical node deployed on a faulty physical node, create a new logical node with the same service attributes as the target logical node; read the data to be written from each logical volume synchronized with the target logical node, and send the data to be written to the storage pool, on a logical volume basis; and, upon determining that the data to be written for any target logical volume corresponding to the target logical node has been written, receive data write requests for the target logical volume through the new logical node, thereby completing the service processing for the target logical volume.

[0026] The faulty physical node is configured to, upon receiving a write data request through a logical node, determine the data to be written carried in the write data request and the logical volume identifier of the logical volume to be written; cache the data to be written in the cache space corresponding to the logical volume identifier through the logical node, and synchronize the data to be written and the logical volume identifier to the target physical node; read the data to be written cached in the cache space corresponding to the logical volume identifier through the logical node, send the data to be written to the storage pool, and synchronize the sent data to be written to the target physical node.

[0027] The target physical node is the node for service recovery, and the faulty physical node is the node for which service recovery is needed. Before the faulty physical node fails, it stores each piece of data and then sends that data to the target physical node. After the faulty physical node fails, the target physical node performs service recovery based on the data synchronized from the faulty physical node.

[0028] In one instance, the management node is also used for:

[0029] Obtain the bandwidth and read / write limits configured by the user for each of the logical volumes;

[0030] Based on the correspondence between physical nodes and logical nodes, and the correspondence between logical nodes and logical volumes, the management node determines the read / write pressure of each physical node.

[0031] In the case that the read and write pressure of each physical node is uneven, the overloaded physical nodes with read and write pressure greater than the pressure upper limit and the unloaded physical nodes with read and write pressure less than the pressure lower limit are identified.

[0032] At least one logical volume corresponding to the overloaded physical node is assigned to a logical node deployed on the unloaded physical node.

[0033] The management node performs load balancing on each logical volume based on the pressure of the logical volumes corresponding to each physical node, so that the logical volume resources of the distributed system can be effectively utilized.

[0034] Thirdly, this application also provides a service recovery apparatus. The apparatus is applied to physical nodes in a distributed system, which further includes management nodes and other physical nodes. Each physical node deploys at least one logical node, and each logical node corresponds to at least one logical volume. The apparatus includes:

[0035] The receiving module is configured to create a new logical node with the same service attributes as the target logical node when it receives a service recovery instruction sent by the management node for a target logical node in a faulty physical node;

[0036] The sending module is used to read the target data to be written from each logical volume contained in the target logical node, on a logical volume basis, and send the target data to be written to the storage pool.

[0037] The recovery module is used to receive data write requests for the target logical volume through the new logical node and complete the business processing service for the target logical volume when it is determined that the target data to be written to any target logical volume corresponding to the target logical node has been written.

[0038] In one embodiment, the receiving module includes:

[0039] The information determination unit is used to determine the resource configuration information and connection configuration information of the target logical node when it receives a service recovery instruction sent by the management node for the target logical node in the faulty physical node.

[0040] The construction unit is used to construct a new logical node and configure the new logical node according to the resource configuration information and the connection configuration information.

[0041] In one embodiment, the sending module is specifically used for:

[0042] The system iterates through each logical volume contained in the target logical volume, reads the target data to be written corresponding to each logical volume in the cache in a preset order, and sends the target data to be written to the storage pool.

[0043] In one embodiment, the above-described apparatus further includes:

[0044] The logical volume determination module is used to determine the data to be written and the logical volume identifier of the logical volume to be written, carried by the write data request after receiving a write data request through a logical node.

[0045] The caching module is used to cache the data to be written to the cache space corresponding to the logical volume identifier through logical nodes, and to synchronize the data to be written and the logical volume identifier to other physical nodes;

[0046] The write module is used to read the data to be written cached in the cache space corresponding to the logical volume identifier through the logical node, send the data to be written to the storage pool, and synchronize the sent data to be written to the other physical nodes.

[0047] Fourthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0048] Upon receiving a service recovery instruction from the management node for a target logical node in a faulty physical node, a new logical node with the same service attributes as the target logical node is created.

[0049] Read the target data to be written from each logical volume contained in the target logical node, and send the target data to be written to the storage pool, on a logical volume basis.

[0050] If the target data to be written to any target logical volume corresponding to the target logical node is completed, the new logical node receives the data write request for the target logical volume and completes the business processing service for the target logical volume.

[0051] Fifthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0052] Upon receiving a service recovery instruction from the management node for a target logical node in a faulty physical node, a new logical node with the same service attributes as the target logical node is created.

[0053] Read the target data to be written from each logical volume contained in the target logical node, and send the target data to be written to the storage pool, on a logical volume basis.

[0054] If the target data to be written to any target logical volume corresponding to the target logical node is completed, the new logical node receives the data write request for the target logical volume and completes the business processing service for the target logical volume.

[0055] Sixthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0056] Upon receiving a service recovery instruction from the management node for a target logical node in a faulty physical node, a new logical node with the same service attributes as the target logical node is created.

[0057] Read the target data to be written from each logical volume contained in the target logical node, and send the target data to be written to the storage pool, on a logical volume basis.

[0058] If the target data to be written to any target logical volume corresponding to the target logical node is completed, the new logical node receives the data write request for the target logical volume and completes the business processing service for the target logical volume.

[0059] The aforementioned business service recovery methods, devices, computer equipment, storage media, and computer program products divide logical volumes based on logical nodes and use logical volumes for data recovery. Since the business service is completed first for the logical volumes whose data is being recovered, rather than waiting for the data of all logical volumes to be recovered before restoring the business service, the business interruption time is reduced. Attached Figure Description

[0060] Figure 1 This is an application environment diagram of a service recovery method in one embodiment;

[0061] Figure 2 This is a flowchart illustrating a service recovery method in one embodiment;

[0062] Figure 3A , 3B 3C and 3D are flowcharts illustrating the service recovery method in another embodiment;

[0063] Figure 4 This is a structural block diagram of a service device in one embodiment;

[0064] Figure 5This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0066] In the field of distributed storage, a distributed storage system is generally the front end of a distributed storage system. It is used to receive and process write requests, and then, after processing the write requests, it sends them to the back-end storage pool to complete the disk write operation (writing the data to the disk).

[0067] In related technologies, distributed storage systems typically include multiple physical nodes, each of which includes at least one logical node (LNODE), which is a virtualized storage unit hosted on the physical node. When a physical node fails, such as becoming inaccessible due to network issues or power outages, it is necessary to restore the services handled by the failed physical node. This restoration is generally performed at the physical node level. In the distributed storage system, another target physical node is selected to take over all the services handled by the failed physical node. Only after all data from the failed physical node has been persisted to disk by the target physical node can the services handled by the original failed physical node be restored.

[0068] In related technologies, there are also services recovery methods that use logical nodes as the granularity. In this case, one or more physical nodes need to be selected in the distributed storage system to take over the services that the logical nodes on the failed physical node are handling. For example, if there are three logical nodes running on the failed physical node X, namely logical nodes A, B, and C, physical node Y in the distributed storage system can take over the services that logical nodes A, B, and C are handling, or physical node Y can take over the services that logical node A is handling, and physical node Z can take over the services that logical nodes B and C are handling.

[0069] The above-mentioned service recovery method requires that after the logical nodes deployed on the faulty physical node are migrated to other normal physical nodes, the data that was not written to disk during the failure (i.e., all cached data to be written) needs to be sent to the storage pool before the logical nodes can continue to receive write requests and complete the recovery of the services undertaken by the faulty physical node. Whether the service recovery is performed at the physical node level or at the logical node level, the recovery efficiency is not very high.

[0070] Based on this, this application proposes a service recovery method. This method is applied to physical nodes in a distributed system, where the distributed system also includes a management node and other physical nodes. Each physical node deploys at least one logical node, and each logical node corresponds to at least one logical volume. Upon receiving a service recovery instruction from the management node for a target logical node within a failed physical node, the physical node creates a new logical node with the same service attributes as the target logical node. On a volume-by-volume basis, the target data to be written is read from each logical volume contained in the target logical node, and the target data to be written is sent to the storage pool. When it is determined that the target data to be written for any target logical volume corresponding to the target logical node has been written, the new logical node receives data write requests for the target logical volume, completing the service processing for the target logical volume.

[0071] The method in this application divides logical volumes based on logical nodes and uses logical volumes for data recovery. Since the business services are completed first for the logical volumes whose data is being recovered, instead of waiting for the data of all logical volumes to be recovered before restoring the business services, the business interruption time is reduced.

[0072] It should be noted that the physical node used in the service recovery method of this application is a block device, which stores information in fixed-size blocks. Each block has its own address. Therefore, when caching the data to be written, it is written and read in blocks. As a result, when performing service recovery, the physical node or logical node cannot distinguish the correspondence between the cached data and the service. The data of the synchronized faulty physical node must be written to disk before it can continue to receive the data to be written and cache it.

[0073] The distributed system can be a distributed storage system, a distributed file system, a distributed block storage system, etc., as long as the distributed system includes multiple physical nodes and each physical node includes at least one logical node.

[0074] The service recovery method provided in this application embodiment can be applied to, for example, Figure 1 The application environment shown is as follows. In this environment, each physical node communicates with the management node via a network, and the physical nodes can also communicate with each other via the network. Each physical node has at least one logical node deployed, and each logical node corresponds to at least one logical volume.

[0075] In one embodiment, such as Figure 2 As shown, a business service recovery method is provided, which is applicable to, for example... Figure 1 The physical nodes in the distributed system shown include the following steps:

[0076] Step 201: Upon receiving a service recovery instruction from the management node for the target logical node in the faulty physical node, create a new logical node with the same service attributes as the target logical node.

[0077] Business attributes refer to the attributes corresponding to the business being processed. For example, the corresponding business might be the storage of transaction data, or the corresponding business might be the storage of user operation data. Different business attributes result in different data write request processing logics, such as different data carried in the data write request, or different locations of the data to be written within the data request.

[0078] Specifically, the management node communicates with each physical node periodically. If a response is not received from a physical node after a preset time, the physical node is determined to be a faulty physical node. The management node then identifies the target physical node that stores the data stored on the faulty physical node. Based on the load status of each target physical node, the management node identifies the logical nodes on the faulty physical node and the physical nodes that support each logical node. Finally, the management node sends a service recovery command to the target physical node for the target logical nodes in the faulty physical node. For example, a faulty physical node X has three logical nodes, A, B, and C. The management node determines that physical nodes Y and Z are synchronized with all the data of the faulty physical node. Physical node Y can take over the services previously handled by logical nodes A, B, and C. In this case, the management node will send a service recovery command to physical node Y for logical nodes A, B, and C in the faulty physical node X. Alternatively, physical node Y can take over the services previously handled by logical node A, and also take over the services previously handled by logical nodes B and C. In this case, the management node will send a service recovery command to physical node Y for logical node A in the faulty physical node X, and send a service recovery command to physical node Z for logical nodes B and C in the faulty physical node X.

[0079] When a physical node receives a service recovery instruction from the management node for a target logical node in a faulty physical node, it determines the service attributes of the target logical node and then creates a new logical node with the same service attributes as the target logical node according to the creation requirements of those service attributes.

[0080] Step 203: Read the target data to be written from each logical volume contained in the target logical node, and send the target data to be written to the storage pool.

[0081] The target data to be written in a logical volume refers to the data to be written that will eventually be written to the disk storage space corresponding to that logical volume.

[0082] Specifically, when a logical node caches data to be written, it does so on a logical volume basis. Therefore, when other physical nodes synchronize the data to be written from this logical node, they also cache it on a logical volume basis. Thus, a physical node reads the target data to be written from each logical volume contained in the target logical node, and then sends the target data to the storage pool, on a logical volume basis. For example, if the target logical node contains three logical volumes, A, B, and C, it first reads the target data to be written stored in logical volume A, then sequentially sends the target data to be written stored in logical volume A to the storage pool, then reads the target data to be written stored in logical volume B and sends it to the storage pool, and finally reads the target data to be written stored in logical volume C and sends it to the storage pool.

[0083] Step 205: If the target data to be written to any target logical volume corresponding to the target logical node has been written, the new logical node receives the data write request for the target logical volume and completes the business processing service for the target logical volume.

[0084] Specifically, if a physical node determines that the target data to be written to any target logical volume corresponding to the target logical node has been written, for example, if it determines that the cache space corresponding to a certain logical volume is empty, then it determines that the target data to be written to that logical volume has been written. Alternatively, it may receive a response from a thread corresponding to a certain logical volume. This indicates that the failed physical node failed to write the data to disk for the target logical volume corresponding to the target logical node. The physical node can then receive data write requests for the target logical volume through a new logical node and complete the business processing service for the target logical volume.

[0085] In this embodiment, logical volumes are partitioned based on logical nodes, and data recovery is performed using logical volumes. Since the business services are completed first for the logical volumes whose data is being recovered, instead of waiting for the data of all logical volumes to be recovered before restoring the business services, the business interruption time is reduced.

[0086] In one embodiment, step 201 specifically includes:

[0087] Step 201A: Upon receiving a service recovery instruction from the management node for the target logical node in the faulty physical node, determine the resource configuration information and connection configuration information of the target logical node.

[0088] Different logical nodes require different resources and connection configurations.

[0089] Specifically, after identifying the target physical node from among the failed physical nodes, the management node queries the pre-stored resource configuration and connection configuration information of the target logical node based on its logical node identifier. Simultaneously with sending the service recovery command for the target logical node from among the failed physical nodes to the target physical node, the management node also sends the resource configuration and connection configuration information of the target logical node to the target physical node.

[0090] Step 201B: Construct a new logical node and configure it according to the resource configuration information and connection configuration information.

[0091] Specifically, after receiving the service recovery instruction sent by the management node for the target logical node in the faulty physical node, the physical node determines the resource configuration information and connection configuration information of the target logical node, constructs a new logical node, and then allocates resources for the new logical node according to the resource configuration information, such as memory, CPU resources, storage space, bandwidth, etc., to complete the resource configuration of the new logical node. Then, according to the connection configuration information, it configures the connection with the client, management node, etc., to complete the configuration processing of the new logical node.

[0092] In this embodiment, after receiving the service recovery instruction, the physical node creates a logical node with the same attributes as the target logical node to take over the service of the target logical node.

[0093] In one embodiment, step 203 specifically includes:

[0094] Traverse each logical volume contained in the target logical volume, read the target data to be written corresponding to each logical volume in the cache in a preset order, and send the target data to be written to the storage pool.

[0095] Specifically, the physical node determines the identifiers of each logical volume corresponding to the target logical node based on the node identifier of the target logical node. Then, it traverses each logical volume contained in the target logical volume, reads the target data to be written corresponding to each cached logical volume in a preset order, and sends the read target data to be written to the storage pool. For each logical volume, according to the order of the target data to be written cached for that logical volume, the target data to be written is read from the cache space corresponding to that logical volume, and the read target data to be written is sent to the storage pool.

[0096] In one embodiment, after the physical node determines the identifiers of each logical volume corresponding to the target logical node based on the node identifier of the target logical node, it starts multiple service recovery threads based on the number of logical volumes corresponding to the target logical node. Each service recovery thread is used to read the target data to be written corresponding to a logical volume and send the read target data to be written to the storage pool. After any service recovery thread sends all the target data to be written of its corresponding logical volume to the storage pool, it sends feedback to the physical node that the logical volume has been written.

[0097] In this embodiment, the data to be written is read in the cache order to prevent data errors caused by incorrect writing order.

[0098] In one embodiment, the above method further includes:

[0099] Step 207: After receiving the write data request through the logical node, determine the data to be written and the logical volume identifier of the logical volume to be written carried in the write data request.

[0100] Specifically, the physical node receives the write data request through the logical node, parses the write data request to obtain the data to be written carried by the write data request, and then determines the logical volume identifier of the logical volume to be written according to the pre-configured settings, such as the correspondence between the data type of the write request and the logical volume, or the correspondence between the network address of the write request and the logical volume.

[0101] Step 209: Cache the data to be written to the cache space corresponding to the logical volume identifier through the logical node, and synchronize the data to be written and the logical volume identifier to other physical nodes.

[0102] Specifically, logical nodes pre-allocate different cache spaces for different logical volumes. Physical nodes cache the data to be written to the cache space corresponding to the specific logical volume identifier through the logical nodes. At the same time as caching, the data to be written and the logical volume identifier are synchronized to other physical nodes so that other physical nodes can back up the target data to be written for each logical volume cached by the physical node.

[0103] Step 211: Read the target data to be written from the cache space corresponding to the logical volume identifier through the logical node, send the target data to be written to the storage pool, and synchronize the sent target data to be written to other physical nodes.

[0104] Specifically, physical nodes read the target data to be written from the cache space corresponding to the logical volume identifier through logical nodes, then send the target data to be written to the storage pool, and synchronize the sent target data to be written to other physical nodes, so that other physical nodes can delete the sent target data to be written from the synchronized data, thus maintaining the consistency of synchronized data between physical nodes.

[0105] Steps 209 and 211 can be two independent threads. In step 209, the physical node caches each piece of data to be written to the cache space corresponding to each logical volume identifier through one thread. In step 211, the physical node reads the target data to be written cached in the cache space corresponding to each logical volume identifier through one thread, or it can read the target data to be written cached in the cache space corresponding to each logical volume identifier through multiple threads.

[0106] In this embodiment, each time a physical node stores a piece of data, it synchronizes the data to other physical nodes so that other physical nodes can restore the services undertaken by the physical node based on the synchronized data after the physical node fails.

[0107] In one embodiment, this application also provides a distributed system, wherein the system includes a management node and multiple physical nodes, wherein at least one physical node is deployed on each physical node, and the logical node corresponds to at least one logical volume.

[0108] In this process, if the management node determines that any physical node has failed, it will identify the target physical node that has backed up the cached data of the failed physical node.

[0109] Specifically, the management node stores the backup relationships between various physical nodes. When a physical node fails, the management node identifies the target physical node that has backed up the cached data of the failed physical node from among the other unfailed physical nodes. Then, based on the load status of each unfailed physical node and the logical nodes in the failed physical node, the management node determines the target physical node to take over each logical node in the failed physical node and sends a service recovery instruction for the target logical node deployed on the failed physical node to the target physical node.

[0110] Upon receiving a service recovery instruction from the management node for a target logical node deployed on a faulty physical node, each target physical node creates a new logical node with the same service attributes as the target logical node. Then, on a volume-by-volume basis, it reads the data to be written from each logical volume synchronized with the target logical node and sends the data to be written to the storage pool. Once it is determined that the data to be written for any target logical volume corresponding to the target logical node has been written, the new logical node receives the data write request for the target logical volume and completes the service processing for the target logical volume.

[0111] Before a failure occurs, a faulty physical node, upon receiving a write data request through a logical node, determines the data to be written and the logical volume identifier of the logical volume to be written, carried in the write data request. The logical node caches the data to be written in the cache space corresponding to the logical volume identifier and synchronizes the data to be written and the logical volume identifier to the target physical node. The logical node then reads the data to be written cached in the cache space corresponding to the logical volume identifier, sends the data to be written to the storage pool, and synchronizes the sent data to the target physical node.

[0112] In this embodiment, the target physical node is the node that performs service recovery, and the faulty physical node is the node that needs service recovery. Before the faulty physical node fails, it stores each piece of data and then stores the data in the target physical node. After the faulty physical node fails, the target physical node performs service recovery based on the data synchronized by the faulty physical node.

[0113] In one embodiment, the management node of the above system is further used for:

[0114] The management node obtains the bandwidth and read / write limits configured by the user for each logical volume. Then, based on the correspondence between physical nodes and logical nodes, and between logical nodes and logical volumes, the management node determines the read / write pressure on each physical node. If the read / write pressure is uneven across physical nodes, the management node identifies overloaded physical nodes with read / write pressure exceeding the upper limit and idle physical nodes with read / write pressure below the lower limit. The management node then allocates at least one logical volume corresponding to an overloaded physical node to a logical node deployed on an idle physical node.

[0115] In this distributed storage system, each logical volume is pre-configured by the user, including bandwidth and read / write limits. Specifically, the management node reads the user's configuration for each logical volume to obtain the bandwidth and read / write limits for each logical volume.

[0116] In one embodiment, when the read / write pressure of the various physical nodes is uneven, the management node identifies overloaded physical nodes with read / write pressure exceeding the upper limit and idle physical nodes with read / write pressure below the lower limit. It then sends migration instructions for the target logical volume to the overloaded physical nodes. Upon receiving the migration instruction, the overloaded physical node stops receiving write data requests for the target logical volume and sends the cached target data to be written to the storage pool, completing the disk write operation for the target logical volume. Upon receiving the migration instruction from the management node, the idle physical node constructs a logical node corresponding to the target logical volume and begins receiving write data requests for the target logical volume.

[0117] In this embodiment, the management node performs load balancing on each logical volume based on the pressure of the logical volume corresponding to each physical node, so that the logical volume resources of the distributed system can be effectively utilized.

[0118] like Figures 3A-3D The diagram shown is a flowchart illustrating a service recovery method according to a specific embodiment of this application. It assumes that the distributed storage nodes include physical nodes A, B, and C, logical nodes 1, 2, and 3, and logical volumes 1, 2, 3, and 4. The clients in the diagram represent the sum of all clients sending requests; multiple clients are illustrated as a single entity within the diagram. Figure 3A This is a deployment diagram of all physical nodes, logical nodes, and logical volumes before physical node A fails. Figure 3B After a failure of physical node A, the services corresponding to logical volume 1 are restored on physical node B; Figure 3C After a failure of physical node A, the services corresponding to logical volume 2 are restored on physical node B, meaning that all services undertaken by physical node A are restored on physical node B. Figure 3D This diagram illustrates the deployment of physical nodes, logical nodes, and logical volumes after physical node B takes over all the services of physical node A and the distributed storage system performs load balancing.

[0119] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0120] Based on the same inventive concept, this application also provides a business service recovery apparatus for implementing the business service recovery method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more business service recovery apparatus embodiments provided below can be found in the limitations of the business service recovery method described above, and will not be repeated here.

[0121] In one embodiment, such as Figure 4 As shown, a business service recovery device is provided. The device is applied to physical nodes in a distributed system. The distributed system also includes management nodes and other physical nodes. Each physical node deploys at least one logical node, and each logical node corresponds to at least one logical volume. The device includes:

[0122] The receiving module 401 is configured to create a new logical node with the same service attributes as the target logical node when it receives a service recovery instruction sent by the management node for a target logical node in a faulty physical node;

[0123] The sending module 403 is used to read the target data to be written from each logical volume contained in the target logical node, and send the target data to be written to the storage pool, on a unit of logical volume.

[0124] The recovery module 405 is used to receive a data write request for the target logical volume through the new logical node when it is determined that the target data to be written for any target logical volume corresponding to the target logical node has been written, and to complete the business processing service for the target logical volume.

[0125] In one embodiment, the receiving module includes:

[0126] Information determination unit 401A (not shown in the figure) is used to determine the resource configuration information and connection configuration information of the target logical node when it receives a service recovery instruction sent by the management node for the target logical node in the faulty physical node.

[0127] The construction unit 401B (not shown in the figure) is used to construct a new logical node and to configure the new logical node according to the resource configuration information and the connection configuration information.

[0128] In one embodiment, the sending module 403 is specifically used for:

[0129] The system iterates through each logical volume contained in the target logical volume, reads the target data to be written corresponding to each logical volume in the cache in a preset order, and sends the target data to be written to the storage pool.

[0130] In one embodiment, the above-described apparatus further includes:

[0131] The logical volume determination module 407 (not shown in the figure) is used to determine the data to be written and the logical volume identifier of the logical volume to be written carried by the write data request after receiving the write data request through the logical node.

[0132] The caching module 409 (not shown in the figure) is used to cache the data to be written to the cache space corresponding to the logical volume identifier through the logical node, and to synchronize the data to be written and the logical volume identifier to other physical nodes;

[0133] The write module 411 (not shown in the figure) is used to read the data to be written cached in the cache space corresponding to the logical volume identifier through the logical node, send the data to be written to the storage pool, and synchronize the sent data to be written to the other physical nodes.

[0134] Each module in the aforementioned service recovery device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can invoke and execute the operations corresponding to each module.

[0135] In one embodiment, a computer device is provided, which may be a physical node in a distributed storage system, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data to be written. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a business service recovery method.

[0136] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0137] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described embodiments of the business service recovery methods.

[0138] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described embodiments of the service recovery methods.

[0139] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described embodiments of the business service recovery methods.

[0140] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0141] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0142] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0143] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A service restoration method, characterized by, The method is applied to a target physical node in a distributed system, which also includes a management node and other physical nodes. Each physical node deploys at least one logical node, and the logical node corresponds to at least one logical volume. The method includes: Upon receiving a service recovery instruction sent by the management node for a target logical node in a faulty physical node, a new logical node with the same service attributes as the target logical node is created. The target physical node is the successor physical node determined by the management node from other physical nodes that have synchronized cached data of the faulty physical node, based on the load status of each of the other physical nodes. Based on the logical volume identifier of each logical volume corresponding to the target logical node, the target data to be written for each logical volume is obtained from the cache data of the target physical node, and the target data to be written is sent to the storage pool for disk writing. If the target data to be written to any target logical volume corresponding to the target logical node is completed and written to disk, the new logical node receives the data write request for the target logical volume and completes the business processing service for the target logical volume.

2. The method according to claim 1, characterized in that, Upon receiving a service recovery instruction from the management node for a target logical node among the faulty physical nodes, the step of creating a new logical node with the same service attributes as the target logical node includes: Upon receiving a service recovery instruction from the management node for a target logical node among the faulty physical nodes, the resource configuration information and connection configuration information of the target logical node are determined. A new logical node is constructed, and the new logical node is configured according to the resource configuration information and the connection configuration information.

3. The method according to claim 1, characterized in that, The step of retrieving target data to be written for each logical volume from the cached data of the target physical node based on the logical volume identifier of each logical volume corresponding to the target logical node, and sending the target data to be written to the storage pool for disk persistence, includes: Traverse each logical volume contained in the target logical node, and in a preset order, obtain the target data to be written corresponding to each logical volume from the cache data of the target physical node based on the logical volume identifier of each logical volume corresponding to the target logical node, and send the target data to be written to the storage pool for disk writing.

4. The method according to claim 1, characterized in that, The method further includes: If the physical node does not fail, after receiving a write data request through the logical node, the data to be written and the logical volume identifier of the logical volume to be written are determined in the write data request. The data to be written is cached in the cache space corresponding to the logical volume identifier through the logical node, and the data to be written and the logical volume identifier are synchronized to other physical nodes; The system reads the data to be written from the cache space corresponding to the logical volume identifier through the logical node, sends the data to be written to the storage pool, and synchronizes the sent data to be written to the other physical nodes.

5. A distributed system, characterized in that, The system includes a management node and multiple physical nodes. Each physical node deploys at least one logical node, and each logical node corresponds to at least one logical volume. The management node is used to determine a target physical node that has backed up the cached data of the failed physical node when it is determined that any of the physical nodes has failed. The target physical node is used to create a new logical node with the same service attributes as the target logical node when it receives a service recovery instruction from the management node for a target logical node deployed on a faulty physical node; based on the logical volume identifier of each logical volume corresponding to the target logical node, it retrieves the target data to be written for each logical volume from the cached data of the target physical node, and sends the target data to be written to the storage pool for disk write; when it is determined that the data to be written for any target logical volume corresponding to the target logical node has been written to disk, it receives data write requests for the target logical volume through the new logical node to complete the service processing for the target logical volume; The faulty physical node, in the absence of a fault, upon receiving a write data request through a logical node, determines the data to be written and the logical volume identifier of the logical volume to be written carried in the write data request; caches the data to be written in the cache space corresponding to the logical volume identifier through the logical node, and synchronizes the data to be written and the logical volume identifier to other physical nodes; reads the data to be written cached in the cache space corresponding to the logical volume identifier through the logical node, sends the data to be written to the storage pool for disk persistence, and synchronizes the sent data to be written to the other physical nodes.

6. The system according to claim 5, characterized in that, The management node is also used for: Obtain the bandwidth and read / write limits configured by the user for each of the logical volumes; Based on the correspondence between physical nodes and logical nodes, and the correspondence between logical nodes and logical volumes, the read / write pressure of each physical node is determined. In the case that the read and write pressure of each physical node is uneven, the overloaded physical nodes with read and write pressure greater than the pressure upper limit and the unloaded physical nodes with read and write pressure less than the pressure lower limit are identified. At least one logical volume corresponding to the overloaded physical node is assigned to a logical node deployed on the unloaded physical node.

7. A business service restoration device, characterized in that, The apparatus is applied to a target physical node in a distributed system, which also includes a management node and other physical nodes. Each physical node is deployed with at least one logical node, and the logical node corresponds to at least one logical volume. The apparatus includes: The receiving module is used to create a new logical node with the same service attributes as the target logical node when it receives a service recovery instruction sent by the management node for the target logical node in the faulty physical node. The target physical node is the receiving physical node determined by the management node from other physical nodes that have synchronized cached data of the faulty physical node, based on the load status of each of the other physical nodes. The sending module is used to obtain the target data to be written corresponding to each logical volume from the cache data of the target physical node based on the logical volume identifier of each logical volume corresponding to the target logical node, and send the target data to be written to the storage pool for disk writing; The recovery module is used to receive data write requests for the target logical volume through the new logical node and complete the business processing service for the target logical volume when it is determined that the target data to be written to any target logical volume corresponding to the target logical node has been written to disk.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images