Data writing method and electronic equipment

Abstract

The invention discloses a data writing method and electronic equipment, and relates to the technical field of storage, the method comprises the steps that after writing data sent by a host is received, first detection information is sent to a second node, and the writing data is the writing data sent to a first node after the host determines that the first node joins a cluster; receiving a first detection result sent by the second node; when it is determined that the first detection result is that the second node does not write all data in the cache of the second node into the rear-end hard disk, whether the node state of the second node is an online state or not is determined; and when the node state of the second node is determined to be an online state, forwarding the write data to the second node, so that the second node writes the write data into a rear-end hard disk. According to the method and the device, the problem that the redundancy capability and the anti-fault capability of the cluster are poor due to the fact that the node leaving the cluster cannot join the cluster as soon as possible after recovery and cannot perform data processing is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of storage, in particular to a data writing method and an electronic device. BACKGROUND

[0002] With the development of big data, cloud computing and storage technology, the requirement for data processing speed is also getting higher and higher. For a cluster of two nodes, the scenario of a single node leaving the cluster often occurs, which makes only the single node in the cluster process data, and there is a risk of failure, which makes the risk redundancy performance poor. Therefore, the node that leaves the cluster needs to be able to quickly join the cluster again after recovery, to ensure the reliability of the cluster and improve the risk resistance of the cluster.

[0003] In the related art, when the single node that leaves the cluster joins the cluster again, the data cached in the single node that leaves the cluster is old data compared with the data cached in the node that has been in the cluster all the time. If the host reads the old data from the single node that leaves the cluster, there will be a data inconsistency problem. Therefore, the single node that leaves the cluster needs to clear the old data in the cache before joining the cluster, and the node that has been in the cluster all the time needs to flush all the data in the cache to the back-end hard disk, so as to keep the data in the caches of the two nodes consistent, and at this time the single node that leaves the cluster can join the cluster. In this way, it takes several hours or even longer for the node that has been in the cluster all the time to flush all the data in the cache to the back-end hard disk, which makes the node that leaves the cluster unable to join the cluster as soon as possible after recovery, and also unable to process data, resulting in poor redundancy and fault tolerance of the cluster. SUMMARY

[0004] The present application provides a data writing method to at least solve the problem that the node that leaves the cluster cannot join the cluster as soon as possible after recovery, and also cannot process data, resulting in poor redundancy and fault tolerance of the cluster in the related art.

[0005] The present application provides a data writing method, which is applied to the scenario that a first node joins a cluster again after clearing the data in the cache of the first node, the cluster is composed of the first node and a second node, the method is performed by the first node, and includes: after receiving write data sent by a host, sending first detection information to the second node, the write data being write data sent by the host to the first node after determining that the first node joins the cluster; receiving first detection results sent by the second node, the first detection results being detection results of whether the second node writes all the data in the cache of the second node to a back-end hard disk based on the first detection information; when it is determined that the first detection results are that the second node does not write all the data in the cache of the second node to the back-end hard disk, determining whether a node state of the second node is an online state; when it is determined that the node state of the second node is the online state, forwarding the write data to the second node, so that the second node writes the write data to the back-end hard disk.

[0006] The application provides another data writing method, which is applied to a scenario that a first node empties data in a cache of the first node and then joins a cluster again, the cluster being composed of the first node and a second node, and the method comprising: receiving first detection information sent by the first node, the first detection information being detection information generated by the first node after receiving first type write data sent by a host, the first type write data being write data sent by the host to the first node after determining that the first node joins the cluster; based on the first detection information, detecting whether to write all data in a cache of the second node to a back-end hard disk, obtaining a first detection result, and sending the first detection result to the first node, so that the first node determines that the first detection result is that the second node does not write all data in the cache of the second node to the back-end hard disk, and determines that a node state of the second node is an online state, and forwards the first type write data to the second node; and after receiving the first type write data forwarded by the first node, writing the first type write data to the back-end hard disk.

[0007] The application also provides a data writing device, which is applied to a first node; the data writing device comprising: a first transceiving module, configured to send first detection information to a second node after receiving write data sent by a host, the write data being write data sent by the host to the first node after determining that the first node joins a cluster; and receive first detection result sent by the second node, the first detection result being a detection result of the second node based on the first detection information, detecting whether to write all data in a cache of the second node to a back-end hard disk.

[0008] A first processing module, configured to determine whether a node state of the second node is an online state when it is determined that the first detection result is that the second node does not write all data in the cache of the second node to the back-end hard disk; and forward the write data to the second node when it is determined that the node state of the second node is the online state, so that the second node writes the write data to the back-end hard disk.

[0009] The application also provides another data writing device, which is applied to a second node; the data writing device comprising: A second transceiving module, configured to receive first detection information sent by the first node, the first detection information being detection information generated by the first node after receiving first type write data sent by a host, the first type write data being write data sent by the host to the first node after determining that the first node joins a cluster.

[0010] A second processing module, configured to, based on the first detection information, detect whether to write all data in a cache of the second node to a back-end hard disk, obtain a first detection result, and send the first detection result to the first node, so that the first node determines that the first detection result is that the second node does not write all data in the cache of the second node to the back-end hard disk, and determines that a node state of the second node is an online state, and forwards the first type write data to the second node.

[0011] a write module, configured to write the first type of write data into the backend hard disk when the first type of write data forwarded by the first node is received.

[0012] The application further provides an electronic device, comprising a memory configured to store a computer program, and a processor configured to execute the computer program to implement the steps of any of the data write methods.

[0013] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of any of the data write methods.

[0014] The application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of any of the data write methods.

[0015] According to the application, the first node can directly join the cluster again after the data in the cache of the first node is emptied, without waiting for the second node to write all the data in the cache of the second node into the backend hard disk. Therefore, the speed of the first node joining the cluster can be improved, the cluster integration time during node restart, fault recovery or expansion is greatly shortened, and the service capability loss time caused by node offline can be significantly reduced. In addition, the second node does not write all the data in the cache of the second node into the backend hard disk, that is, the first node has joined the cluster, but the data of the first node and the second node are not consistent, and the first node cannot directly process data and write data into the backend hard disk. At this time, when the first node receives the write data sent by the host, the first node can temporarily forward the write data to the second node for processing. In this way, even if the data of the first node and the second node are not consistent after the first node joins the cluster, the first node can still receive and process the write request of the host through the forwarding mechanism, avoid service interruption caused by node data misalignment, and ensure the continuity and availability of the business layer data write operation. Even if the first node is in a data inconsistent state when the first node joins the cluster, the first node can still participate in the cluster work through the forwarding mechanism, avoiding the risk of write request loss when the second node is overloaded or fails, and indirectly improving the redundancy and fault tolerance of the cluster. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0017] Figure 1 A topology structure diagram of a data write system provided by the embodiments of the application is shown in FIG. 1. Figure 2 A flowchart of a data writing method provided by an embodiment of the present application is shown in FIG. 1. Figure 3 A method for determining the node state of a second node provided by an embodiment of the present application is shown in FIG. 2. Figure 4 A flowchart of another data writing method provided by an embodiment of the present application is shown in FIG. 3. Figure 5 A flowchart of another data writing method provided by an embodiment of the present application is shown in FIG. 4. Figure 6 A device structure block diagram of a data writing device provided by an embodiment of the present application is shown in FIG. 5. Figure 7 A device structure block diagram of another data writing device provided by an embodiment of the present application is shown in FIG. 6. Figure 8 A hardware structure diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 7. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, any other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0019] It should be noted that, in the description of the present application, the terms “comprise”, “contain” or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. The terms “first”, “second” and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0020] In order for those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0021] The embodiments of the present application are applied to the node leaving the cluster rejoining the cluster.

[0022] In the related art, the node leaving the cluster includes the following scenarios. Scenario one: when a node needs to be repaired due to hardware failure, the node needs to enter the service mode, so that the node will leave the cluster, and at this time, the cluster only has a single node working. After the fault is repaired, the fault node is added to the cluster. Taking a cluster including node A and node B as an example, in order to avoid interruption of customer service during version upgrade, the nodes are upgraded one by one. After node A is successfully upgraded, node B is upgraded. When node A is upgraded, node A needs to leave first. If the cluster includes node A and node B, the cluster needs to be opened. After the upgrade is completed, node A is added to the cluster. Node B is the same. If a node has an unpredictable software problem, the node will restart and the single node will leave the cluster. After the fault is automatically or manually recovered, the node will join the cluster.

[0023] In the related art, when the node leaving the cluster joins the cluster, all the data in the cache of the node leaving the cluster needs to be cleared. After the data in the cache of the node always in the cluster is written to the back-end hard disk, the node leaving the cluster is added to the cluster. Taking a cluster including node A and node B as an example, node B leaves the cluster due to some reason. At this time, only node A provides services to users in the cluster. The data in the cache of node A is gradually updated to the latest data. When node B is recovered and needs to join the cluster, the data in the cache of node B is old data compared with the data in the cache of node A. The data inconsistency problem caused by the host reading data from the cache of node B needs to be cleared before the cache of node B joins the cluster. After the data in the cache of node A is written, node B can join the cluster. The reason is that node B has cleared all the data, and the data in the caches of the two nodes is inconsistent. Therefore, the data in the cache of node A needs to be written to the hard disk to ensure that the data in the caches of the two nodes is consistent. If the processing speed of the back-end hard disk is slow, it may take several hours or even longer to write the data. Even if the back-end hard disk is an SSD disk, it also needs a certain amount of time to write the dirty data. If the node leaving the cluster does not join the cluster, it cannot provide services to customers, and the single node needs to process user data, which is slower than the dual nodes. The fault recovery time is too long, and the user experience is poor.

[0024] To solve the above technical problems, an embodiment of the present application provides a data writing method. The method can join the cluster immediately without waiting for the node always in the cluster to write the data in the cache to the back-end hard disk before the first node leaving the cluster joins the cluster. The data sent by the host to the first node is forwarded to the second node for processing before the second node writes all the data in the cache to the hard disk. At the same time, the data sent by the host to the first node can be written to the back-end hard disk, so that the first node can process data and ensure the rapid recovery of the cluster.

[0025] The following is an exampleFigure 1 The method provided in this application embodiment is described using the data writing system shown as an example.

[0026] like Figure 1 As shown, Figure 1 This is a topology diagram of a data writing system provided in an embodiment of this application. Figure 1 In this system, the data writing system 100 includes a first node 101, a second node 102, a host 103, and a backend hard disk 104. The first node 101 and the second node 102 form a cluster.

[0027] The first node 101 can be a node that has left and rejoined the cluster. The first node 101 includes a first forwarding module and a first caching module. The first forwarding module is used to forward data or information to the second node. The first caching module is used to cache data sent from the host to the first node.

[0028] The second node 102 can be a node that is always online in the cluster. The second node 102 includes a second forwarding module and a second caching module. The second forwarding module is used to forward data or information to the first node. The second caching module is used to cache data sent from the host to the second node.

[0029] Host 103 can be any device with communication and computing capabilities. For example, host 103 can be a server or a cloud server.

[0030] The back-end hard drive 104 can be any type of hard drive or hard drive group. The back-end hard drive 104 can be a solid-state drive or a hard disk drive.

[0031] Figure 1 The data writing system illustrated is for illustrative purposes only and is not intended to limit the technical solutions of this application. Those skilled in the art should understand that, in specific implementations, the data writing system may include more nodes, without limitation.

[0032] The embodiments of this application provide a data writing method, applied to a scenario where a first node rejoins the cluster after clearing the data in its cache; applied to Figure 1 The first node shown is as follows: Figure 2 As shown, Figure 2 This is a flowchart illustrating a data writing method provided in an embodiment of this application. The data writing method includes the following steps: S201, after receiving the write data sent by the host, sends the first detection information to the second node.

[0033] Among them, write data refers to the write data sent by the host to the first node after determining that the first node has joined the cluster.

[0034] The first detection information is used to indicate whether the data in the cache of the second node is all written to the backend hard disk.

[0035] For example, the host sends write data to the first node. After receiving the write data sent by the host, the first node sends first detection information to the second node.

[0036] S202, receiving the first detection result sent by the second node.

[0037] The first detection result is a detection result of whether the data in the cache of the second node is all written to the backend hard disk based on the first detection information.

[0038] For example, the second node detects whether the data in the cache of the second node is all written to the backend hard disk based on the first detection information, obtains the first detection result, and sends the first detection result to the first node. The first node receives the first detection result sent by the second node.

[0039] Optionally, when the first detection result is that the data in the cache of the second node is all written to the backend hard disk, the first node determines that the first node is normal and writes the write data to the backend hard disk.

[0040] S203, when the first detection result is that the data in the cache of the second node is not all written to the backend hard disk, determining whether the node state of the second node is an online state.

[0041] The node state includes an online state and an offline state.

[0042] For example, when the first detection result is that the data in the cache of the second node is not all written to the backend hard disk, the first node sends online detection information to the second node. The second node receives the online detection information, detects whether the node state of the second node is an online state, and sends the detected node state to the first node. The first node receives the node state and determines whether the node state of the second node is an online state.

[0043] S204, when the node state of the second node is an online state, forwarding the write data to the second node, so that the second node writes the write data to the backend hard disk.

[0044] For example, when the node state of the second node is an online state, the first node forwards the write data to the second node, so that the second node writes the write data to the backend hard disk.

[0045] Based on Figure 2The method shown, after the first node empties the data in the cache of the first node and re-joins the cluster, after receiving the write data sent by the host, sends first detection information to the second node; receives the first detection result sent by the second node; when it is determined that the first detection result is that the second node has not written all the data in its cache to the backend hard disk, it determines whether the node state of the second node is an online state; when it is determined that the node state of the second node is an online state, it forwards the write data to the second node, for the second node to write the write data to the backend hard disk.

[0046] In the method, after the first node empties the data in the cache of the first node, the first node can directly re-join the cluster without waiting for the second node to write all the data in its cache to the backend hard disk. Therefore, the speed of the first node joining the cluster can be improved, the cluster integration time during node restart, fault recovery or expansion is greatly shortened, and the service capability loss time caused by node offline can be significantly reduced. Since the second node has not written all the data in its cache to the backend hard disk, the first node has not been able to keep data consistent with the second node after joining the cluster, and cannot directly process data and write data to the backend hard disk. At this time, when the first node receives the write data sent by the host, it can temporarily forward the write data to the second node for processing. In this way, after the first node joins the cluster, even if there is a temporary inconsistency in data synchronization with the second node, the first node can still receive and process the write request of the host through the forwarding mechanism, avoid service interruption caused by node data misalignment, and ensure the continuity and availability of business layer data write operation. Even if the first node is in a data inconsistency state after joining the cluster, it can still participate in cluster work through the forwarding mechanism, avoiding the risk of write request loss when the second node is overloaded or fails, and indirectly improving the redundancy and fault tolerance of the cluster.

[0047] In an optional example, based on the foregoing embodiment, when it is determined that the first detection result is that the second node has not written all the data in its cache to the backend hard disk, it is determined whether the node state of the second node is an online state, as follows: Figure 3 As shown, Figure 3 The method for determining the node state of the second node provided in the embodiments of the present application includes: S301, periodically obtaining first bitmap information sent by the second node.

[0048] The first bitmap information is used to indicate the node state of the second node. The first bitmap information can be 0x10 or 0x00. 0x10 indicates that the node state of the second node is an online state. 0x00 indicates that the node state of the second node is an offline state.

[0049] For example, the second node periodically detects a node state of the second node, generates first bitmap information based on the node state of the second node, and sends the first bitmap information to the first node; and the first node periodically acquires the first bitmap information sent by the second node.

[0050] In S302, if the first bitmap information is preset bitmap information, it is determined that the node state of the second node corresponding to the first bitmap information is in an online state.

[0051] The preset bitmap information is used to indicate that the node state of the second node is in the online state. For example, the preset bitmap information can be 0x10.

[0052] For example, if the first bitmap information is the preset bitmap information, the first node determines that the node state of the second node corresponding to the first bitmap information is in the online state.

[0053] In S303, or, if the first bitmap information is non-preset bitmap information, it is determined that the node state of the second node corresponding to the first bitmap information is in an offline state.

[0054] The non-preset bitmap information is used to indicate that the node state of the second node is in the offline state. For example, the non-preset bitmap information can be 0x00.

[0055] For example, if the first bitmap information is the non-preset bitmap information, the first node determines that the node state of the second node corresponding to the first bitmap information is in the offline state.

[0056] It can be understood that the first node also periodically detects a node state of the first node, generates second bitmap information based on the node state of the first node, and sends the second bitmap information to the first node; and the second node periodically acquires the second bitmap information sent by the first node. This interaction mode can enable the two nodes to quickly find abnormalities by periodically receiving bitmap information of the other node, and if the bitmap information of the other node is not received in a period, it can be immediately determined that the other node has a problem.

[0057] Further, in an optional example, on the basis of the foregoing embodiment, as introduced above, when it is determined that the node state of the second node is in the online state, after the first node forwards the write data to the second node, the first node can further send second detection information to the second node; and when the second detection result indicates that all the data in the cache of the second node is written to the backend hard disk, the first node stops sending the latest acquired write data to the second node.

[0058] The second detection information is used to instruct the second node to detect again whether all the data in the cache of the second node is written to the backend hard disk, obtain a second detection result, and send the second detection result to the first node.

[0059] It can be understood that the first node detects the cache flush of the second node again, which can timely stop sending new data to the second node, so as to avoid that the second node still continuously receives new data after writing cache data to the hard disk, which increases the processing pressure of the second node and prolongs the data landing time.

[0060] In an optional example, on the basis of the foregoing embodiment, when it is determined that the node state of the second node is the online state, before the first node forwards the write data to the second node, the first node can further perform the following method steps, as introduced in the foregoing, and specifically refer to Figure 4 as shown, Figure 4 A flowchart of another data writing method provided by the embodiment of the application is shown in the figure, which includes the following steps: S401, obtaining the current cache utilization rate, the current input / output rate, the maximum input / output rate supported by the second node, and the weight value corresponding to the cache utilization rate of the second node at the current moment.

[0061] The weight value is used to indicate the proportion of the importance of the cache utilization rate in the load evaluation.

[0062] S402, determining the load index of the second node according to the current cache utilization rate, the current input / output rate, the maximum input / output rate, and the weight value.

[0063] In an example, the first node determines the load index of the second node according to the current cache utilization rate, the current input / output rate, the maximum input / output rate, and the weight value, and calculates through the following expression:

[0064] wherein, represents the load index of the second node; represents the weight value; represents the current cache utilization rate; represents the current input / output rate; represents the maximum input / output rate.

[0065] It can be understood that the load index comprehensively considers the cache utilization rate (small influence when the weight is low) and the input / output rate, and is closer to the real load capacity of the second node. If it is monitored that the second node frequently loses data due to cache overflow, the weight of the cache utilization rate can be temporarily increased, so that the load index more strictly limits the forwarding of data to the node; if the node frequently stalls due to input / output bottleneck, the weight of the input / output rate related indicators can be increased, and the data forwarding is preferentially controlled through the input / output load. Further, the write data of the first node can be written into the backend hard disk as soon as possible, so as to guarantee the overall performance of the cluster.

[0066] S403, if the load index is less than or equal to a preset threshold, it is determined that the write data is forwarded to the second node.

[0067] The preset threshold value can be set according to actual needs, and is not limited. For example, the load index can be 80%.

[0068] For example, if the load index is less than or equal to the preset threshold value, the first node determines to forward the write data to the second node.

[0069] In an example, if the load index is greater than the preset threshold value, the first node determines to temporarily cache the write data in the first node until the load index of the second node is less than or equal to the preset threshold value, or the second node completely writes the data in the cache to the backend hard disk.

[0070] It can be understood that the load of the second node is evaluated by the load index, and then when the load index is less than or equal to the preset threshold value, i.e., when the load of the second node is small, the write data is directly forwarded to the second node; when the load index is greater than the preset threshold value, i.e., when the load of the second node is large, the write data is temporarily cached in the first node; the pressure input to the high-load second node is reduced, service interruption or data damage caused by overloading operation is avoided, and the stable operation of the second node is protected. In addition, the node load can be balanced, and the overall data processing efficiency can be improved.

[0071] Embodiments of the present application provide a data writing method, applied to Figure 1 The second node is shown in FIG. 1, and the first node is shown in FIG. 2. Figure 5 As shown in FIG. 3, Figure 5 FIG. 4 is a flow diagram of another data writing method provided by embodiments of the present application. The data writing method comprises the following steps: S501, receiving the first detection information sent by the first node.

[0072] The first detection information is detection information generated by the first node after receiving the first type of write data sent by the host.

[0073] The first type of write data is write data sent by the host to the first node after determining that the first node joins the cluster.

[0074] S502, based on the first detection information, detecting whether to completely write the data in the cache to the backend hard disk, obtaining a first detection result, and sending the first detection result to the first node.

[0075] The first detection result is used for the first node to determine that the first detection result is that the second node does not completely write the data in the cache to the backend hard disk, and determine that the node state of the second node is online, and then forward the first type of write data to the second node.

[0076] In some optional embodiments, when the first detection result is that the data in the self cache is not completely written to the backend hard disk, a first average write time delay of the second node for writing data to the backend hard disk in a last period is obtained; if the first average write time delay is greater than a first threshold, a synchronous write strategy is started and executed.

[0077] The synchronous write strategy is used to instruct the second node to synchronize the first type of data in the self cache to the first node.

[0078] The first type of data is data in the second node to be synchronized to the first node.

[0079] In an example, during the process of synchronizing the first type of data in the self cache to the first node, a second average write time delay of the second node for writing data to the backend hard disk is periodically obtained, and an average synchronization time delay of the second node for synchronizing the first type of data to the first node is periodically obtained; when the second average write time delay is greater than the first threshold and the average synchronization time delay is less than or equal to a second threshold, the amount of data of the first type of data synchronized to the first node is gradually increased according to a first preset proportion.

[0080] Alternatively, in another example, when the second average write time delay is less than or equal to the first threshold, the amount of data of the first type of data synchronized to the first node is gradually reduced according to a second preset proportion.

[0081] Alternatively, in another example, when the average synchronization time delay is greater than the second threshold, the first type of data that has not been synchronized is no longer synchronized to the first node.

[0082] The first preset proportion and the second preset proportion can be set according to actual needs. The first preset proportion and the second preset proportion can be the same or different. For example, the first preset proportion and the second preset proportion can both be 10%.

[0083] S503, after receiving the first type of write data forwarded by the first node, writing the first type of write data to the backend hard disk.

[0084] In some optional embodiments, after the second node completely writes the first type of write data to the backend hard disk, the second node generates first response information; and the second node sends the first response information to the first node, so that the first node forwards the first response information to the host.

[0085] The first response information is used to indicate that the first type of write data has been completely written to the backend hard disk.

[0086] In some optional embodiments, the second type of write data issued by the host is received in real time in the process of writing data in the self cache to the backend hard disk; the third average write time delay of the second type of write data issued by the host to the second node is periodically acquired; the third average write time delay is used to determine the data write resource of the second node for writing the second type of write data to the backend hard disk corresponding to the third average write time delay; all resources of the second node except the data write resource of the second type of write data to the backend hard disk are used to write the first type of data to the backend hard disk.

[0087] In some optional embodiments, the second type of write data is the write data issued by the host to the second node.

[0088] In an example, when the third average write time delay is greater than the third threshold value and less than the fourth threshold value, the data write resource of the second node for writing the second type of write data of the host to the backend hard disk is determined as the first data write resource; when the third average write time delay is greater than the fourth threshold value, the data write resource of the second node for writing the second type of write data of the host to the backend hard disk is determined as the second data write resource.

[0089] In some optional embodiments, the second data write resource is greater than the first data write resource. For example, the first data write resource can account for 60% of the total data write resource; the second data write resource can account for 70% of the total data write resource.

[0090] It can be understood that the second type of write data is the write data issued by the host in real time, which has a high requirement for processing time delay. By periodically monitoring the third average write time delay, the processing efficiency of the second type of write data can be accurately determined. When the third average write time delay increases, sufficient data write resource is allocated for the second type of write data, so that the write data issued by the host in real time can be written into the backend hard disk in time, the processing delay caused by insufficient resource is avoided, and the smooth operation of the business relying on the write data issued by the host in real time is ensured.

[0091] Based on Figure 5The method shown, the first node can directly join the cluster again after emptying the data in the cache of the first node, without waiting for the second node to write all the data in its cache to the backend hard disk. Therefore, the speed of the first node joining the cluster can be improved, the cluster integration time during node restart, fault recovery or expansion is greatly shortened, and the service capability loss time caused by node offline can be significantly reduced. Since the second node does not write all the data in its cache to the backend hard disk, the first node cannot directly process data and write data to the backend hard disk although it has joined the cluster and the data is not consistent with the second node. At this time, when the first node receives the write data sent by the host, it can temporarily forward the write data to the second node for processing. In this way, even if the data of the first node and the second node are temporarily not synchronized after the first node joins the cluster, the write request of the host can still be received and processed through the forwarding mechanism, avoiding service interruption caused by node data misalignment, and ensuring the continuity and availability of business layer data write operation. Even if the first node is in a data inconsistency state when it just joins the cluster, it can still participate in cluster work through the forwarding mechanism, avoiding the risk of write request loss when the second node is overloaded or fails, indirectly improving the redundancy and fault tolerance of the cluster.

[0092] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment.

[0093] The embodiment of the present application also provides a data writing device, which is applied to a first node; as Figure 6 shown, Figure 6 The device structure block diagram of a data writing device provided by the embodiment of the present application; the data writing device comprises: a first transceiving module 601, configured to send first detection information to a second node after receiving write data sent by a host, the write data being write data sent by the host to the first node after determining that the first node joins a cluster; and receive first detection results sent by the second node, the first detection results being detection results of the second node based on the first detection information, detecting whether all the data in the cache of the second node is written to a backend hard disk.

[0094] A first processing module 602, configured to determine whether the node state of the second node is an online state when it is determined that the first detection results are that the second node does not write all the data in its cache to the backend hard disk; and forward the write data to the second node when it is determined that the node state of the second node is the online state, so that the second node writes the write data to the backend hard disk.

[0095] In some optional implementations, the first processing module 602 is specifically used to periodically acquire the first bitmap information sent by the second node, the first bitmap information being used to indicate the node status of the second node; if the first bitmap information is preset bitmap information, then it is determined that the node status of the second node corresponding to the first bitmap information is online; or, if the first bitmap information is not preset bitmap information, then it is determined that the node status of the second node corresponding to the first bitmap information is offline.

[0096] In some optional implementations, when the node status of the second node is determined to be online, after forwarding the write data to the second node, the first transceiver module 601 is further used to send second detection information to the second node. The second detection information is used to instruct the second node to check again whether all the data in its own cache has been written to the backend hard disk, obtain the second detection result, and send the second detection result to the first node; when the second detection result indicates that all the data in the second node's cache has been written to the backend hard disk, the sending of the latest acquired write data to the second node is stopped.

[0097] In some optional implementations, when the node status of the second node is determined to be online, before forwarding the write data to the second node, the first processing module 602 is further configured to obtain the current cache utilization rate, current input / output rate, maximum input / output rate supported by the second node, and weight value corresponding to the cache utilization rate of the second node at the current moment; determine the load index of the second node based on the current cache utilization rate, current input / output rate, maximum input / output rate, and weight value; if the load index is less than or equal to a preset threshold, then determine to forward the write data to the second node.

[0098] Embodiments of this application also provide another data writing device, applied to a second node; such as Figure 7 As shown, Figure 7 A device structure block diagram of another data writing apparatus provided in this application embodiment; the data writing apparatus includes: The second transceiver module 701 is used to receive the first detection information sent by the first node. The first detection information is the detection information generated by the first node after receiving the first type of write data sent by the host. The first type of write data is the write data sent by the host to the first node after determining that the first node has joined the cluster.

[0099] The second processing module 702 is used to detect whether all the data in its own cache has been written to the backend hard disk based on the first detection information, obtain the first detection result, and send the first detection result to the first node. When the first node determines that the first detection result is that the second node has not written all the data in its own cache to the backend hard disk, and determines that the node status of the second node is online, it forwards the first type of write data to the second node.

[0100] The writing module 703 is configured to write the first type of write data into the backend hard disk after receiving the first type of write data forwarded by the first node.

[0101] In some optional embodiments, the second processing module 702 is further configured to, when the first detection result is that the data in the cache of the second node is not completely written into the backend hard disk, acquire a first average write time delay of the second node for writing data into the backend hard disk in a last period; and if the first average write time delay is greater than a first threshold, start and execute a synchronous write strategy, and the synchronous write strategy is used to instruct the second node to synchronize the first type of data in the cache of the second node to the first node, and the first type of data is data in the second node to be synchronized to the first node.

[0102] In some optional embodiments, the second processing module 702 is specifically configured to, in the process of synchronizing the first type of data in the cache of the second node to the first node, periodically acquire a second average write time delay of the second node for writing data into the backend hard disk, and periodically acquire an average synchronization time delay of the second node for synchronizing the first type of data to the first node; when the second average write time delay is greater than the first threshold and the average synchronization time delay is less than or equal to a second threshold, gradually increase the data amount of the first type of data when being synchronized to the first node according to a first preset proportion; or when the second average write time delay is less than or equal to the first threshold, gradually reduce the data amount of the first type of data when being synchronized to the first node according to a second preset proportion; or when the average synchronization time delay is greater than the second threshold, no longer synchronize the first type of data that has not been executed to the first node.

[0103] In some optional embodiments, the second transceiving module 701 is further configured to, in the process of writing the data in the cache of the second node into the backend hard disk, receive a second type of write data issued by the host in real time; and the second processing module 702 is further configured to periodically acquire a third average write time delay of the second type of write data issued by the host to the second node; determine, according to the third average write time delay, data write resources of the second node for writing the second type of write data into the backend hard disk corresponding to the third average write time delay; and use all resources of the second node except the data write resources of the second type of write data into the backend hard disk to write the first type of data into the backend hard disk.

[0104] In some optional embodiments, the second processing module 702 is further configured to, after the first type of write data is completely written into the backend hard disk, generate first response information, and the first response information is used to indicate that the first type of write data has been completely written into the backend hard disk; and the second transceiving module 701 is further configured to send the first response information to the first node, so that the first node forwards the first response information to the host.

[0105] The features of the embodiments of the data writing device can be referred to the related descriptions of the embodiments of the data writing method, which will not be repeated here.

[0106] An embodiment of the present application further provides an electronic device, such as Figure 8 as shown in the figure, Figure 8 A hardware structure schematic diagram of an electronic device provided by an embodiment of the present application. The electronic device comprises a processor 10 and a memory 20, the memory 20 storing a computer program, and the processor 10 is configured to run the computer program to perform the steps in any of the above data writing method embodiments.

[0107] An embodiment of the present application further provides a computer readable storage medium, which stores a computer program, wherein the computer program is configured to perform the steps in any of the above data writing method embodiments when running.

[0108] In one exemplary embodiment, the above computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0109] An embodiment of the present application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps in any of the above data writing method embodiments.

[0110] An embodiment of the present application further provides another computer program product, which comprises a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in any of the above data writing method embodiments.

[0111] The skilled person can further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0112] The above describes in detail the data writing method and the electronic device provided by the present application. The principles and implementation manners of the present application are described by using specific examples, and the above description of the embodiments is only applicable to help understand the method of the present application and the core idea thereof. It should be pointed out that, for those skilled in the art, without departing from the principles of the present application, the present application can be improved and modified in several ways, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A data write method, characterized by, The method is applied to a scenario that a first node empties data in a cache of the first node and then joins a cluster again, the cluster is composed of the first node and a second node, the method is executed by the first node, and comprises: after receiving write data sent by a host, sending first detection information to the second node, the write data being write data sent by the host to the first node after it is determined that the first node joins the cluster; receiving first detection results sent by the second node, the first detection results being detection results of the second node based on the first detection information, whether to write all data in a cache of the second node to a back-end hard disk; when it is determined that the first detection results are that the second node does not write all data in the cache of the second node to the back-end hard disk, determining whether a node state of the second node is an online state; when it is determined that the node state of the second node is the online state, forwarding the write data to the second node, so that the second node writes the write data to the back-end hard disk.

2. The method of claim 1, wherein, The method further comprises: periodically acquiring first bitmap information sent by the second node, the first bitmap information being used to indicate the node state of the second node; if the first bitmap information is preset bitmap information, determining that the node state of the second node corresponding to the first bitmap information is in the online state; or, if the first bitmap information is non-prest bitmap information, determining that the node state of the second node corresponding to the first bitmap information is in an offline state.

3. The method of claim 2, wherein, The method further comprises: sending second detection information to the second node, the second detection information being used to instruct the second node to detect again whether to write all data in the cache of the second node to the back-end hard disk, obtain second detection results, and send the second detection results to the first node; when the second detection results indicate that all data in the cache of the second node is written to the back-end hard disk, stopping sending newly acquired write data to the second node.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: acquiring a current cache utilization rate, a current input / output rate, a maximum input / output rate supported by the second node, and a weight value corresponding to the cache utilization rate of the second node at a current moment; determining a load index of the second node according to the current cache utilization rate, the current input / output rate, the maximum input / output rate, and the weight value; if the load index is less than or equal to a preset threshold, determining to forward the write data to the second node.

5. A data writing method characterized by, The method is applied to a scenario that a first node empties data in a cache of the first node and then joins a cluster again, the cluster is composed of the first node and a second node, the method is executed by the second node, and comprises the following steps: receiving first detection information sent by the first node, the first detection information being detection information generated by the first node after receiving first write data of a host, the first write data being write data sent by the host to the first node after the host determines that the first node joins the cluster; based on the first detection information, detecting whether to write all data in a cache of the second node to a backend hard disk, obtaining a first detection result, and sending the first detection result to the first node, so that when the first node determines that the first detection result is that the second node does not write all data in the cache of the second node to the backend hard disk and determines that a node state of the second node is an online state, the first node forwards the first write data to the second node; after receiving the first write data forwarded by the first node, writing the first write data to the backend hard disk.

6. The method of claim 5, wherein, The method further comprises the following steps: when the first detection result is that it is detected that all data in the cache of the second node is not written to the backend hard disk, obtaining a first average write latency of write data of the second node to the backend hard disk in a last period; if the first average write latency is greater than a first threshold value, starting and executing a synchronous write strategy, the synchronous write strategy being used to instruct the second node to synchronize first type data in the cache of the second node to the first node, the first type data being data in the second node to be synchronized to the first node.

7. The method of claim 6, wherein, If the first average write latency is greater than a first threshold value, starting and executing a synchronous write strategy, comprising the following steps: in the process of synchronizing the first type data in the cache of the second node to the first node, periodically obtaining a second average write latency of write data of the second node to the backend hard disk, and periodically obtaining an average synchronization latency of the second node synchronizing the first type data to the first node; when the second average write latency is greater than the first threshold value and the average synchronization latency is less than or equal to a second threshold value, gradually increasing a data amount of the first type data when being synchronized to the first node according to a first preset proportion; or, when the second average write latency is less than or equal to the first threshold value, gradually reducing the data amount of the first type data when being synchronized to the first node according to a second preset proportion; or, when the average synchronization latency is greater than the second threshold value, no longer synchronizing the first type data which does not execute the synchronization operation to the first node.

8. The method of claim 6, wherein, The method further comprises the following steps: in the process of writing data in the cache of the second node to the backend hard disk, real-time receiving second write data issued by the host; periodically obtaining a third average write latency of the host issuing the second write data to the second node; According to the third average write latency, it is determined that the second node corresponding to the third average write latency writes the second type of write data to the data write resource of the back-end hard disk; All resources of the second node except the data write resource of the second type of write data to the back-end hard disk are used to write the first type of data to the back-end hard disk.

9. The method according to any one of claims 5-8, characterized in that, The method further comprises: After the first type of write data is completely written to the back-end hard disk, first response information is generated, the first response information being used to indicate that the first type of write data has been completely written to the back-end hard disk; The first response information is sent to the first node, so that the first node forwards the first response information to the host.

10. An electronic device, comprising: Comprise: A memory for storing a computer program; A processor for executing the computer program to realize the steps of the data write method according to any one of claims 1 to 9.

Images