A data writing method, an electronic device

Abstract

This application discloses a data writing method and an electronic device, relating to the field of storage technology. The method includes: after receiving write data sent by a host, sending first detection information to a second node, wherein the write data is the write data sent by the host to the first node after determining that the first node has joined the cluster; receiving a first detection result sent by the second node; when it is determined that the first detection result indicates that the second node has not written all the data in its cache to the backend hard disk, determining whether the node status of the second node is online; when it is determined that the node status of the second node is online, forwarding the write data to the second node so that the second node can write the write data to the backend hard disk. This application solves the problem that nodes that have left the cluster cannot quickly rejoin the cluster after recovery and cannot process data, resulting in poor cluster redundancy and fault tolerance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of storage, in particular to a data writing method and an electronic device. BACKGROUND

[0002] With the development of big data, cloud computing and storage technology, the requirement for data processing speed is also getting higher and higher. For a cluster of two nodes, the scenario of a single node leaving the cluster often occurs, which makes only the single node in the cluster process data, and there is a risk of failure, which makes the risk redundancy performance poor. Therefore, the node that leaves the cluster needs to be able to quickly join the cluster again after recovery, to ensure the reliability of the cluster and improve the risk resistance of the cluster.

[0003] In the related art, when the single node that leaves the cluster joins the cluster again, the data cached in the single node that leaves the cluster is old data compared with the data cached in the node that has been in the cluster all the time. If the host reads the old data from the single node that leaves the cluster, there will be a data inconsistency problem. Therefore, the single node that leaves the cluster needs to clear the old data in the cache before joining the cluster, and the node that has been in the cluster all the time needs to flush all the data in the cache to the back-end hard disk, so as to keep the data in the caches of the two nodes consistent, and at this time the single node that leaves the cluster can join the cluster. In this way, it takes several hours or even longer for the node that has been in the cluster all the time to flush all the data in the cache to the back-end hard disk, which makes the node that leaves the cluster unable to join the cluster as soon as possible after recovery, and also unable to process data, resulting in poor redundancy and fault tolerance of the cluster. SUMMARY

[0004] The present application provides a data writing method to at least solve the problem that the node that leaves the cluster cannot join the cluster as soon as possible after recovery, and also cannot process data, resulting in poor redundancy and fault tolerance of the cluster in the related art.

[0005] The present application provides a data writing method, which is applied to the scenario that a first node joins a cluster again after clearing the data in the cache of the first node, the cluster is composed of the first node and a second node, the method is performed by the first node, and includes: after receiving write data sent by a host, sending first detection information to the second node, the write data being write data sent by the host to the first node after determining that the first node joins the cluster; receiving first detection results sent by the second node, the first detection results being detection results of whether the second node writes all the data in the cache of the second node to a back-end hard disk based on the first detection information; when it is determined that the first detection results are that the second node does not write all the data in the cache of the second node to the back-end hard disk, determining whether a node state of the second node is an online state; when it is determined that the node state of the second node is the online state, forwarding the write data to the second node, so that the second node writes the write data to the back-end hard disk.

[0006] This application provides another data writing method, applied to a scenario where a first node rejoins a cluster after clearing its cache. The cluster consists of a first node and a second node. The method includes: receiving first detection information sent by the first node, wherein the first detection information is generated by the first node after receiving a first type of write data sent by the host, and the first type of write data is the write data sent by the host to the first node after determining that the first node has joined the cluster; based on the first detection information, detecting whether all the data in its own cache has been written to the backend hard disk, obtaining a first detection result, and sending the first detection result to the first node, so that when the first node determines that the first detection result indicates that the second node has not written all the data in its own cache to the backend hard disk, and determines that the node status of the second node is online, forwarding the first type of write data to the second node; and upon receiving the first type of write data forwarded by the first node, writing the first type of write data to the backend hard disk.

[0007] This application also provides a data writing device applied to a first node; the data writing device includes: a first transceiver module, configured to send first detection information to a second node after receiving write data sent by the host, wherein the write data is the write data sent by the host to the first node after determining that the first node has joined the cluster; and to receive a first detection result sent by the second node, wherein the first detection result is the detection result of the second node based on the first detection information to determine whether to write all the data in its own cache to the backend hard disk.

[0008] The first processing module is used to determine whether the node status of the second node is online when the first detection result indicates that the second node has not written all the data in its own cache to the backend hard disk; when the node status of the second node is determined to be online, the write data is forwarded to the second node so that the second node can write the write data to the backend hard disk.

[0009] This application also provides another data writing device for use in a second node; the data writing device includes:

[0010] The second transceiver module is used to receive the first detection information sent by the first node. The first detection information is the detection information generated by the first node after receiving the first type of write data sent by the host. The first type of write data is the write data sent by the host to the first node after determining that the first node has joined the cluster.

[0011] The second processing module is used to detect whether all the data in its own cache has been written to the backend hard disk based on the first detection information, obtain the first detection result, and send the first detection result to the first node. When the first node determines that the first detection result is that the second node has not written all the data in its own cache to the backend hard disk, and determines that the node status of the second node is online, it forwards the first type of write data to the second node.

[0012] The write module is used to write the first type of write data to the backend hard disk after receiving the first type of write data forwarded by the first node.

[0013] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the steps of any of the above data writing methods when executing the computer program.

[0014] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described data writing methods.

[0015] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described data writing methods.

[0016] This application allows the first node to rejoin the cluster directly after clearing its cache, without waiting for the second node to write all its cached data to the backend disk. This improves the speed at which the first node joins the cluster, significantly reduces cluster integration time during node restarts, fault recovery, or expansion, and substantially reduces service capacity loss due to node offline. Furthermore, because the second node has not written all its cached data to the backend disk, the first node, although joined, is not yet aligned with the second node's data and cannot directly process data or write it to the backend disk. In this case, when the first node receives write data from the host, it can temporarily forward the write data to the second node for processing. In this approach, even if the first node's data is temporarily out of sync with the second node after joining the cluster, it can still receive and process the host's write requests through the forwarding mechanism, avoiding service interruptions caused by misaligned node data and ensuring the continuity and availability of data write operations at the business layer. Even if the first node is in an inconsistent data state when it first joins the cluster, it can still participate in the cluster work through the forwarding mechanism, avoiding the risk of write request loss when the second node is overloaded or fails, and indirectly improving the redundancy and fault resistance of the cluster. Attached Figure Description

[0017] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A topology diagram of a data writing system provided in this application embodiment;

[0019] Figure 2 A flowchart illustrating a data writing method provided in an embodiment of this application;

[0020] Figure 3 The method for determining the node state of a second node provided in the embodiments of this application;

[0021] Figure 4 A flowchart illustrating yet another data writing method provided in an embodiment of this application;

[0022] Figure 5 A flowchart illustrating another data writing method provided in an embodiment of this application;

[0023] Figure 6 A device structure block diagram of a data writing apparatus provided in an embodiment of this application;

[0024] Figure 7 A device structure block diagram of another data writing apparatus provided in the embodiments of this application;

[0025] Figure 8 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0027] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0028] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] The embodiments of this application are applied to nodes that have left the cluster and are now rejoining the cluster.

[0030] In related technologies, node departure from a cluster includes the following scenarios. Scenario 1: When a node needs maintenance due to hardware failure, it needs to be put into service mode, causing it to leave the cluster. At this point, only a single node remains operational. The faulty node is then added back to the cluster after maintenance is complete. For example, in a cluster containing nodes A and B, to avoid service interruption during a version upgrade, each node needs to be upgraded sequentially, with node A upgraded successfully before node B. Upgrading node A requires node A to leave the cluster first. If a two-node cluster includes nodes A and B, the cluster needs to be started, and node A rejoins after the upgrade is complete. The same applies to node B. Another scenario is when an unpredictable software problem on a node causes a hot restart failure. During this failure, a single node may leave the cluster. After automatic or manual recovery, the node will rejoin the cluster.

[0031] In related technologies, when a node that left the cluster rejoins, all data in the cache of the node that left the cluster needs to be cleared. Data in the cache of nodes that have always been in the cluster needs to be flushed to the backend hard drive before the node that left the cluster rejoins. Taking a cluster consisting of nodes A and B as an example, if node B leaves the cluster for some reason, only node A remains to provide services to users. The data in node A's cache will gradually be updated to the latest data. When node B recovers and wants to rejoin the cluster, the data in node B's cache is older than the data in node A's cache. This data inconsistency causes the host to read data from node B's cache, requiring node B to clear its cache before rejoining the cluster. Node B can only join the cluster after node A has flushed its cache data because node B has already cleared all its data, and the two node caches are inconsistent. Therefore, all data in node A's cache needs to be flushed to the hard drive to ensure data consistency between the two node caches. If the backend hard drive's processing speed is slow, flushing the data may take several hours or even longer. Even if the backend hard drive is an SSD, it still takes time to flush the stale data. If a node leaves the cluster and does not rejoin the cluster, it cannot provide services to customers. This will result in a single node processing user data, which is less efficient than dual-node processing, leading to longer fault recovery times and a poor user experience.

[0032] To address the aforementioned technical issues, this application provides a data writing method. This method allows a first node leaving the cluster to immediately join the cluster without waiting for the second node, which is already in the cluster, to write the cached data to the backend hard drive. Before the second node flushes all the cached data to the hard drive, the method forwards the data sent from the host to the first node to the second node for processing. Simultaneously, it ensures that the data sent from the host to the first node is written to the backend hard drive, enabling the first node to process the data and ensuring rapid cluster recovery.

[0033] The following is based on Figure 1 The method provided in this application embodiment is described using the data writing system shown as an example.

[0034] like Figure 1 As shown, Figure 1 This is a topology diagram of a data writing system provided in an embodiment of this application. Figure 1 In this system, the data writing system 100 includes a first node 101, a second node 102, a host 103, and a backend hard disk 104. The first node 101 and the second node 102 form a cluster.

[0035] The first node 101 can be a node that has left and rejoined the cluster. The first node 101 includes a first forwarding module and a first caching module. The first forwarding module is used to forward data or information to the second node. The first caching module is used to cache data sent from the host to the first node.

[0036] The second node 102 can be a node that is always online in the cluster. The second node 102 includes a second forwarding module and a second caching module. The second forwarding module is used to forward data or information to the first node. The second caching module is used to cache data sent from the host to the second node.

[0037] Host 103 can be any device with communication and computing capabilities. For example, host 103 can be a server or a cloud server.

[0038] The back-end hard drive 104 can be any type of hard drive or hard drive group. The back-end hard drive 104 can be a solid-state drive or a hard disk drive.

[0039] Figure 1 The data writing system illustrated is for illustrative purposes only and is not intended to limit the technical solutions of this application. Those skilled in the art should understand that, in specific implementations, the data writing system may include more nodes, without limitation.

[0040] The embodiments of this application provide a data writing method, applied to a scenario where a first node rejoins the cluster after clearing the data in its cache; applied toFigure 1 The first node shown is as follows: Figure 2 As shown, Figure 2 This is a flowchart illustrating a data writing method provided in an embodiment of this application. The data writing method includes the following steps:

[0041] S201, after receiving the write data sent by the host, sends the first detection information to the second node.

[0042] Among them, write data refers to the write data sent by the host to the first node after determining that the first node has joined the cluster.

[0043] The first detection information is used to instruct the second node to check whether all the data in its own cache has been written to the backend hard disk.

[0044] For example, the host sends write data to the first node. After receiving the write data from the host, the first node sends first detection information to the second node.

[0045] S202, Receive the first detection result sent by the second node.

[0046] The first detection result is the result of the second node's detection of whether it has written all the data in its own cache to the backend hard disk based on the first detection information.

[0047] For example, based on the first detection information, the second node checks whether all the data in its own cache should be written to the backend hard disk, obtains the first detection result, and sends the first detection result to the first node. The first node receives the first detection result sent by the second node.

[0048] Optionally, if the first node determines that the first detection result is that the second node has written all the data in its own cache to the backend hard disk, then the first node is considered to be normal and writes the data to the backend hard disk.

[0049] S203, when it is determined that the first detection result is that the second node has not written all the data in its own cache to the backend hard disk, determine whether the node status of the second node is online.

[0050] The node status includes online status and offline status.

[0051] For example, when the first node determines that the first detection result indicates that the second node has not written all the data in its cache to the backend hard drive, the first node sends online detection information to the second node. The second node receives the online detection information, checks whether its own node status is online, and sends the detected node status to the first node. The first node receives the node status and determines whether the second node's node status is online.

[0052] S204: When it is determined that the node status of the second node is online, the write data is forwarded to the second node so that the second node can write the write data to the backend hard disk.

[0053] For example, when the first node determines that the second node is online, it forwards the write data to the second node so that the second node can write the write data to the backend hard disk.

[0054] based on Figure 2 The method shown is as follows: when the first node rejoins the cluster after clearing the data in its cache, it sends a first detection message to the second node after receiving write data from the host; it receives the first detection result sent by the second node; when it is determined that the first detection result indicates that the second node has not written all the data in its cache to the backend hard disk, it determines whether the node status of the second node is online; when it is determined that the node status of the second node is online, it forwards the write data to the second node so that the second node can write the write data to the backend hard disk.

[0055] In this approach, the first node can rejoin the cluster immediately after clearing its cached data, without waiting for the second node to write all its cached data to the backend disk. This improves the speed at which the first node joins the cluster, significantly reduces cluster integration time during node restarts, fault recovery, or expansion, and substantially reduces service capacity loss due to node offline. Since the second node hasn't written all its cached data to the backend disk, the first node, although joined, is not yet data-consistent with the second node and cannot directly process data or write it to the backend disk. In this case, when the first node receives write data from the host, it can temporarily forward the write data to the second node for processing. In this method, even if the first node's data is temporarily out of sync with the second node after joining the cluster, it can still receive and process write requests from the host through the forwarding mechanism, avoiding service interruptions due to misaligned node data and ensuring the continuity and availability of data write operations at the business layer. Even if the first node is in an inconsistent data state when it first joins the cluster, it can still participate in cluster operations through the forwarding mechanism, avoiding the risk of write request loss when the second node is overloaded or malfunctioning, indirectly improving the cluster's redundancy and fault tolerance.

[0056] In an optional example, based on the aforementioned embodiments, as described above, when it is determined that the first detection result indicates that the second node has not written all the data in its cache to the backend hard disk, the node status of the second node is determined to be online, as follows: See the detailed steps below. Figure 3 As shown, Figure 3 The method for determining the node state of a second node provided in the embodiments of this application includes:

[0057] S301, periodically acquire the first bit graph information sent by the second node.

[0058] The first bit of the graph information is used to indicate the node status of the second node. This first bit can be 0x10 or 0x00. 0x10 indicates that the second node is online. 0x00 indicates that the second node is offline.

[0059] For example, the second node periodically detects its own node status, generates first-order graph information based on its own node status, and sends the first-order graph information to the first node; the first node periodically obtains the first-order graph information sent by the second node.

[0060] S302, if the first bitmap information is a preset bitmap information, then determine that the node status of the second node corresponding to the first bitmap information is online.

[0061] The preset bitmap information is used to indicate that the second node is online. For example, the preset bitmap information can be 0x10.

[0062] For example, if the first bitmap information is a preset bitmap information, then the first node determines that the node status of the second node corresponding to the first bitmap information is online.

[0063] S303, or, if the first bitmap information is not preset bitmap information, then determine that the node state of the second node corresponding to the first bitmap information is offline.

[0064] The non-preset bitmap information is used to indicate that the second node's node status is offline. For example, the non-preset bitmap information can be 0x00.

[0065] For example, if the first bitmap information is not a preset bitmap information, the first node determines that the node state of the second node corresponding to the first bitmap information is offline.

[0066] Understandably, the first node will also periodically check its own node status and generate a second bitmap based on its own node status, and then send the second bitmap information to the first node; the second node will periodically receive the second bitmap information sent by the first node. This interaction method allows both nodes to quickly detect anomalies by periodically receiving each other's bitmap information. If the other node does not receive the bitmap within a certain period, it can be immediately determined that there is a problem with the other node.

[0067] Furthermore, in an optional example, based on the aforementioned embodiments, as described above, when it is determined that the node status of the second node is online, after forwarding the write data to the second node, the first node can also send second detection information to the second node; when the second detection result indicates that all the data in the second node's cache has been written to the backend hard disk, the sending of the latest acquired write data to the second node is stopped.

[0068] The second detection information is used to instruct the second node to check again whether all the data in its own cache has been written to the backend hard disk, obtain the second detection result, and send the second detection result to the first node.

[0069] Understandably, if the first node checks the second node's cache again to clear it, it can promptly stop sending new data to the second node. This prevents the second node from continuing to receive new data after writing the cached data to the hard drive, which would increase its processing pressure and prolong the data persistence time.

[0070] In an optional example, based on the foregoing embodiments, as described above, when the node status of the second node is determined to be online, before forwarding the write data to the second node, the first node may also perform the following method steps, see details below. Figure 4 As shown, Figure 4 A flowchart illustrating another data writing method provided in this application embodiment includes:

[0071] S401, obtain the current cache utilization rate, current input / output rate, maximum input / output rate supported by the second node, and weight value corresponding to the cache utilization rate at the current moment.

[0072] The weight value indicates the proportion of importance of cache utilization in load assessment.

[0073] S402 determines the load index of the second node based on the current cache utilization, current input / output rate, maximum input / output rate, and weight value.

[0074] In one example, the first node determines the load index of the second node based on the current cache utilization, current input / output rate, maximum input / output rate, and weight value, calculated using the following expression:

[0075]

[0076] in, This indicates the load index of the second node; Indicates the weight value; Indicates the current cache utilization rate; Indicates the current input / output rate; This indicates the maximum input / output rate.

[0077] Understandably, the load index combines cache utilization (with minimal impact at low weights) and I / O rates, more closely reflecting the actual load capacity of the second node. If the second node is frequently found to be losing data due to cache overflow, the weight of cache utilization can be temporarily increased, allowing the load index to more strictly limit data forwarding to that node. Conversely, if a node frequently experiences stuttering due to I / O bottlenecks, the weight of I / O rate-related metrics can be increased, prioritizing data forwarding control through I / O load. This ensures that write data from the first node can be written to the backend hard drive as quickly as possible, guaranteeing the overall performance of the cluster.

[0078] S403, if the load index is less than or equal to the preset threshold, then determine to forward the write data to the second node.

[0079] The preset threshold can be set according to actual needs and is not restricted. For example, the load index can be 80%.

[0080] For example, if the load index is less than or equal to a preset threshold, the first node will decide to forward the write data to the second node.

[0081] In one example, if the load index is greater than a preset threshold, the first node will temporarily cache the write data on the first node until the load index of the second node is less than or equal to the preset threshold, or the second node will write all the data in its cache to the backend hard disk.

[0082] Understandably, by assessing the load of the second node through the load index, and then directly forwarding write data to the second node when the load index is less than or equal to a preset threshold, i.e. when the load of the second node is low; when the load index is greater than the preset threshold, i.e. when the load of the second node is high, the write data is temporarily cached on the first node. This reduces the pressure input on the high-load second node, avoids service interruption or data corruption caused by overload, protects the stable operation of the second node, and can balance the node load, thereby improving the overall data processing efficiency.

[0083] The embodiments of this application provide a data writing method, applied to Figure 1 The second node shown is as follows: Figure 5 As shown, Figure 5 The following is a flowchart illustrating another data writing method provided in an embodiment of this application. The data writing method includes the following steps:

[0084] S501 receives the first detection information sent by the first node.

[0085] The first detection information is the detection information generated by the first node after receiving the first type of write data sent by the host.

[0086] The first type of write data is the write data sent by the host to the first node after it has determined that the first node has joined the cluster.

[0087] S502, based on the first detection information, checks whether all the data in its own cache has been written to the backend hard disk, obtains the first detection result, and sends the first detection result to the first node.

[0088] The first detection result is used by the first node to determine that the second node has not written all the data in its own cache to the backend hard disk, and to determine that the node status of the second node is online, and then forwards the first type of write data to the second node.

[0089] In some optional implementations, when the first detection result is that not all the data in its own cache has been written to the backend hard disk, the first average write latency of the second node writing data to the backend hard disk in the previous period is obtained; if the first average write latency is greater than the first threshold, the synchronous write strategy is started and executed.

[0090] Among them, the synchronous flushing strategy is used to instruct the second node to synchronize the first type of data in its own cache to the first node.

[0091] The first type of data is the data in the second node that needs to be synchronized to the first node.

[0092] In one example, during the process of synchronizing the first type of data in its own cache to the first node, the second average write latency of the second node to the backend hard disk is periodically obtained, as well as the average synchronization latency of the second node to synchronize the first type of data to the first node is periodically obtained; when the second average write latency is greater than the first threshold and the average synchronization latency is less than or equal to the second threshold, the amount of the first type of data synchronized to the first node is gradually increased according to the first preset ratio.

[0093] Alternatively, in another example, when the second average write latency is less than or equal to the first threshold, the amount of the first type of data synchronized to the first node is gradually reduced according to the second preset ratio.

[0094] Alternatively, in another example, when the average synchronization latency is greater than the second threshold, the first type of data that has not undergone synchronization will no longer be synchronized to the first node.

[0095] The first and second preset ratios can be set according to actual needs. The first and second preset ratios can be the same or different. For example, both the first and second preset ratios can be 10%.

[0096] S503, after receiving the first type of write data forwarded by the first node, writes the first type of write data to the backend hard disk.

[0097] In some alternative implementations, after the second node has written all the first type of write data to the backend hard disk, it generates a first response message and sends the first response message to the first node so that the first node can forward the first response message to the host.

[0098] The first response information is used to indicate that all of the first type of write data has been written to the back-end hard disk.

[0099] In some optional implementations, during the process of writing data from its own cache to the backend hard disk, the second type of write data sent by the host is received in real time; the third average write latency of the host sending the second type of write data to the second node is periodically obtained; based on the third average write latency, the data write resources of the second node for writing the second type of write data to the backend hard disk corresponding to the third average write latency are determined; and all resources in the second node other than the data write resources for writing the second type of write data to the backend hard disk are used to write the first type of data to the backend hard disk.

[0100] The second type of write data is the write data sent from the host to the second node.

[0101] In one example, when the third average write latency is greater than the third threshold but less than the fourth threshold, the data write resource of the second node writing host to the backend hard disk is determined as the first data write resource; when the third average write latency is greater than the fourth threshold, the data write resource of the second node writing host to the backend hard disk is determined as the second data write resource.

[0102] The second data flushing resource is greater than the first data flushing resource. For example, the first data flushing resource can account for 60% of the total data flushing resource, while the second data flushing resource can account for 70%.

[0103] Understandably, the second type of write data is data sent by the host in real time, which has high requirements for processing latency. By periodically monitoring the third average write latency, the processing efficiency of the second type of write data can be accurately determined. When the third average write latency increases, sufficient data write resources will be allocated to ensure that the write data sent by the host in real time can be written to the backend hard drive in a timely manner, avoiding processing delays caused by insufficient resources, thereby ensuring the smooth operation of services that rely on write data sent by the host in real time.

[0104] based on Figure 5The method shown allows the first node to rejoin the cluster directly after clearing its cache, without waiting for the second node to write all its cached data to the backend disk. This improves the speed at which the first node joins the cluster, significantly reduces cluster integration time during node restarts, fault recovery, or expansion, and substantially reduces service capacity loss due to node offline. However, because the second node has not written all its cached data to the backend disk, the first node, although joined, is not yet aligned with the second node's data and cannot directly process data or write it to the backend disk. In this case, when the first node receives write data from the host, it can temporarily forward the write data to the second node for processing. In this approach, even if the first node is temporarily out of sync with the second node after joining the cluster, it can still receive and process write requests from the host through the forwarding mechanism, avoiding service interruptions caused by misaligned node data and ensuring the continuity and availability of data write operations at the business layer. Even if the first node is in an inconsistent data state when it first joins the cluster, it can still participate in the cluster work through the forwarding mechanism, avoiding the risk of write request loss when the second node is overloaded or fails, and indirectly improving the redundancy and fault resistance of the cluster.

[0105] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0106] Embodiments of this application also provide a data writing device applied to a first node; such as Figure 6 As shown, Figure 6 This application provides a data writing device structure block diagram; the data writing device includes: a first transceiver module 601, configured to send first detection information to a second node after receiving write data sent by a host, wherein the write data is the write data sent by the host to the first node after determining that the first node has joined the cluster; and receive a first detection result sent by the second node, wherein the first detection result is the detection result of the second node based on the first detection information to determine whether to write all the data in its own cache to the backend hard disk.

[0107] The first processing module 602 is used to determine whether the node status of the second node is online when the first detection result indicates that the second node has not written all the data in its own cache to the backend hard disk; when the node status of the second node is determined to be online, the write data is forwarded to the second node so that the second node can write the write data to the backend hard disk.

[0108] In some optional implementations, the first processing module 602 is specifically used to periodically acquire the first bitmap information sent by the second node, the first bitmap information being used to indicate the node status of the second node; if the first bitmap information is preset bitmap information, then it is determined that the node status of the second node corresponding to the first bitmap information is online; or, if the first bitmap information is not preset bitmap information, then it is determined that the node status of the second node corresponding to the first bitmap information is offline.

[0109] In some optional implementations, when the node status of the second node is determined to be online, after forwarding the write data to the second node, the first transceiver module 601 is further used to send second detection information to the second node. The second detection information is used to instruct the second node to check again whether all the data in its own cache has been written to the backend hard disk, obtain the second detection result, and send the second detection result to the first node; when the second detection result indicates that all the data in the second node's cache has been written to the backend hard disk, the sending of the latest acquired write data to the second node is stopped.

[0110] In some optional implementations, when the node status of the second node is determined to be online, before forwarding the write data to the second node, the first processing module 602 is further configured to obtain the current cache utilization rate, current input / output rate, maximum input / output rate supported by the second node, and weight value corresponding to the cache utilization rate of the second node at the current moment; determine the load index of the second node based on the current cache utilization rate, current input / output rate, maximum input / output rate, and weight value; if the load index is less than or equal to a preset threshold, then determine to forward the write data to the second node.

[0111] Embodiments of this application also provide another data writing device, applied to a second node; such as Figure 7 As shown, Figure 7 A device structure block diagram of another data writing apparatus provided in this application embodiment; the data writing apparatus includes:

[0112] The second transceiver module 701 is used to receive the first detection information sent by the first node. The first detection information is the detection information generated by the first node after receiving the first type of write data sent by the host. The first type of write data is the write data sent by the host to the first node after determining that the first node has joined the cluster.

[0113] The second processing module 702 is used to detect whether all the data in its own cache has been written to the backend hard disk based on the first detection information, obtain the first detection result, and send the first detection result to the first node. When the first node determines that the first detection result is that the second node has not written all the data in its own cache to the backend hard disk, and determines that the node status of the second node is online, it forwards the first type of write data to the second node.

[0114] The write module 703 is used to write the first type of write data to the backend hard disk after receiving the first type of write data forwarded by the first node.

[0115] In some optional implementations, the second processing module 702 is further configured to, when the first detection result is that not all the data in its own cache has been written to the backend hard disk, obtain the first average write latency of the second node writing data to the backend hard disk in the previous period; if the first average write latency is greater than the first threshold, start and execute the synchronous write strategy, the synchronous write strategy is used to instruct the second node to synchronize the first type of data in its own cache to the first node, the first type of data being the data in the second node to be synchronized to the first node.

[0116] In some optional implementations, the second processing module 702 is specifically used to periodically obtain the second average write latency of the second node writing data to the backend hard disk and the average synchronization latency of the second node synchronizing the first type of data to the first node during the process of synchronizing the first type of data in its own cache to the first node; when the second average write latency is greater than a first threshold and the average synchronization latency is less than or equal to the second threshold, the amount of first type of data synchronized to the first node is gradually increased according to a first preset ratio; or, when the second average write latency is less than or equal to the first threshold, the amount of first type of data synchronized to the first node is gradually decreased according to a second preset ratio; or, when the average synchronization latency is greater than the second threshold, the first type of data that has not undergone synchronization operation is no longer synchronized to the first node.

[0117] In some optional implementations, the second transceiver module 701 is further configured to receive second-type write data sent by the host in real time during the process of writing data in its own cache to the back-end hard disk; the second processing module 702 is further configured to periodically obtain the third average write latency of the host sending second-type write data to the second node; determine the data write resources of the second node for writing second-type write data to the back-end hard disk corresponding to the third average write latency based on the third average write latency; and use all resources in the second node except for the data write resources for writing second-type write data to the back-end hard disk for writing first-type data to the back-end hard disk.

[0118] In some optional implementations, the second processing module 702 is further configured to generate a first response message after all the first type of write data has been written to the backend hard disk, the first response message being used to indicate that all the first type of write data has been written to the backend hard disk; the second transceiver module 701 is further configured to send the first response message to the first node, so that the first node forwards the first response message to the host.

[0119] For a description of the features in the embodiment corresponding to the data writing device, please refer to the relevant description of the embodiment corresponding to the data writing method, which will not be repeated here.

[0120] Embodiments of this application also provide an electronic device, such as... Figure 8 As shown, Figure 8 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device includes a processor 10 and a memory 20, in which a computer program is stored. The processor 10 is configured to run the computer program to perform the steps in any of the above-described data writing method embodiments.

[0121] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described data writing method embodiments when running.

[0122] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0123] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described data writing method embodiments.

[0124] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described data writing method embodiments.

[0125] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0126] The data writing method and electronic device provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to help understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A data write method, characterized by, The method is applied to a scenario that a first node empties data in a cache of the first node and then joins a cluster again, the cluster is composed of the first node and a second node, the method is executed by the first node, and comprises: after receiving write data sent by a host, sending first detection information to the second node, the write data being write data sent by the host to the first node after it is determined that the first node joins the cluster; receiving first detection results sent by the second node, the first detection results being detection results of the second node based on the first detection information, whether to write all data in a cache of the second node to a back-end hard disk; when it is determined that the first detection results are that the second node writes all data in the cache of the second node to the back-end hard disk, it is determined that the first node is normal, and the write data is written to the back-end hard disk; when it is determined that the first detection results are that the second node does not write all data in the cache of the second node to the back-end hard disk, it is determined whether a node state of the second node is an online state; when it is determined that the node state of the second node is the online state, the write data is forwarded to the second node, so that the second node writes the write data to the back-end hard disk.

2. The method of claim 1, wherein, The method further comprises: periodically acquiring first bitmap information sent by the second node, the first bitmap information being used for indicating the node state of the second node; if the first bitmap information is preset bitmap information, it is determined that the node state of the second node corresponding to the first bitmap information is in the online state; or, if the first bitmap information is non-pre-set bitmap information, it is determined that the node state of the second node corresponding to the first bitmap information is in an offline state.

3. The method of claim 2, wherein, The method further comprises: after it is determined that the node state of the second node is the online state, the method further comprises: sending second detection information to the second node, the second detection information being used for indicating that the second node detects again whether all data in a cache of the second node is written to the back-end hard disk, obtains second detection results, and sends the second detection results to the first node; 4. The method according to any one of claims 1-3, characterized in that, when the second detection results indicate that all data in the cache of the second node is written to the back-end hard disk, stopping sending newly acquired write data to the second node. The method further comprises: before it is determined that the node state of the second node is the online state, the method further comprises: acquiring a current cache utilization rate, a current input / output rate, a maximum input / output rate supported by the second node, and a weight value corresponding to the cache utilization rate of the second node at a current moment; determining a load index of the second node according to the current cache utilization rate, the current input / output rate, the maximum input / output rate, and the weight value; If the load index is less than or equal to a preset threshold, it is determined that the write data is to be forwarded to the second node.

5. A data writing method characterized by comprising: The method is applied to a scenario that a first node empties data in a cache of the first node and then joins a cluster again, the cluster being composed of the first node and a second node, and the method is performed by the second node and includes: receiving first detection information sent by the first node, the first detection information being detection information generated by the first node after receiving first write data of a first type sent by a host, the first write data of the first type being write data sent by the host to the first node after determining that the first node joins the cluster; based on the first detection information, detecting whether to write all data in a cache of the second node to a backend hard disk, obtaining a first detection result, and sending the first detection result to the first node, so that the first node determines that the first detection result is that the second node does not write all data in the cache of the second node to the backend hard disk, and determines that a node state of the second node is an online state, and forwards the first write data of the first type to the second node, and the first node determines that the first detection result is that the second node writes all data in the cache of the second node to the backend hard disk, and determines that the first node is normal, and writes the write data to the backend hard disk; after receiving the first write data of the first type forwarded by the first node, writing the first write data of the first type to the backend hard disk.

6. The method of claim 5, wherein, The method further includes: when the first detection result is that it is detected that all data in the cache of the second node is not written to the backend hard disk, obtaining a first average write time delay of the second node in a previous period; if the first average write time delay is greater than a first threshold, starting and executing a synchronous write strategy, the synchronous write strategy being used to instruct the second node to synchronize first type data in the cache of the second node to the first node, the first type data being data in the second node to be synchronized to the first node.

7. The method of claim 6, wherein, The method further includes: in the process of synchronizing the first type data in the cache of the second node to the first node, periodically obtaining a second average write time delay of the second node writing data to the backend hard disk, and periodically obtaining an average synchronization time delay of the second node synchronizing the first type data to the first node; when the second average write time delay is greater than the first threshold and the average synchronization time delay is less than or equal to a second threshold, gradually increasing a data amount of the first type data when being synchronized to the first node according to a first preset proportion; or, when the second average write time delay is less than or equal to the first threshold, gradually reducing the data amount of the first type data when being synchronized to the first node according to a second preset proportion; or, when the average synchronization time delay is greater than the second threshold, no longer synchronizing the first type data that has not been synchronized to the first node to the first node.

8. The method of claim 6, wherein, The method further includes: In the process of writing data in the self cache to the backend hard disk, the second type of write data issued by the host is received in real time; Periodically acquiring a third average write time delay of the host issuing the second type of write data to the second node; According to the third average write time delay, determining the second node write data to the backend hard disk data write resource corresponding to the third average write time delay; All resources in the second node except the data write resource of the second type of write data to the backend hard disk are used to write the first type of data to the backend hard disk.

9. The method according to any one of claims 5-8, characterized in that, The method further comprises: After all the first type of write data is written to the backend hard disk, generating first response information, the first response information is used to indicate that the first type of write data has been written to the backend hard disk; The first response information is sent to the first node, and the first response information is forwarded to the host by the first node.

10. An electronic device, comprising: Comprise: Memory, for storing computer programs; Processor, for executing the computer program to realize the steps of the data writing method in any one of claims 1 to 9.

Images