Data writing method, device, equipment, computer readable medium and program product

Abstract

Embodiments of the present disclosure disclose a data writing method, device, equipment, computer readable medium and program product. A specific embodiment of the method comprises: in response to receiving a data cache request, determining master cache cluster information in a cache configuration; performing distributed lock adding processing on a master cache cluster corresponding to the master cache cluster information; in response to determining that the master cache cluster is successfully locked, writing a data set to be cached into the master cache cluster; in response to determining that the data set to be cached is successfully written into the master cache cluster, and there is at least one slave cache cluster information in the cache configuration, writing the data set to be cached into at least one slave cache cluster corresponding to the at least one slave cache cluster information. The embodiment is related to data caching, and can guarantee consistency of writing the data set to be cached into the master cache cluster and the at least one slave cache cluster.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of computer technology, and more specifically to data writing methods, apparatus, devices, computer-readable media, and program products. Background Technology

[0002] Currently, most systems use caching services to store data in order to improve responsiveness. The typical approach to using caching services is to store data using at least one pre-defined caching service.

[0003] However, the inventors discovered that when using the above method to store data, the following technical problems often arise:

[0004] When each cache service writes data, some cache services are often interfered with by related tasks, resulting in inconsistent data writing across different cache services, which affects the subsequent access to the cached dataset.

[0005] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0007] Some embodiments of this disclosure provide data writing methods, apparatuses, devices, computer-readable media, and program products to address the technical problems mentioned in the background section above.

[0008] In a first aspect, some embodiments of this disclosure provide a data writing method, including: in response to receiving a data write cache request, determining primary cache cluster information in the cache configuration; performing distributed lock addition processing on the primary cache cluster corresponding to the primary cache cluster information; in response to determining that the primary cache cluster has been successfully locked, writing the dataset to be cached into the primary cache cluster; in response to determining that the dataset to be cached has been successfully written into the primary cache cluster, and that at least one secondary cache cluster information exists in the cache configuration, writing the dataset to be cached into at least one secondary cache cluster corresponding to the at least one secondary cache cluster information.

[0009] Optionally, the above method further includes: in response to determining that the dataset to be cached has been successfully written to at least one of the slave cache clusters, deleting the distributed lock for the master cache cluster.

[0010] Optionally, the method further includes: generating write failure information in response to determining that the dataset to be cached has not been successfully written to the main cache cluster; determining the number of write failures corresponding to the main cache cluster within a target time period; adding a predetermined value to the write failure information to obtain a summed write failure count; receiving first configuration modification information for the cache configuration in response to determining that the summed write failure count is greater than or equal to a predetermined threshold and that at least one slave cache cluster information exists in the cache configuration, wherein the first configuration modification information is information that determines any one of the slave cache cluster information in the at least one slave cache cluster information in the cache configuration as the main cache cluster information; determining the modified main cache cluster information in response to the first configuration modification information; and writing the dataset to be cached into the cache cluster corresponding to the modified main cache cluster information.

[0011] Optionally, the above method further includes: in response to determining that the number of write failures after addition is less than the predetermined threshold, sending cache failure information indicating that the data to be cached has failed to be written to the upstream server.

[0012] Optionally, the above method further includes: in response to determining that the fault repair of the target primary cache cluster is completed, receiving second configuration modification information, wherein the second configuration modification information indicates that the target primary cache cluster is determined as a secondary cache cluster, and the target primary cache cluster is the cache cluster corresponding to the primary cache cluster information that was replaced in the cache configuration; determining the target primary cache cluster as the cluster corresponding to the secondary cache cluster information according to the second configuration modification information; and writing the dataset to be cached into the target primary cache cluster.

[0013] Optionally, the above method further includes: in response to determining that at least one of the slave cache cluster information does not exist in the above cache configuration, deleting the distributed lock for the above master cache cluster.

[0014] Optionally, the above method further includes: in response to determining that locking the main cache cluster failed, determining locking failure type information; in response to determining that the locking failure type information indicates a main cache cluster loading failure, re-adding the distributed lock a predetermined number of times.

[0015] Optionally, the above method further includes: in response to determining that the above locking failure type information indicates a task occupancy loading failure and that the relevant occupancy task has finished executing, performing a distributed lock addition operation for the above main cache cluster.

[0016] Secondly, some embodiments of this disclosure provide a data writing apparatus, including: a determining unit configured to determine primary cache cluster information in a cache configuration in response to receiving a data write cache request; an adding unit configured to perform distributed lock addition processing on the primary cache cluster corresponding to the primary cache cluster information; a first writing unit configured to write the dataset to be cached to the primary cache cluster in response to determining that the primary cache cluster has been successfully locked; and a second writing unit configured to write the dataset to be cached to at least one secondary cache cluster corresponding to the at least one secondary cache cluster information in response to determining that the dataset to be cached has been successfully written to the primary cache cluster and that at least one secondary cache cluster information exists in the cache configuration.

[0017] Optionally, the above apparatus further includes: in response to determining that the dataset to be cached has been successfully written to at least one of the slave cache clusters, deleting the distributed lock for the master cache cluster.

[0018] Optionally, the apparatus further includes: generating write failure information in response to determining that the dataset to be cached has not been successfully written to the main cache cluster; determining the number of write failures corresponding to the main cache cluster within a target time period; adding a predetermined value to the write failure information to obtain a summed write failure count; receiving first configuration modification information for the cache configuration in response to determining that the summed write failure count is greater than or equal to a predetermined threshold and that at least one slave cache cluster information exists in the cache configuration, wherein the first configuration modification information is information that determines any one of the slave cache cluster information in the at least one slave cache cluster information in the cache configuration as the main cache cluster information; determining the modified main cache cluster information in response to the first configuration modification information; and writing the dataset to be cached into the cache cluster corresponding to the modified main cache cluster information.

[0019] Optionally, the above apparatus further includes: in response to determining that the number of write failures after addition is less than the predetermined threshold, sending cache failure information indicating that the data to be cached has failed to be written to an upstream server.

[0020] Optionally, the above apparatus further includes: in response to determining that the fault repair of the target primary cache cluster has been completed, receiving second configuration modification information, wherein the second configuration modification information indicates that the target primary cache cluster is determined as a secondary cache cluster, and the target primary cache cluster is the cache cluster corresponding to the primary cache cluster information that was replaced in the cache configuration; determining the target primary cache cluster as the cluster corresponding to the secondary cache cluster information according to the second configuration modification information; and writing the dataset to be cached into the target primary cache cluster.

[0021] Optionally, the above apparatus further includes: in response to determining that at least one slave cache cluster information does not exist in the above cache configuration, deleting the distributed lock for the above master cache cluster.

[0022] Optionally, the above apparatus further includes: in response to determining that locking the main cache cluster failed, determining locking failure type information; in response to determining that the locking failure type information indicates a main cache cluster loading failure, re-adding the distributed lock a predetermined number of times.

[0023] Optionally, the above apparatus further includes: in response to determining that the above-mentioned locking failure type information indicates a task occupancy loading failure and that the relevant occupancy task has finished executing, performing a distributed lock addition operation for the above-mentioned main cache cluster.

[0024] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, such that when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0025] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method as described in any implementation of the first aspect.

[0026] Fifthly, some embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0027] The above embodiments of this disclosure have the following beneficial effects: the data writing method of some embodiments of this disclosure can ensure the consistency of the data set to be cached written to the main cache cluster and at least one secondary cache cluster. Specifically, the reason for the inconsistency in writing the data set to be cached is that when each cache service writes data, some cache services are often interfered with by related tasks, resulting in inconsistency in the data written by each cache service, which affects the subsequent retrieval of the data set to be cached. Based on this, the data writing method of some embodiments of this disclosure firstly, in response to receiving a data write cache request, determines the main cache cluster information in the cache configuration for subsequent writing of the data set to be cached and performing a distributed lock addition operation. Then, a distributed lock addition process is performed on the main cache cluster corresponding to the above main cache cluster information to ensure that there is no interference from other tasks during the execution of the task of writing the data set to be cached to the cache cluster. Next, in response to determining that the main cache cluster has been successfully locked, the data set to be cached can be accurately written to the main cache cluster so that the corresponding data can be retrieved from the main cache cluster later. Finally, in response to the confirmation that the dataset to be cached has been successfully written to the primary cache cluster and that at least one secondary cache cluster exists in the cache configuration, the dataset to be cached is written to at least one secondary cache cluster corresponding to the at least one secondary cache cluster. Here, by implementing distributed locking on the primary cache cluster, it can be ensured that the dataset to be cached is not interfered with by other tasks while being written to the primary cache cluster and at least one secondary cache cluster, thus guaranteeing the consistency of the cached dataset's write operation. Attached Figure Description

[0028] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0029] Figure 1 This is a schematic diagram illustrating an application scenario of a data writing method according to some embodiments of the present disclosure;

[0030] Figure 2 These are flowcharts of some embodiments of the data writing method according to this disclosure;

[0031] Figure 3 This is a flowchart of some other embodiments of the data writing method according to this disclosure;

[0032] Figure 4 These are schematic diagrams illustrating the structure of some embodiments of the data writing apparatus according to this disclosure;

[0033] Figure 5This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0034] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0035] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0036] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0037] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0038] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0039] Before performing any of the operations involving the collection, storage, or use of user personal information (such as datasets to be cached) as disclosed in this disclosure, the relevant organizations or individuals shall fulfill their obligations, including conducting personal information security impact assessments, informing personal information subjects, and obtaining prior authorization and consent from personal information subjects.

[0040] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0041] Figure 1 This is a schematic diagram of an application scenario of a data writing method according to some embodiments of the present disclosure.

[0042] exist Figure 1In this application scenario, firstly, in response to receiving a data write-to-cache request, electronic device 101 can determine the primary cache cluster information 1021 in cache configuration 102. In this application scenario, the primary cache cluster information 1021 can be: {Primary cache cluster information: Cluster A information}. Then, electronic device 101 can perform distributed lock addition processing on the primary cache cluster 103 corresponding to the primary cache cluster information 1021. Next, in response to determining that the primary cache cluster 103 has been successfully locked, electronic device 101 can write the dataset 105 to be cached into the primary cache cluster 103. Finally, in response to determining that the dataset 105 to be cached has been successfully written into the primary cache cluster 103, and that there is at least one secondary cache cluster information 1022 in the cache configuration 102, electronic device 101 can write the dataset 105 to be cached into at least one secondary cache cluster 104 corresponding to the at least one secondary cache cluster information 1022. In this application scenario, the at least one secondary cache cluster information 1022 can be: {Secondary cache cluster information: Cluster B information, Cluster C information, Cluster D information}. Cluster B information corresponds to at least one slave cache cluster 1041 in slave cache cluster 104. Cluster C information corresponds to at least one slave cache cluster 1042 in slave cache cluster 104. Cluster D information corresponds to at least one slave cache cluster 1043 in slave cache cluster 104.

[0043] It should be noted that the aforementioned electronic device 101 can be either hardware or software. When the electronic device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the electronic device is software, it can be installed in the hardware devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here.

[0044] It should be understood that Figure 1 The number of electronic devices shown is merely illustrative. Any number of electronic devices can be used depending on the implementation requirements.

[0045] Continue to refer to Figure 2 The diagram illustrates a flow 200 of some embodiments of a data writing method according to the present disclosure. The data writing method includes the following steps:

[0046] Step 201: In response to receiving a data write to cache request, determine the main cache cluster information in the cache configuration.

[0047] In some embodiments, in response to receiving a data write cache request, the execution body of the above data write method (e.g., Figure 1The electronic device 101 shown can determine the primary cache cluster information in the cache configuration. The aforementioned data write-to-cache request can be a request instructing the cached dataset to be written to the cache cluster. The aforementioned cache configuration is the configuration information of the cache cluster that needs to write the cached dataset. The aforementioned cache configuration can include: primary cache cluster information. In practice, the aforementioned primary cache cluster information can be the cluster name information of the primary cache cluster. The aforementioned cache configuration can be configuration information generated by a configuration center. The aforementioned configuration center is an information management center for generating and modifying cache configurations. The aforementioned primary cache cluster can be the cluster to be queried when the cached data is subsequently queried. The primary cache cluster can be a group of independent computers interconnected by a high-speed network that can be used for caching data. The aforementioned primary cache cluster can be, but is not limited to, one of the following: a ZooKeeper cluster, a Nacos cluster.

[0048] Step 202: Perform distributed lock addition processing on the main cache cluster corresponding to the above main cache cluster information.

[0049] In some embodiments, the aforementioned execution entity can add distributed locks to the main cache cluster corresponding to the aforementioned main cache cluster information. The purpose of adding distributed locks is to control the orderly processing of shared resources by the distributed system, maintaining consistency through the principle of task mutual exclusion. For example, the tasks processing shared resources include a first task and a second task. When the first task has added a distributed lock and is processing the shared resource, the second task cannot process the shared resource because it cannot acquire the lock information corresponding to the first task's distributed lock. The second task can only execute its resource processing after the first task has finished processing the shared resource.

[0050] Step 203: In response to the successful locking of the main cache cluster, the dataset to be cached is written to the main cache cluster.

[0051] In some embodiments, in response to determining that locking the main cache cluster was successful, the execution entity may write the dataset to be cached into the main cache cluster. The dataset to be cached may be the dataset corresponding to the data write-to-cache request.

[0052] In some optional implementations of certain embodiments, after step 203, the steps may further include:

[0053] The first step, in response to the determination that locking the aforementioned primary cache cluster failed, is for the execution entity to determine the locking failure type information. This locking failure type information characterizes the type of locking failure. In practice, the locking failure type may include: primary cache cluster loading failure.

[0054] The second step involves determining that the lock acquisition failure type information indicates a main cache cluster loading failure. The executing entity can then re-add the distributed lock a predetermined number of times until the addition of the distributed lock is successful. For example, the predetermined number could be "5".

[0055] Optionally, the steps may also include:

[0056] In response to the determination that the above-mentioned locking failure type information indicates a task occupancy loading failure and that the relevant occupancy task has finished executing, the aforementioned execution entity can perform a distributed lock addition operation on the aforementioned main cache cluster. The relevant occupancy task can be a task currently processing resources on the main cache set.

[0057] Step 204: In response to determining that the dataset to be cached has been successfully written to the main cache cluster and that there is at least one slave cache cluster information in the cache configuration, the dataset to be cached is written to at least one slave cache cluster corresponding to the at least one slave cache cluster information.

[0058] In some embodiments, in response to determining that the dataset to be cached has been successfully written to the primary cache cluster and that at least one secondary cache cluster exists in the cache configuration, the executing entity may write the dataset to be cached to at least one secondary cache cluster corresponding to the at least one secondary cache cluster. The number of secondary cache clusters included in the at least one secondary cache cluster can be preset. For example, the number of secondary cache clusters may be one. The secondary cache cluster can be, but is not limited to, one of the following: a ZooKeeper cluster or a Nacos cluster.

[0059] Optionally, the cluster type of the cache cluster corresponding to the cache cluster information can be the same as or different from the cluster type of the cache cluster corresponding to the main cache cluster information.

[0060] In some optional implementations of certain embodiments, after step 204, the above steps further include:

[0061] In response to determining that the aforementioned dataset to be cached has been successfully written to at least one of the aforementioned slave cache clusters, the aforementioned execution entity may remove the distributed lock against the aforementioned master cache cluster.

[0062] Here, the distributed lock for the aforementioned primary cache cluster is removed to allow subsequent tasks to execute data processing tasks for both the primary cache cluster and at least one secondary cache cluster.

[0063] In some optional implementations of certain embodiments, after step 204, the above steps further include:

[0064] In response to determining that at least one of the slave cache clusters is not present in the above cache configuration, the execution entity may delete the distributed lock for the master cache cluster.

[0065] Here, if there is no at least one slave cache cluster in the cache configuration, then since only the master cache cluster needs to write the dataset to be cached, there will be no problem with inconsistent cache dataset writing.

[0066] The above embodiments of this disclosure have the following beneficial effects: the data writing method of some embodiments of this disclosure can ensure the consistency of the data set to be cached written to the main cache cluster and at least one secondary cache cluster. Specifically, the reason for the inconsistency in writing the data set to be cached is that when each cache service writes data, some cache services are often interfered with by related tasks, resulting in inconsistency in the data written by each cache service, which affects the subsequent retrieval of the data set to be cached. Based on this, the data writing method of some embodiments of this disclosure firstly, in response to receiving a data write cache request, determines the main cache cluster information in the cache configuration for subsequent writing of the data set to be cached and performing a distributed lock addition operation. Then, a distributed lock addition process is performed on the main cache cluster corresponding to the above main cache cluster information to ensure that there is no interference from other tasks during the execution of the task of writing the data set to be cached to the cache cluster. Next, in response to determining that the main cache cluster has been successfully locked, the data set to be cached can be accurately written to the main cache cluster so that the corresponding data can be retrieved from the main cache cluster later. Finally, in response to the confirmation that the dataset to be cached has been successfully written to the primary cache cluster and that at least one secondary cache cluster exists in the cache configuration, the dataset to be cached is written to at least one secondary cache cluster corresponding to the at least one secondary cache cluster. Here, by implementing distributed locking on the primary cache cluster, it can be ensured that the dataset to be cached is not interfered with by other tasks while being written to the primary cache cluster and at least one secondary cache cluster, thus guaranteeing the consistency of the cached dataset's write operation.

[0067] Further reference Figure 3 The diagram illustrates a flow 300 of another embodiment of the data writing method according to this disclosure. This data writing method includes the following steps:

[0068] Step 301: In response to receiving a data write to cache request, determine the main cache cluster information in the cache configuration.

[0069] Step 302: Perform distributed lock addition processing on the main cache cluster corresponding to the above main cache cluster information.

[0070] Step 303: In response to the successful locking of the main cache cluster, the dataset to be cached is written to the main cache cluster.

[0071] Step 304: In response to determining that the dataset to be cached has been successfully written to the primary cache cluster and that there is at least one secondary cache cluster information in the cache configuration, the dataset to be cached is written to at least one secondary cache cluster corresponding to the at least one secondary cache cluster information.

[0072] In some embodiments, the specific implementation of steps 301-304 and the resulting technical effects can be found in [reference needed]. Figure 2 Steps 201-204 in the corresponding embodiments will not be repeated here.

[0073] Step 305: In response to determining that the above-mentioned dataset to be cached has not been successfully written to the above-mentioned main cache cluster, a write failure message is generated.

[0074] In some embodiments, in response to determining that the dataset to be cached has not been successfully written to the main cache cluster, the execution entity (e.g. Figure 1 The electronic device 101 shown can generate write failure information. This write failure information can be an identifier indicating that the dataset to be cached has not been successfully written to the main cache cluster. For example, the write failure information can be "1", which indicates that the dataset to be cached has not been successfully written to the main cache cluster.

[0075] Step 306: Determine the number of write failures corresponding to the above-mentioned main cache cluster within the target time period.

[0076] In some embodiments, the execution entity can determine the number of write failures corresponding to the main cache cluster within a target time period. The target time period can be a pre-set time period. The number of write failures corresponding to the main cache cluster can be the number of times the main cache cluster failed to write to the cached dataset. For example, the number of write failures is 4.

[0077] Step 307: For the above write failure information, add the above write failure count to a predetermined value to obtain the summed write failure count.

[0078] In some embodiments, the execution entity may add a predetermined value to the number of write failures for the aforementioned write failure information to obtain a summed write failure count. For example, the write failure count may be "4". The predetermined value may be "1". The summed write failure count may be "5".

[0079] Step 308: In response to determining that the number of write failures after addition is greater than or equal to a predetermined threshold and that at least one of the above-mentioned cache cluster information exists in the above-mentioned cache configuration, first configuration modification information for the above-mentioned cache configuration is received.

[0080] In some embodiments, in response to determining that the number of write failures after addition is greater than or equal to a predetermined threshold, and that at least one slave cache cluster information exists in the cache configuration, the execution entity may receive first configuration modification information for the cache configuration. The first configuration modification information is information that determines any one of the at least one slave cache cluster information in the cache configuration as the master cache cluster information. For example, the predetermined threshold is 3.

[0081] In some optional implementations of certain embodiments, after step 308, the steps further include:

[0082] In response to determining that the number of write failures after the above addition is less than the predetermined threshold, the execution entity may send cache failure information indicating that the above-mentioned data to be cached has failed to be written to the upstream server.

[0083] Step 309: Based on the first configuration modification information mentioned above, determine the modified main cache cluster information.

[0084] In some embodiments, the execution entity can determine the modified main cache cluster information based on the first configuration modification information. The modified main cache cluster information is at least one cluster information determined to be the main cache cluster information from the cache cluster information.

[0085] Step 310: Write the above-mentioned dataset to be cached into the cache cluster corresponding to the modified main cache cluster information.

[0086] In some embodiments, the execution entity may write the dataset to be cached into the cache cluster corresponding to the modified main cache cluster information.

[0087] In some optional implementations of certain embodiments, after step 310, the steps further include:

[0088] In the first step, in response to the determination that the target primary cache cluster has completed fault repair, the aforementioned execution entity can receive the second configuration modification information. This second configuration modification information indicates that the target primary cache cluster has been designated as a secondary cache cluster, and the target primary cache cluster is the cache cluster corresponding to the primary cache cluster information that was replaced in the aforementioned cache configuration.

[0089] It should be noted that the number of primary cache clusters for data caching can be one. The target primary cache cluster can be a cluster that previously had a fault, was replaced in the first configuration modification, or was originally the primary cache cluster (i.e., any one of the at least one secondary cache cluster is selected as the primary cache cluster, and the original faulty primary cache cluster is repaired). Therefore, after the target primary cache cluster is repaired, it can be used as a secondary cache cluster for subsequent data synchronization and caching.

[0090] In the second step, the aforementioned execution entity can modify the aforementioned second configuration information to determine the target primary cache cluster as the cluster corresponding to the secondary cache cluster information.

[0091] Third, the aforementioned execution entity can write the aforementioned dataset to be cached into the aforementioned target main cache cluster.

[0092] Here, the consistency of data caching is ensured by writing the dataset to be cached into the target main cache cluster mentioned above.

[0093] from Figure 3 It can be seen from this that, with Figure 2 Compared to the description of some corresponding embodiments, Figure 3 In some corresponding embodiments, the data writing method process 300 involves the following steps: First, when it is determined that the dataset to be cached has failed to be successfully written to the main cache cluster, a write failure message is generated. Next, by statistically analyzing the number of write failures in the main cache cluster within a target time period, it is determined whether the main cache cluster information needs to be adjusted. Then, when the sum of the number of write failures and a predetermined value exceeds a predetermined threshold, and at least one slave cache cluster exists in the cache configuration, the main cache cluster information is adjusted using first configuration modification information. Finally, the dataset to be cached is written to the cache cluster corresponding to the modified main cache cluster information. This cache cluster disaster recovery method enables efficient and secure writing of the dataset to be cached.

[0094] Further reference Figure 4 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a data writing device, which are similar to... Figure 2 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0095] like Figure 4As shown, a data writing device 400 includes: a determining unit 401, an adding unit 402, a first writing unit 403, and a second writing unit 404. The determining unit 401 is configured to determine the main cache cluster information in the cache configuration in response to receiving a data write cache request; the adding unit 402 is configured to perform distributed lock addition processing on the main cache cluster corresponding to the main cache cluster information; the first writing unit 403 is configured to write the dataset to be cached to the main cache cluster in response to determining that the lock on the main cache cluster is successful; and the second writing unit 404 is configured to write the dataset to be cached to at least one secondary cache cluster corresponding to the at least one secondary cache cluster information in response to determining that the dataset to be cached has been successfully written to the main cache cluster and that at least one secondary cache cluster information exists in the cache configuration.

[0096] In some optional implementations of certain embodiments, the apparatus 400 further includes a deletion unit (not shown in the figure). The deletion unit can be configured to delete the distributed lock on the primary cache cluster in response to determining that the dataset to be cached has been successfully written to at least one secondary cache cluster.

[0097] In some optional implementations of certain embodiments, the apparatus 400 further includes: an information writing unit, a count determination unit, an addition unit, a first receiving unit, an information determination unit, and a third writing unit (not shown in the figure). The information writing unit can be configured to: generate write failure information in response to determining that the dataset to be cached has not been successfully written to the main cache cluster. The count determination unit can be configured to: determine the number of write failures corresponding to the main cache cluster within a target time period. The addition unit can be configured to: add a predetermined value to the write failure information to obtain a summed write failure count. The receiving unit can be configured to: receive first configuration modification information for the cache configuration in response to determining that the summed write failure count is greater than or equal to a predetermined threshold and that at least one slave cache cluster information exists in the cache configuration, wherein the first configuration modification information determines any one of the at least one slave cache cluster information in the cache configuration as the main cache cluster information. The information determination unit can be configured to: determine the modified main cache cluster information based on the first configuration modification information. The third write unit can be configured to write the above-mentioned dataset to be cached into the cache cluster corresponding to the modified main cache cluster information.

[0098] In some optional implementations of certain embodiments, the apparatus 400 further includes a sending unit (not shown in the figure). The sending unit can be configured to send cache failure information indicating a failure to write the cached data to an upstream server in response to determining that the number of write failures after addition is less than a predetermined threshold.

[0099] In some optional implementations of certain embodiments, the apparatus 400 further includes: a second receiving unit, a cluster information determining unit, and a fourth writing unit (not shown in the figure). The second receiving unit can be configured to: receive second configuration modification information in response to determining that the target primary cache cluster fault repair is complete, wherein the second configuration modification information indicates that the target primary cache cluster is determined as a secondary cache cluster, and the target primary cache cluster is the cache cluster corresponding to the primary cache cluster information replaced in the cache configuration. The cluster information determining unit can be configured to: determine the target primary cache cluster as the cluster corresponding to the secondary cache cluster information based on the second configuration modification information. The fourth writing unit can be configured to: write the dataset to be cached into the target primary cache cluster.

[0100] In some optional implementations of certain embodiments, the apparatus 400 further includes a deletion unit (not shown in the figure). The deletion unit can be configured to delete the distributed lock for the primary cache cluster in response to determining that at least one secondary cache cluster information does not exist in the cache configuration.

[0101] In some optional implementations of certain embodiments, the apparatus 400 further includes a type information determination unit and a re-adding unit (not shown in the figure). The type information determination unit can be configured to determine lock failure type information in response to determining that locking the main cache cluster has failed. The re-adding unit can be configured to re-add the distributed lock a predetermined number of times in response to determining that the lock failure type information indicates a main cache cluster loading failure.

[0102] In some optional implementations of certain embodiments, the apparatus 400 further includes an execution unit (not shown in the figure). The execution unit can be configured to perform a distributed lock addition operation on the main cache cluster in response to determining that the lock failure type information indicates a task occupancy loading failure and that the relevant occupancy task has finished executing.

[0103] It is understandable that the units described in the device 400 are related to the reference. Figure 2 The steps in the described method correspond accordingly. Therefore, the operations, features, and beneficial effects described above for the method also apply to device 400 and the units contained therein, and will not be repeated here.

[0104] The following is for reference. Figure 5 It illustrates electronic devices suitable for implementing some embodiments of this disclosure (e.g., Figure 1 A schematic diagram of the structure of electronic device 101)500. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0105] like Figure 5 As shown, the electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0106] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 5 Each box shown can represent a device or multiple devices as needed.

[0107] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 509, or installed from storage device 508, or installed from ROM 502. When the computer program is executed by processing device 501, it performs the functions defined in the methods of some embodiments of this disclosure.

[0108] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0109] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0110] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: in response to receiving a data write-to-cache request, determine the primary cache cluster information in the cache configuration; perform distributed lock addition processing on the primary cache cluster corresponding to the primary cache cluster information; in response to determining that locking the primary cache cluster is successful, write the dataset to be cached into the primary cache cluster; and in response to determining that the dataset to be cached has been successfully written into the primary cache cluster, and that at least one secondary cache cluster information exists in the cache configuration, write the dataset to be cached into at least one secondary cache cluster corresponding to the at least one secondary cache cluster information.

[0111] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0112] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0113] The units described in some embodiments of this disclosure can be implemented in software or in hardware. The described units can also be located in a processor; for example, a processor can be described as including: a determining unit, an adding unit, a first writing unit, and a second writing unit. The names of these units do not necessarily limit the unit itself; for example, the determining unit can also be described as "a unit that, in response to receiving a data write cache request, determines the main cache cluster information in the cache configuration."

[0114] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0115] Some embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements any of the data writing methods described above.

[0116] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A data writing method, comprising: In response to receiving a data write to cache request, the main cache cluster information in the cache configuration is determined, wherein the cache configuration is generated by the configuration center, which is an information management center for generating and modifying cache configurations; Perform distributed lock addition processing on the main cache cluster corresponding to the main cache cluster information; In response to the successful locking of the main cache cluster, the dataset to be cached is written to the main cache cluster; In response to determining that the dataset to be cached has been successfully written to the primary cache cluster and that there is at least one secondary cache cluster information in the cache configuration, the dataset to be cached is written to at least one secondary cache cluster corresponding to the at least one secondary cache cluster information.

2. The method according to claim 1, wherein, The method further includes: In response to determining that the dataset to be cached has been successfully written to the at least one slave cache cluster, the distributed lock for the master cache cluster is removed.

3. The method according to claim 1, wherein, The method further includes: In response to determining that the dataset to be cached has not been successfully written to the main cache cluster, a write failure message is generated; Determine the number of write failures corresponding to the main cache cluster within the target time period; For the write failure information, add a predetermined value to the number of write failures to obtain the summed write failure count; In response to determining that the number of write failures after addition is greater than or equal to a predetermined threshold and that at least one slave cache cluster information exists in the cache configuration, the system receives first configuration modification information for the cache configuration, wherein the first configuration modification information is information that determines any one of the slave cache cluster information in the at least one slave cache cluster information in the cache configuration as the master cache cluster information. Based on the first configuration modification information, determine the modified main cache cluster information; Write the dataset to be cached into the cache cluster corresponding to the modified main cache cluster information.

4. The method according to claim 3, wherein, The method further includes: In response to determining that the number of write failures after addition is less than the predetermined threshold, cache failure information indicating that the data to be cached has failed to be written is sent to the upstream server.

5. The method according to claim 3, wherein, The method further includes: In response to determining that the fault repair of the target primary cache cluster is completed, a second configuration modification information is received, wherein the second configuration modification information indicates that the target primary cache cluster is determined as a secondary cache cluster, and the target primary cache cluster is the cache cluster corresponding to the primary cache cluster information that was replaced in the cache configuration; Based on the second configuration modification information, the target primary cache cluster is determined as the cluster corresponding to the secondary cache cluster information; Write the dataset to be cached into the target main cache cluster.

6. The method according to claim 1, wherein, The method further includes: In response to determining that at least one slave cache cluster information does not exist in the cache configuration, the distributed lock for the master cache cluster is deleted.

7. The method according to claim 1, wherein, The method further includes: In response to determining that locking the main cache cluster failed, the type of locking failure information is determined; In response to determining that the locking failure type information indicates a main cache cluster loading failure, the distributed lock is re-added a predetermined number of times.

8. The method according to claim 7, wherein, The method further includes: In response to determining that the locking failure type information indicates a task loading failure and that the relevant task has finished executing, a distributed lock addition operation is performed for the main cache cluster.

9. A data writing device, comprising: The determining unit is configured to determine the main cache cluster information in the cache configuration in response to receiving a data write cache request, wherein the cache configuration is generated by a configuration center, which is an information management center for generating and modifying cache configurations; The addition unit is configured to perform distributed lock addition processing on the main cache cluster corresponding to the main cache cluster information; The first write unit is configured to write the dataset to be cached into the main cache cluster in response to determining that locking the main cache cluster has been successful. The second write unit is configured to write the dataset to be cached to at least one slave cache cluster corresponding to the at least one slave cache cluster in response to determining that the dataset to be cached has been successfully written to the primary cache cluster and that there is at least one slave cache cluster information in the cache configuration.

10. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-8.

11. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the method as described in any one of claims 1-8.

12. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-8.

Images