Data multi-copy caching method and device, computer storage medium, electronic equipment and computer program product

Abstract

The invention discloses a data multi-copy caching method and device, a computer storage medium, electronic equipment and a computer program product, and relates to the technical field of data caching. And determining a backup space of a far-end node corresponding to each node and a backup space address of the far-end node corresponding to each node based on a preset copy relationship, and in a data caching process, under the condition that home-end data is stored in a corresponding storage space, storing the home-end data in the corresponding storage space. And writing home terminal data of the node into a backup space of the remote node based on the backup space address of the corresponding remote node so as to perform multi-copy caching. According to the method, the storage spaces and the backup spaces of the nodes are pre-applied, the backup space corresponding to each node is mapped, and data memory writing is directly performed in the IO processing process, so that the processing steps in the IO processing process are reduced, the problem of relatively high IO processing time delay in related technologies is solved, and the data caching performance is greatly improved.

Description

Technical Field

[0001] The present application relates to the technical field of data caching, and in particular to a method for caching multiple copies of data, a device for caching multiple copies of data, a computer-readable storage medium, an electronic device, and a computer program product. Background Art

[0002] Data caching refers to temporarily storing data in memory or other high-speed storage media to facilitate quick access and reading of data. In order to increase the reliability and availability of data, in large-scale data centers and other distributed systems involving multi-node collaboration, multiple copies of data are usually used to save data between multiple nodes.

[0003] In the related technology, the IO (Input / Output) processing process needs to complete data mirroring through the coordination of multiple functional steps to achieve multi-copy caching of data, resulting in high IO processing latency. Summary of the invention

[0004] The present application provides a method and device for caching multiple copies of data, a computer storage medium, an electronic device, and a computer program product. Before executing IO processing, the storage space and backup space of the node are pre-applied and the backup space corresponding to each node is mapped, so that only the step of writing data to the memory space is executed during the IO processing, which reduces the processing steps in the IO processing process, solves the problem of high IO processing latency in related technologies, and greatly improves data caching performance.

[0005] The present application provides a multi-copy caching method for data, which is applied to a multi-node cluster. The multi-copy caching method for data includes: before data caching, determining storage space and at least one backup space of multiple nodes, wherein the storage space is used to store local data of the corresponding node; determining the backup space of a remote node corresponding to each node based on a preset copy relationship, wherein the backup space of the remote node is used to store backup data of the current node; performing information pre-interaction based on the preset copy relationship to determine the backup space address of the remote node corresponding to each node; during the data caching process, when there is a node's local data stored in the corresponding storage space, writing the node's local data into the corresponding remote node's backup space based on the corresponding remote node's backup space address to perform multi-copy caching of the node's local data.

[0006] The present application also provides a multi-copy caching device for data, which is applied to a multi-node cluster, and the multi-copy caching device for data includes: a first determination module, which is used to determine the storage space and at least one backup space of multiple nodes before data caching, wherein the storage space is used to store the local data of the corresponding node; a second determination module, which is used to determine the backup space of the remote node corresponding to each node based on a preset copy relationship, wherein the backup space of the remote node is used to store the backup data of the current node; a third determination module, which is used to perform information pre-interaction based on the preset copy relationship to determine the backup space address of the remote node corresponding to each node; and a control module, which is used to write the local data of the node into the backup space of the corresponding remote node during the data caching process when the local data of the node is stored in the corresponding storage space, so as to perform multi-copy caching of the local data of the node.

[0007] The present application also provides a computer-readable storage medium on which a multi-copy cache program for data is stored. When the multi-copy cache program for data is executed by a processor, the multi-copy cache method for data described above is implemented.

[0008] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned methods for caching multiple copies of data when executing the computer program.

[0009] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned multiple copy caching methods for data when the computer program is executed by a processor.

[0010] Through the present application, since the storage space and backup space of the node are pre-applied and the backup mapping relationship is established before executing IO processing, only the data writing step is executed during the IO processing. Therefore, the technical problem of high IO processing latency in related technologies can be solved, and the technical effect of improving data caching performance can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0012] Figure 1 A flowchart of a method for caching multiple copies of data provided in an embodiment of the present application; Figure 2 Schematic diagram of the spatial division of nodes in a specific embodiment of the present application Figure 1 ; Figure 3 Schematic diagram of the spatial division of nodes in a specific embodiment of the present application Figure 2 ; Figure 4 Schematic diagram of the spatial division of nodes in a specific embodiment of the present application Figure 3 ; Figure 5 A flowchart of a method for caching multiple copies of data according to a specific embodiment of the present application; Figure 6 A schematic diagram of the connection of a multi-copy cache device for data in an embodiment of the present application; Figure 7 A block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0014] It should be noted that, in the description of this application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0015] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0016] An embodiment of the present application provides a method for caching multiple copies of data. The method for caching multiple copies of data is applied to a multi-node cluster. The method is described in detail below in conjunction with the execution flow of the method for caching multiple copies of data.

[0017] Figure 1 The present invention is a flowchart of a method for caching multiple copies of data according to an embodiment of the present application.

[0018] like Figure 1 As shown, the multi-copy caching method for data in the embodiment of the present application is applied to a multi-node cluster, and the method may include: S1, before data caching, determining storage space of multiple nodes and at least one backup space, wherein the storage space is used to store local data of the corresponding node; S2, determining the backup space of the remote node corresponding to each node based on the preset replica relationship, wherein the backup space is used to store the backup data of the remote node corresponding to the current node; S3, pre-interacting information based on the preset replica relationship to determine the backup space address of the remote node corresponding to each node; S4, during the data caching process, when the local data of a node is stored in a corresponding storage space, the local data of the node is written into the backup space of the corresponding remote node based on the backup space address of the corresponding remote node, so as to cache multiple copies of the local data of the node.

[0019] Specifically, a multi-node cluster is composed of multiple independent computing nodes (usually servers or virtual machines) that are connected together through a network and work together to complete complex computing tasks or provide high-reliability services. The number of multi-node clusters can be set according to actual conditions, such as two, three, eight, or other positive integers.

[0020] Each node includes a storage space, which is used to store the local data of the corresponding node. The local data may include master data and metadata, where master data refers to a collection of raw and unorganized facts that can be used for calculation, reasoning or planning. It includes raw facts, observations, measurements or information representations in various formats, such as numbers, text, images, audio or video. Metadata is data about master data, that is, the description and context of master data. It organizes, finds and understands master data through information such as format, source, creation date, modification date, etc. Metadata helps locate, query, exchange, track and manage master data. Master data and corresponding metadata can be stored in one space at the same time, or they can be stored in different spaces. For example, the storage space is divided into a data area and a metadata area. The data area is used to store master data, and the metadata area is used to store metadata corresponding to the master data.

[0021] The backup space in the node is used to store the backup data of other nodes in the multi-node cluster. The backup data is a data mirror of the local data of the corresponding node, so as to realize multiple copies of data storage in conjunction with the local data of the corresponding node. Therefore, the backup data is the same as the local data of the corresponding node. The backup data also includes the main data and metadata. Then the main data and the corresponding metadata in the backup data can be stored in one space at the same time, or they can be stored in different spaces. For example, each backup space is divided into a data area and a metadata area. The data area is used to store the main data of the backup data, and the metadata area is used to store the metadata corresponding to the main data. In addition, in a multi-node cluster, the number of backup spaces of each node can be set to be the same or different, without specific restrictions, subject to meeting the number of copies.

[0022] by Figure 2 Taking the multi-node cluster shown as an example, the actual number of nodes is three, namely nodes Node1, Node2 and Node3. If three-copy cache is required, each node is divided into a storage space and two backup spaces. For example, the storage space in node Node1 is used to store the local data of node Node1, and the two backup spaces are used to store the backup data of node Node2 and node Node3 respectively; the storage space in node Node2 is used to store the local data of node Node2, and the two backup spaces are used to store the backup data of node Node1 and node Node3 respectively; the storage space in node Node3 is used to store the local data of node Node3, and the two backup spaces are used to store the backup data of node Node1 and node Node2 respectively.

[0023] The preset replica relationship is used to characterize the mapping relationship between each node and the backup space of the remote node, which can be determined according to the cluster size and number of replicas of the multi-node cluster. Figure 2In the illustrated embodiment, the node Node1 includes storage space A, backup space B0 and backup space C0, the node Node2 includes storage space B, backup space A0 and backup space C1, and the node Node3 includes storage space C, backup space A1 and backup space B1. Then the preset replica relationship may include a mapping relationship between the node Node1 and the backup space A0 of the node Node2 and the backup space A1 of the node Node3, that is, the backup data of the node Node1 is stored through the backup space A0 of the node Node2 and the backup space A1 of the node Node3. The mapping relationship between the node Node2 and the backup space B0 of the node Node1 and the backup space B1 of the node Node3, that is, the backup data of the node Node2 is stored through the backup space B0 of the node Node1 and the backup space B1 of the node Node3. The mapping relationship between the node Node3 and the backup space C0 of the node Node1 and the backup space C1 of the node Node2, that is, the backup data of the node Node3 is stored through the backup space C0 of the node Node1 and the backup space C1 of the node Node2. Therefore, the backup spaces of the remote nodes corresponding to the nodes Node1, Node2 and Node3 respectively can be queried according to the preset copy relationship, so as to be used for storing the backup data and realize multi-copy cache.

[0024] Pre-interaction information may include corresponding space addresses, access permission information, etc., to facilitate smooth data mirroring during IO processing. After the multi-node cluster is built, multiple nodes are connected through RDMA (remote direct memory access) links, and multiple nodes can establish communication links between multiple nodes through NVMe over Fabric (Non Volatile Memory Host Controller Interface Specification, a protocol for non-volatile memory to connect storage through a network structure) or other custom protocols. After multiple nodes determine the backup space of the corresponding remote node in the multi-copy caching process of the local data of the corresponding node based on the preset copy relationship, the corresponding space address and access permission information are exchanged according to the preset copy relationship to ensure the smooth multi-copy caching of subsequent data.

[0025] Continue with Figure 2Taking the multi-node cluster embodiment shown as an example, the target number of copies is three, then the three nodes Node1, Node2 and Node3 apply for one storage space and two backup spaces respectively, the storage space of each node is used to store the local data of the corresponding node, and the two backup spaces are used to store the backup data of the other two nodes respectively, and the main data and metadata are stored separately to form a data area and a metadata area. For example, in the node Node1, the storage space includes space A in the data area and space D in the metadata area, a backup space includes space B0 in the data area and space E0 in the metadata area, and a backup space includes space C0 in the data area and F0 in the metadata area. After the three nodes are connected, information pre-interaction is performed to establish a connection between the local data storage area and the backup data storage area between the three nodes, and inform each node of the corresponding relationship of the remote node. Taking the node Node1 as an example, in the information pre-interaction process, the corresponding backup space addresses can be obtained, which are the backup space addresses of the node Node2 (space A0, space D0) and the backup space addresses of the node Node3 (space A1, space D1), and the access rights are obtained. Therefore, during the data caching process, when the local data of node Node1 is stored in its storage space (space A, space D), the local data of Node1 can be directly written into the backup spaces corresponding to nodes Node2 and Node3 using RDMA technology, completing the three-copy caching of the local data of Node1. Therefore, after establishing a multi-node cluster, the corresponding data and metadata space addresses and related information of access rights are exchanged between multiple nodes, so that the subsequent multi-copy data caching work can proceed smoothly.

[0026] During IO processing, i.e., data caching, when the current node receives local data written into the storage space, for example, the local data can be written into the corresponding backup space according to the corresponding backup space address through RDMA technology to achieve mirroring of the local data and meet the need for multiple copies of the local data cache.

[0027] This embodiment is based on the pre-application of storage space and backup space of multiple nodes, and the establishment of a relationship between each node and its corresponding backup space, so that only the operation of writing data to the memory is performed during the IO processing process, reducing the processing steps in the IO processing process and greatly improving performance.

[0028] In one embodiment of the present application, the local data of the node is written into the backup space of the corresponding remote node based on the backup space address of the corresponding remote node, including: based on RDMA technology, the local data of the node is written into the backup space of the corresponding remote node according to the backup space address of the corresponding remote node.

[0029] Specifically, Figure 2Taking the embodiment shown as an example, it is assumed that the main data of the local data is written into the space A of the node Node1, and the metadata of the local data is written into the space D. At the same time, the main data written into the space A is written into the space A0 of the node Node2 and the space A1 of the node Node3 in the form of unicast or multicast through the RDMA technology, and the metadata written into the space D0 of the node Node2 and the space D1 of the node Node3, thereby realizing the three-copy cache of the local data of the node Node1. Among them, the RDMA technology can directly transfer data from the memory of one node to another node without the intervention of the operating systems of both parties. This transmission does not require the participation of the central processing unit, CPU cache or context exchange, and the transmission can be parallel with other system operations, thereby reducing delays and achieving fast message transmission.

[0030] In some embodiments of the present application, the local data of the node is written into the backup space of the corresponding remote node based on the backup space address of the corresponding remote node, including: when the number of remote nodes corresponding to the node is one, writing the local data into the backup space of the corresponding remote node in a unicast manner; when the number of remote nodes corresponding to the node is multiple, writing the local data into the backup spaces of multiple remote nodes in sequence in a unicast manner, or writing the local data into the backup spaces of multiple remote nodes at the same time in a multicast manner.

[0031] Specifically, the remote node corresponding to the node is the backup data storage node of the node. Figure 2 In the embodiment shown, the remote nodes corresponding to the node Node1 are Node2 and Node3, the remote nodes corresponding to the node Node2 are Node1 and Node3, and the remote nodes corresponding to the node Node3 are Node1 and Node2; Figure 3 In the illustrated embodiment, the remote node corresponding to the node Node1 is Node3, the remote node corresponding to the node Node2 is Node1, and the remote node corresponding to the node Node3 is Node2.

[0032] When the number of remote nodes corresponding to a node is one, the backup data is written in unicast mode. Figure 3 Taking the node Node1 shown as an example, the local data of the node Node1 includes primary data and metadata. When the primary data is written in the space A of the data area, the local metadata is generated and written into the space D of the metadata area. At the same time, the primary data written in the space A is written into the space A0 of the node Node3 in a unicast manner through the RDMA technology, and the metadata written in the space D is written into the space D0 of the node Node3, thereby storing the backup data of Node1 in the backup space of the node Node3.

[0033] When the number of remote nodes corresponding to the node is multiple, the backup data can be written in a unicast or multicast manner, without specific restrictions. Figure 2 Taking the node Node1 shown as an example, the local data of the node Node1 includes the main data and metadata. When the main data is written to the space A of the data area, the local metadata is generated and written to the space D of the metadata area. At the same time, the remote memory is operated through RDMA, and the main data written in the space A is written to the space A0 of the node Node2 and the space A1 of the node Node3 in a unicast manner, and the metadata written in the space D is written to the space D0 of the node Node2 and the space D1 of the node Node3 in a unicast manner, so that the backup data of Node1 is stored in the backup space of the node Node2 and the node Node3 respectively. Alternatively, the main data written in the space A is written to the space A0 of the node Node2 and the space A1 of the node Node3 at the same time in a multicast manner, and the metadata written in the space D is written to the space D0 of the node Node2 and the space D1 of the node Node3 at the same time, so that the backup data of Node1 is stored in the backup space of the node Node2 and the node Node3 respectively, so as to improve the data caching speed.

[0034] In addition, during the data writing process, each data is written in the space in sequence, and the new data information is written in the metadata interval in sequence. The mirror storage of backup data can be triggered based on the local data writing, or it can be performed according to a preset frequency, without specific restrictions.

[0035] In some embodiments of the present application, writing local data into the backup space of the corresponding remote node based on the backup space address of the corresponding remote node also includes: when receiving a backup feedback signal from the remote node, determining that the local data is successfully written into the backup space of the remote node.

[0036] That is to say, in the process of writing the local data of node Node1 into the backup space of node Node3, when the writing is successful, node Node3 gives a corresponding backup feedback signal. After receiving the backup feedback signal, node Node1 determines that the local data is successfully written into the backup space of node Node3 and completes the data backup. If the backup feedback signal given by node Node3 is not received within the preset time (which can be calibrated according to the actual situation) or the feedback data of node Node3 is not the correct backup feedback signal, it is considered that the data writing has failed, and it can be rewritten after a delay, or a reminder can be issued to ensure the accuracy of the multi-copy cache action.

[0037] In some embodiments of the present application, determining the storage space and backup space of multiple nodes includes: dividing the space of multiple nodes based on a preset number of nodes and a preset number of replicas to determine the initial storage space and initial backup space of the multiple nodes; after a multi-node cluster is established, obtaining the actual number of nodes and the target number of replicas of the multi-node cluster; adjusting the initial storage space and initial backup space of the multiple nodes based on the actual number of nodes and the target number of replicas to obtain the storage space and backup space of the multiple nodes.

[0038] Specifically, the preset number of nodes and the preset number of replicas may be set according to actual conditions, for example, based on a historical multi-node cluster design, or based on a multi-node cluster that is expected to be constructed.

[0039] First, space is pre-applied for multiple nodes according to the preset number of nodes and the preset number of replicas to determine the initial storage space and initial backup space of each node. For example, if the preset number of nodes is three and the preset number of replicas is three, then an initial storage space and two initial backup spaces of the same size can be applied for in the cache space of each node. Figure 2 It is understood that the preset number of nodes is only used for pre-application of space for the cache space of each node, and the number of nodes for pre-application of space may be different from the preset number of nodes. For example, based on the setting conditions that the preset number of nodes is three and the preset number of replicas is three, space pre-application is performed on eight nodes to apply for an initial storage space and two initial backup spaces of the same size for each of the eight nodes.

[0040] Then, the nodes after the space division are connected to the network to build a multi-node cluster. The number of multi-node clusters actually built may be different from the preset number of nodes, and the required multi-copy cache requirements may also deviate. Therefore, based on the actual number of nodes and the target number of copies of the built multi-node cluster, the initial storage space and initial backup space of each node are adjusted to obtain the storage space and backup space of multiple nodes that meet the target number of copies. For example, multiple nodes apply for an initial storage space and two backup spaces respectively. After the multi-node cluster is established, it is determined that the actual number of nodes is four and the target number of copies is two. Figure 3 As shown, the space of the three nodes that build the multi-node cluster can be adjusted to one storage space and one backup space respectively to meet the multi-copy cache requirements of the data in the multi-node cluster.

[0041] This embodiment pre-divides the space before nodes build a cluster, and then adjusts the space based on the actual number of nodes and the target number of replicas after building the multi-node cluster, so as to speed up the space construction speed of multiple nodes in the multi-node cluster.

[0042] In some embodiments of the present application, the initial storage space and initial backup space of multiple nodes are adjusted based on the actual number of nodes and the target number of replicas, including: using the initial storage space as the storage space; determining the backup space requirements of multiple nodes based on the actual number of nodes and the target number of replicas; and adjusting the initial backup space based on the backup space requirements to determine the backup space for multiple nodes.

[0043] Specifically, the storage space requirements and backup space requirements of multiple nodes can be determined based on the actual number of nodes and the target number of copies in the multi-node cluster, and then the initial storage space of multiple nodes is adjusted based on the storage space requirements, and the backup space of multiple nodes is adjusted based on the backup space requirements.

[0044] In the process of dividing the space of multiple nodes based on the preset number of nodes and the preset number of replicas, each node applies for an initial storage space and multiple backup storage spaces. After being applied to the construction of a multi-node cluster, it can be determined that the storage space requirements of multiple nodes are all one based on the actual number of nodes and the target number of replicas. In order to speed up the space adjustment, the initial storage space of multiple nodes is directly used as the storage space, and the initial backup space is only adjusted based on the determined number of backup space requirements.

[0045] In the process of adjusting the initial storage space and initial backup space of multiple nodes according to the actual number of nodes and the target number of copies, the pre-divided initial storage space is directly used as the storage space, and the initial backup space is adjusted according to the backup space requirement determined by the actual number of nodes and the target number of copies, so as to avoid the waste of cache space while satisfying the multi-copy cache of data. For example, assuming that the initial backup space of node Node1 is two, and the backup space requirement of node Node1 is determined to be one according to the actual number of nodes and the target number of copies, then one initial backup space in node Node1 can be released, and the other initial backup space can be used as the backup space for subsequent multi-copy cache of data; if the backup space requirement of node Node1 is determined to be three according to the actual number of nodes and the target number of copies, then another backup space of node Node1 is expanded to obtain three backup spaces to satisfy the subsequent multi-copy cache of data.

[0046] This embodiment only requires space adjustment of the initial backup space, which reduces the amount of space adjustment tasks and speeds up the division of storage space and backup space of multiple nodes.

[0047] In some embodiments of the present application, the storage space and backup space of each node are determined, including: after a multi-node cluster is established, obtaining the actual number of nodes and the target number of replicas of the multi-node cluster; dividing the space of multiple nodes according to the actual number of nodes and the target number of replicas to obtain the storage space and backup space of multiple nodes.

[0048] That is to say, in addition to the above-mentioned method of pre-dividing the space of multiple nodes based on the preset number of nodes and the preset number of copies, and then adjusting the space according to the actual number of nodes and the target number of copies in the multi-node cluster where the nodes are located to determine the storage space and backup space strategy of multiple nodes, it is also possible to directly apply for storage space and backup space for multiple nodes based on the actual number of nodes and the target number of copies after the multi-node cluster is established to obtain storage space and backup space for multiple nodes to meet the data multi-copy caching requirements.

[0049] In some embodiments of the present application, the storage space includes a first storage subspace and a second storage subspace, the backup space includes a first backup subspace and a second backup subspace, and the multi-copy cache method of the data further includes: when the remaining space of the first storage subspace and / or the first backup subspace of the node is less than a first preset threshold, storing the local data in the second storage subspace of the corresponding node, and storing the backup data in the second backup subspace of the corresponding node. The first preset threshold can be set according to actual conditions.

[0050] Specifically, the first storage subspace corresponds to the second storage subspace, and both are used to store local data of the node where they are located; the first backup subspace corresponds to the second backup subspace, and both are used to store backup data.

[0051] by Figure 2 Taking the node Node1 as an example, it includes one storage space and two backup spaces. In the storage space, the first storage subspace includes space A and space D, and the second storage subspace includes space a and space d; in one backup space, the first backup subspace includes space B0 and space E0, and the second backup subspace includes space b0 and space e0; in another backup space, the first backup subspace includes C0 and space F0, and the second backup subspace includes space c0 and space f0.

[0052] First, the first storage subspace and the two first backup subspaces located in the upper half perform data caching. Specifically, the local data of node Node1 is stored through space A and space D, the backup data of node Node2 is stored through space B0 and space E0, and the backup data of node Node3 is stored through space C0 and space F0.

[0053] In the data caching process, it can be determined whether the data cache space is insufficient based on the remaining space of the first storage subspace, or the first backup subspace, or the remaining space of the first storage subspace and the remaining space of the second backup subspace. When it is determined that the data cache space is insufficient, the first storage subspace and the second storage subspace are spatially converted, and the first backup subspace and the second backup subspace are spatially converted, that is, the local data of node Node1 continues to be stored through space a and space d, the backup data of node Node2 continues to be stored through space b0 and space e0, and the backup data of node Node3 continues to be stored through space c0 and space f0.

[0054] For example, the remaining space of the first storage subspace is monitored in real time, and when it is determined that the remaining space of the first storage subspace is less than the first preset threshold, it is considered that there is insufficient data cache space; or, the remaining space of one of the two first backup subspaces is monitored in real time, and when it is determined that the remaining space is less than the first preset threshold, it is considered that there is insufficient data cache space. The total remaining space of the two first backup subspaces can also be monitored in real time, and when it is determined that the total remaining space of the two first backup subspaces is less than the first preset threshold, it is considered that there is insufficient data cache space; or, the total remaining space of the first storage subspace and the first backup subspace can be monitored in real time, and when it is determined that the total remaining space is less than the first preset threshold, it is considered that there is insufficient data cache space. The remaining space of each subspace can also be compared with the first preset threshold, and when the remaining space of the three subspaces is less than the first preset threshold, it is considered that there is insufficient data cache space.

[0055] In addition, when the primary data and metadata are stored separately, each storage subspace and backup subspace includes a cache subspace for the primary data and a cache subspace for the metadata, then the remaining space of one of the cache subspaces can be used as the remaining space of the storage subspace and the backup subspace. For example, the first storage subspace of the node Node1 includes space A for storing primary data and space D for storing metadata, then the remaining space of space A can be monitored as the remaining space of the first storage subspace.

[0056] During the data caching process, this embodiment determines whether the first storage subspace and the first backup subspace are full through real-time monitoring of the remaining space. When it is determined that the space is full, that is, there is insufficient cache space, the second storage subspace and the second backup subspace respectively take over the functions of the first storage subspace and the first backup subspace to ensure that the data caching work is carried out continuously and stably.

[0057] In some embodiments of the present application, the multi-copy caching method of data also includes: based on a flush operation, flushing the local data in the first storage subspace to the first target storage unit, and flushing the backup data in the first backup subspace to the second target storage unit, and clearing the data in the first storage subspace and the first backup subspace after the flush operation is completed.

[0058] Specifically, the flush operation refers to the process of writing data from memory or other temporary storage media to persistent storage (such as a disk).

[0059] When it is determined that the cache space is insufficient based on the remaining space of the first storage subspace and / or the first backup subspace, the second storage subspace and the second backup subspace are used to take over the function, so as to continue to cache data through the second storage subspace and the second backup subspace, and at the same time, the local data in the first storage subspace is flushed to the first target storage unit, and the backup data in the first backup subspace is flushed to the second target storage unit, wherein the first target storage unit and the second target storage unit can be the same storage unit or different storage units, and there is no specific limitation. After the flushing operation is completed, the data in the first storage subspace and the first backup subspace are cleared to continue to be used for data caching.

[0060] In this embodiment, the local data and backup data are first written into the cache space to quickly respond to the write operation and avoid the delay caused by direct writing to the disk. At the same time, when the remaining space of the first storage subspace and the first backup subspace is insufficient, the data in the first storage subspace and the first backup subspace are continuously cached through the second storage subspace and the second backup subspace, and the data in the first storage subspace and the first backup subspace are flushed to the disk or other persistent storage to ensure the consistency and integrity of the data. The asynchronous flushing operation adopted in this embodiment provides significant performance advantages for high concurrency and I / O intensive scenarios by optimizing resource utilization, reducing latency and improving throughput.

[0061] In some embodiments of the present application, the multi-copy cache method of the data further includes: when the remaining space of the second storage subspace and / or the second backup subspace corresponding to the node is less than the second preset threshold and the first storage subspace and the first backup subspace corresponding to the node complete the flush operation, the local data is stored in the first storage subspace of the corresponding node, and the backup data is stored in the first backup subspace of the corresponding node. The second preset threshold can be set according to actual conditions.

[0062] Specifically, continue with Figure 2For example, when storing the local data of node Node1 through space a and space d, storing the backup data of node Node2 through space b0 and space e0, and storing the backup data of node Node3 through space c0 and space f0, it is determined whether the cache space is full based on the remaining space of the second storage subspace (space a and space d) and / or the second backup subspace (space b0 and space e0, space c0 and space f0), that is, whether there is insufficient cache space. For example, the remaining space of the second storage subspace is monitored in real time, and when it is determined that the remaining space of the first storage subspace is less than the second preset threshold, it is considered that there is insufficient data cache space; or, the remaining space of one of the two second backup subspaces is monitored in real time, and when it is determined that the remaining space is less than the second preset threshold, it is considered that there is insufficient data cache space. The total remaining space of the two second backup subspaces can also be monitored in real time, and when it is determined that the total remaining space of the two second backup subspaces is less than the first preset threshold, it is considered that there is insufficient data cache space; or, the total remaining space of the second storage subspace and the second backup subspace is monitored in real time, and when it is determined that the total remaining space is less than the second preset threshold, it is considered that there is insufficient data cache space. The remaining space of each subspace can also be compared with the second preset threshold, and when the remaining space of the three subspaces is less than the second preset threshold, it is considered that there is insufficient data cache space.

[0063] In addition, during data caching based on the second storage subspace and the second backup subspace, the data in the first storage subspace and the first backup subspace are synchronously flushed to a storage unit (such as a disk) based on a flush operation, and the data is cleared after the data flush is completed.

[0064] When it is determined that the cache space is insufficient and the first storage subspace and the first backup subspace have completed the flushing operation, the second storage subspace, the second backup subspace and the first storage subspace and the second backup subspace after the data is cleared are converted to continue data caching through the first storage subspace and the second backup subspace, while the second storage subspace and the second backup subspace perform data flushing. That is, the local data of node Node1 is stored through space A and space B, the backup data of node Node2 is stored through space B0 and space E0, and the backup data of node Node3 is stored through space C0 and space F0.

[0065] In this embodiment, during the data caching process in the second storage subspace and the second backup subspace, real-time monitoring of the remaining space is used to determine whether there is insufficient cache space in the second storage subspace and the second backup subspace. When it is determined that the cache space is insufficient and the first storage subspace and the first backup subspace have completed the flushing operation, the first storage subspace and the first backup subspace after the data is cleared are used to receive new data, and data caching is continued. The flushing operation is performed on the full second storage subspace and the second backup subspace, and this cycle is repeated continuously to realize an efficient cache mirroring mechanism.

[0066] In some embodiments of the present application, the multi-copy caching method for data also includes: in the event of a node failure, determining a target remote node based on the failed node and a preset copy relationship; and storing the local data of the failed node based on the backup space of the target remote node.

[0067] Specifically, the target remote node is the node that takes over the local data of the faulty node. Specifically, the remote node corresponding to the faulty node can be found based on the preset copy relationship, and a remote node corresponding to the faulty node is used as the target remote node. Since the remote node corresponding to the faulty node stores the backup data of the faulty node, it continues to store the local data of the faulty node as the target remote node, which can ensure the cache continuity and consistency of the local data of the faulty node.

[0068] For example, in Figure 2 In the illustrated embodiment, assuming that the faulty node is node Nope1, node Nope2 can be determined as the target remote node of the faulty node based on the preset copy relationship, and the local data of the faulty node A is received and stored through the backup space in node Nope2 for storing the backup data of node Nope1. At the same time, based on the data flush in node Noe2, the stored local data of the faulty node A is flushed to the storage unit to ensure the continuity and consistency of the data storage of node Nope1.

[0069] In addition, when the backup space of the target remote node stores the local data of the failed node, it is also necessary to establish a copy relationship between the backup space of the target remote node and other remote nodes corresponding to the failed node. Figure 2In the illustrated embodiment, the faulty node is node Nope1, and node Nope2 is the target remote node. The backup space (space A0 and space C0) of node Nope2 stores the local data of node Nop1, and establishes a copy relationship between the backup space (space A0 and space C0) of node Nope2 and the backup space (space A1 and space C1) of node Nope3, so that when the local data of node Nope1 is stored in the backup space (space A0 and space C0) of node Nope2, the local data of node Nope1 is written into the backup space (space A1 and space C1) of node Nope3 based on RDMA technology, thereby completing the multi-copy cache of the local data of node Nope1.

[0070] In some embodiments of the present application, a target remote node is determined based on a faulty node and a preset replica relationship, including: determining the number of remote nodes corresponding to the faulty node based on the faulty node and the preset replica relationship; when the number of remote nodes is one, using the remote node as the target remote node; when the number of remote nodes is multiple, determining the target remote node based on a preset priority.

[0071] Specifically, in Figure 2 In the embodiment shown, it is assumed that the faulty node is node Node1, and the remote nodes used to store the backup data of node Node1 are determined to be Node2 and Node3 according to the preset copy relationship, and the number of remote nodes is 2. If the priority of node Node2 is higher than that of node Node3 according to the preset priority, then node Node2 is used as the target remote node of the faulty node, and the local data of the faulty node Node1 is stored in the backup space (space A0 and space C0) of node Node2, that is, the master data of the local data of node Node1 is stored in space A0, and the metadata of the local data of node Node1 is stored in space D0. The preset priority can be set according to the actual situation.

[0072] exist Figure 3 In the illustrated embodiment, the remote node for storing the replica data of the node Node1 is determined to be Node3 according to the preset replica relationship, and the remote node data volume is 1, then the node Node3 is the target remote node of the faulty node, and the local data of the faulty node Node1 is stored in the backup space of the node Node3, that is, the master data of the local data of the node Node1 is stored in the space A0 in the node Node3, and the metadata of the local data of the node Node1 is stored in the space D0.

[0073] In some embodiments of the present application, the local data includes master data and metadata, and the local data of the faulty node is stored based on the backup space of the target remote node, including: establishing an index structure of the faulty node based on the master data and metadata of the faulty node stored in the backup space of the target remote node, so that the master data of the faulty node is in an activated state, which facilitates the reading of the local data of the faulty node.

[0074] Specifically, the index structure is a data structure used to accelerate database query operations. It improves database query performance and reduces query time by optimizing data storage and access methods. It can be implemented using data structures such as B-tree, B+ tree, and hash table.

[0075] by Figure 2 For example, node Node2 is the target remote node of node Node1. When it is determined that node Node1 fails and cannot perform data caching, the backup data of node Node1 stored in node Node2 is started. Specifically, an index structure is first established based on the main data in space A0 and the metadata in space D0 to restore the data content and control logic based on the metadata area of ​​the backup data, thereby restoring the upper and lower tasks to operate. Among them, the upper task is a data caching task executed based on the upper space, and the lower task is a data flushing task executed based on the lower space.

[0076] In some embodiments of the present application, in the event of a node failure, the multi-copy caching method for the data also includes: if the actual number of nodes is greater than the target number of copies, determining a replacement node for the failed node based on a preset copy relationship; and storing backup data of the remote node corresponding to the failed node based on the backup space of the replacement node.

[0077] Specifically, when there is a faulty node in a multi-node cluster, the backup space of the target remote node takes over the storage and flushes the primary data of the faulty node. At the same time, as the faulty node fails, its backup space is also in a failed state, and a replacement node needs to be determined to take over the storage and flush the backup data stored in the faulty node to meet the multi-copy cache requirements.

[0078] exist Figure 2In the multi-node cluster embodiment shown, the actual number of nodes is three, the target number of copies is three, and assuming that node Node1 fails, the backup space of node Node2 takes over the local data of node Node1 to store and refresh the local data of node Node1. At the same time, with the failure of node Node1, its backup space is also in a failed state, and it is necessary to determine a replacement node to take over the backup data stored in node Node1, that is, to take over the backup data of nodes Node2 and Node3 in the backup space of node Node1, but both nodes Node2 and Node3 have duplicate data with the backup data of node Node1, so there is no replacement node for the failed node in the multi-node cluster.

[0079] exist Figure 3 In the multi-node cluster embodiment shown, the actual number of nodes is three, and the target number of copies is two. Assuming that node Node1 fails, the backup space of node Node3 is used to perform backup on the local data of node Node1. At the same time, new backup space can be added through node Node3 to take over the backup data of node Node2 stored in node Node1. At this time, a copy relationship is established between node Node2 and node Node3. In the subsequent data caching process, the backup data of node Node2 is stored through the backup space of node Node3 to maintain two copies of the local data of node Node2.

[0080] In this embodiment, when a node fails, if the number of nodes in the multi-node cluster is greater than the preset number of replicas, there may be a replacement node for the failed node. At this time, the backup data of the remote node corresponding to the failed node is stored based on the backup space of the replacement node to maintain the multi-copy cache relationship of the node.

[0081] In some embodiments of the present application, a replacement node for a failed node is determined based on a preset replica relationship, including: determining a replacement node set of remote nodes that do not store the failed node in multiple nodes based on the failed node and the preset replica relationship; and determining a replacement node based on a preset rule and the replacement node set.

[0082] Specifically, Figure 4 Taking the four-node cluster in as an example, the target number of replicas is two. Assuming that the failed node is node Node1, according to the preset replica relationship, the backup data stored in the backup space of node Node1 is the backup data of node Node2, and node Node3 and node Node4 do not store the backup data of node Node2, then the replacement node set is determined to be {node Node3, node Node4}.

[0083] The preset rule may be a preset priority or a preset mapping table, etc., which is not specifically limited. Assuming that the preset rule is to determine the replacement node based on the node code in ascending order, then determine the node Node3 in the replacement node set {node Node3, node Node4} as the replacement node for the failed node Node1. Specifically, a spare space may be added to Node3, and a copy relationship between the spare space and the node Node2 may be established, so that the backup data of the node Node2 is stored through the backup space of the node Node3, so that the local data of the node Node2 is kept in two copies of the cache.

[0084] In some embodiments of the present application, the multi-copy caching method of the data also includes: determining a set of nodes in multiple nodes that do not store backup data of the faulty node based on the faulty node and a preset copy relationship; determining a target node based on preset rules and the node set, and establishing a copy relationship between the target node and a target remote node; and backing up the local data of the faulty node stored in the target remote node based on the copy relationship and the backup space of the target node.

[0085] Specifically, continue with Figure 4 For example, the faulty node is Node1. According to the preset replica relationship, the local data of Node1 is backed up to Node4. Node4, as the target remote node, takes over the local data of the faulty node Node1. Node2 and Node3 do not store the backup data of Node1, so the node set is {Node2, Node3}.

[0086] The preset rule may be a preset priority or a preset mapping table, etc., which is not specifically limited. Assuming that the preset rule is to determine the replacement node based on the node code in descending order, then determine the node Node3 in the node set {node Node2, node Node3} as the target node of the faulty node Node1. Specifically, a spare space may be added to the node Node3, and a copy relationship may be established between the spare space of the node Node3 and the node Node4, so as to back up the local data of the node Nope1 stored in the space A0 and space D0 of the node Node4 to the node Node3, so as to realize the two-copy cache of the local data of the faulty node Node1.

[0087] As a specific embodiment of the present application, Figure 5 As shown, the method for caching multiple copies of data may include the following steps: S101, after the multi-node cluster is established, obtaining the actual number of nodes and the target number of replicas of the multi-node cluster; S102, dividing the node space according to the actual number of nodes and the target number of replicas to obtain storage space and backup space of multiple nodes; S103, determining a preset replica relationship based on the cluster size and the target number of replicas; S104, determining the backup space of the remote node corresponding to each node based on the preset replica relationship; S105, establishing a communication connection between multiple nodes, and performing information pre-interaction based on a preset replica relationship to determine a backup space address of a remote node corresponding to each node; S106, during the data caching process, when there is local data of the node stored in the corresponding storage space, the local data of the node is written into the backup space of the corresponding remote node according to the backup space address of the corresponding remote node, so as to cache multiple copies of the local data of the node; In addition, the storage space includes a first storage subspace and a second storage subspace, and the backup space includes a first backup subspace and a second backup subspace. The first storage subspace and the first backup subspace are in the upper half of the space, and the second storage subspace and the second backup subspace are in the lower half of the space. In the data caching process, data is first cached through the first storage subspace and the first backup subspace. When the cache space of the first storage subspace and the first backup subspace is insufficient, data is cached through the second storage subspace and the second backup subspace, and the data in the first storage subspace and the first backup subspace is flushed down, and the cycle is repeated to achieve continuous caching of data.

[0088] S107, determine whether the failed node is stored. If yes, execute step S108; if no, continue to execute step S106; S108, determining a target remote node based on the relationship between the failed node and the preset replica; S109, storing the local data of the failed node based on the backup space of the target remote node; S110, determining a replacement node for the failed node based on a preset replica relationship; S111, storing backup data of a remote node corresponding to the failed node based on the backup space of the replacement node; S112, determining a set of nodes that do not store backup data of the failed node among the multiple nodes based on the failed node and the preset replica relationship; S113, determining a target node based on a preset rule and a node set, and establishing a replica relationship between the target node and a target remote node; S114, backing up the local data of the failed node stored in the target remote node based on the replica relationship and the backup space of the target node.

[0089] Therefore, in the multi-node communication scenario, this embodiment realizes multi-copy mirroring in the cache mirroring process through pre-application of memory space, data mirroring, and switching of the upper and lower halves of the space, removes the participation and processing of the remote node CPU in the IO processing process, reduces the processing steps in the IO processing process, and greatly improves the performance. The implementation of this method can be divided into five parts: one part is that after the current node is started, it applies for local data, metadata memory and remote data and metadata memory for backup; the second part is that the current node establishes a connection with other nodes in the cluster and pre-interacts with memory; the third part is that after the data falls into the cache, RDMA is used to implement the multi-copy backup process; the fourth part is that after the data space is full, it is converted into the lower flush space, and after the lower flush space data is flushed, it is converted into the new space of the upper half; the fifth part is that when a node fails, other nodes can restore the behavior of the upper and lower halves of the cache based on the data area and metadata area.

[0090] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0091] Therefore, according to the multi-copy caching method of data in the embodiment of the present application, before data caching, first determine the storage space and at least one backup space of multiple nodes, wherein the storage space is used to store the local data of the corresponding node, and determine the backup space of the remote node corresponding to each node based on the preset copy relationship, wherein the backup space of the remote node is used to store the backup data of the current node, and pre-interact information based on the preset copy relationship to determine the backup space address of the remote node corresponding to each node, and then in the data caching process, in the case where the local data of the node is stored in the corresponding storage space, the local data of the node is written to the backup space of the corresponding remote node based on the backup space address of the corresponding remote node, so as to perform multi-copy caching of the local data of the node. Therefore, the method pre-applies for the storage space and backup space of the node and maps the backup space corresponding to each node before executing IO processing, so that only the step of writing data into the memory space is executed during the IO processing, reducing the processing steps in the IO processing process, solving the problem of high IO processing delay in the related technology, and greatly improving the data caching performance.

[0092] The embodiment of the present application also provides a multi-copy cache device for data, the multi-copy cache device for data is applied to a multi-node cluster, such as Figure 6 As shown, the multi-copy cache device for data may include: a first determination module 10 , a second determination module 20 , a third determination module 30 and a data cache module 40 .

[0093] Among them, the first determination module 10 is used to determine the storage space and at least one backup space of multiple nodes before data caching, wherein the storage space is used to store the local data of the corresponding node. The second determination module 20 is used to determine the backup space of the remote node corresponding to each node based on the preset copy relationship, wherein the backup space of the remote node is used to store the backup data of the current node. The third determination module 30 is used to perform information pre-interaction based on the preset copy relationship to determine the backup space address of the remote node corresponding to each node. The data caching module 40 is used to write the local data of the node into the backup space of the corresponding remote node based on the backup space address of the corresponding remote node during the data caching process when the local data of the node is stored in the corresponding storage space, so as to cache multiple copies of the local data of the node.

[0094] According to some embodiments of the present application, the data cache module 40 writes the local data of the node into the backup space of the corresponding remote node based on the backup space address of the corresponding remote node, and is specifically used for: writing the local data of the node into the backup space of the corresponding remote node according to the backup space address of the corresponding remote node based on RDMA technology.

[0095] According to some embodiments of the present application, the data cache module 40 writes the local data of the node into the backup space of the corresponding remote node based on the backup space address of the corresponding remote node, and is specifically used for: when the number of remote nodes corresponding to the node is one, writing the local data to the backup space of the corresponding remote node in a unicast manner; when the number of remote nodes corresponding to the node is multiple, writing the local data to the backup spaces of multiple remote nodes in sequence in a unicast manner, or writing the local data to the backup spaces of multiple remote nodes at the same time in a multicast manner.

[0096] According to some embodiments of the present application, the data cache module 40 writes the local data into the backup space of the corresponding remote node based on the backup space address of the corresponding remote node, and is also used to: when receiving a backup feedback signal from the remote node, determine that the local data is successfully written into the backup space of the remote node.

[0097] According to some embodiments of the present application, the first determination module 10 determines the storage space and backup space of multiple nodes, and is specifically used to: divide the space of multiple nodes based on a preset number of nodes and a preset number of replicas to determine the initial storage space and initial backup space of multiple nodes; after a multi-node cluster is established, obtain the actual number of nodes and the target number of replicas of the multi-node cluster; adjust the initial storage space and initial backup space of multiple nodes based on the actual number of nodes and the target number of replicas to obtain the storage space and backup space of multiple nodes.

[0098] According to some embodiments of the present application, the first determination module 10 performs space adjustments on the initial storage space and the initial backup space of multiple nodes based on the actual number of nodes and the target number of replicas, and is specifically used to: use the initial storage space as the storage space; determine the backup space requirement for each node based on the actual number of nodes and the target number of replicas; and adjust the space quantity of the initial backup space based on the backup space requirement to determine the backup space for multiple nodes.

[0099] According to some embodiments of the present application, the first determination module 10 determines the storage space and backup space of multiple nodes, and is specifically used to: after the multi-node cluster is established, obtain the actual number of nodes and the target number of copies of the multi-node cluster; divide the space of multiple nodes according to the actual number of nodes and the target number of copies to obtain the storage space and backup space of multiple nodes.

[0100] According to some embodiments of the present application, the storage space includes a first storage subspace and a second storage subspace, the backup space includes a first backup subspace and a second backup subspace, and the data cache module 40 is also used to: when the remaining space of the first storage subspace and / or the first backup subspace of the node is less than a first preset threshold, store the local data in the second storage subspace of the corresponding node, and store the backup data in the second backup subspace of the corresponding node.

[0101] According to some embodiments of the present application, the data cache module 40 is also used to: flush the local data in the first storage subspace to the first target storage unit based on the flush operation, and flush the backup data in the first backup subspace to the second target storage unit, and clear the data in the first storage subspace and the first backup subspace after the flush operation is completed.

[0102] According to some embodiments of the present application, the data cache module 40 is also used to: when the remaining space of the second storage subspace and / or the second backup subspace corresponding to the node is less than a second preset threshold and the first storage subspace and the first backup subspace corresponding to the node complete the flush operation, store the local data in the first storage subspace of the corresponding node, and store the backup data in the first backup subspace of the corresponding node.

[0103] According to some embodiments of the present application, the data cache module 40 is also used to: in the event of a node failure, determine a target remote node based on the failed node and a preset replica relationship; and store the local data of the failed node based on the backup space of the target remote node.

[0104] According to some embodiments of the present application, the data cache module 40 determines the target remote node based on the faulty node and the preset replica relationship, and is specifically used to: determine the number of remote nodes corresponding to the faulty node based on the faulty node and the preset replica relationship; when the number of remote nodes is one, use the remote node as the target remote node; when the number of remote nodes is multiple, determine the target remote node based on the preset priority.

[0105] According to some embodiments of the present application, the local data includes master data and metadata, and the data cache module 40 stores the local data of the faulty node based on the backup space of the target remote node, and is specifically used to: establish an index structure of the faulty node based on the master data and metadata of the faulty node stored in the backup space of the target remote node, so that the master data of the faulty node is in an activated state, which facilitates the reading of the local data of the faulty node.

[0106] According to some embodiments of the present application, in the event of a node failure, the data cache module 40 is also used to: if the number of nodes is greater than the preset number of copies, determine a replacement node for the failed node based on the preset copy relationship; and store backup data of the remote node corresponding to the failed node based on the backup space of the replacement node.

[0107] According to some embodiments of the present application, the data cache module 40 determines a replacement node for a failed node based on a preset replica relationship, including: determining a replacement node set of remote nodes corresponding to the failed node that are not stored in multiple nodes based on the actual failed node and the target replica relationship; and determining a replacement node based on preset rules and a replacement node set.

[0108] According to some embodiments of the present application, the data cache module 40 is also used to: determine a set of nodes in multiple nodes that do not store backup data of the faulty node based on the faulty node and a preset replica relationship; determine a target node based on preset rules and a set of nodes, and establish a replica relationship between the target node and a target remote node; and back up the local data of the faulty node stored in the target remote node based on the replica relationship and the backup space of the target node.

[0109] For the description of the features in the embodiments corresponding to the multi-copy cache device for data, reference may be made to the relevant description of the embodiments corresponding to the multi-copy cache method for data, which will not be described in detail here.

[0110] According to the multi-copy cache device of data of the embodiment of the present application, before data caching, the storage space and at least one backup space of multiple nodes are determined by the first determination module, wherein the storage space is used to store the local data of the corresponding node, the backup space of the remote node corresponding to each node is determined by the second determination module based on the preset copy relationship, wherein the backup space of the remote node is used to store the backup data of the current node, and the information pre-interaction is performed based on the preset copy relationship by the third determination module to determine the backup space address of the remote node corresponding to each node, and in the data process, the data cache module writes the local data of the node into the backup space of the corresponding remote node when the local data of the node is stored in the corresponding storage space, so as to cache the local data of the node in multiple copies. Thus, the device pre-applies for the storage space and backup space of the node and establishes the backup mapping relationship before executing IO processing, so that only the data writing step is executed during the IO processing, reducing the processing steps during the IO processing, solving the problem of high IO processing delay in the related technology, and greatly improving the data cache performance.

[0111] An embodiment of the present application further provides a computer-readable storage medium on which a multi-copy cache program for data is stored. When the multi-copy cache program for data is executed by a processor, the multi-copy cache method for data described above is implemented.

[0112] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0113] The embodiment of the present application also provides an electronic device, such as Figure 7 As shown, the electronic device 100 of the embodiment of the present application includes a memory 110 and a processor 120, wherein the memory 110 stores a computer program, and the processor 120 is configured to run the computer program to execute the steps in any of the above-mentioned data multi-copy caching method embodiments.

[0114] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, implementing the steps in any of the above-mentioned multiple-copy caching method embodiments of data.

[0115] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0116] The above is a detailed introduction to a method and device for caching multiple copies of data, a computer storage medium, an electronic device, and a computer program product provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A method for caching multiple copies of data, characterized in that: Applied to a multi-node cluster, the method comprises: Before data caching, determine storage spaces of multiple nodes and at least one backup space, wherein the storage space is used to store local data of the corresponding node; Determine the backup space of the remote node corresponding to each node based on the preset replica relationship, wherein the backup space of the remote node is used to store the backup data of the current node; Perform information pre-interaction based on the preset replica relationship to determine the backup space address of the remote node corresponding to each node; During the data caching process, when the local data of a node is stored in the corresponding storage space, the local data of the node is written to the backup space of the corresponding remote node based on the backup space address of the corresponding remote node to cache multiple copies of the local data of the node.

2. The method for caching multiple copies of data according to claim 1, characterized in that: The step of writing the local data of the node into the backup space of the corresponding remote node based on the backup space address of the corresponding remote node includes: Based on the RDMA technology, the local data of the node is written into the backup space of the corresponding remote node according to the backup space address of the corresponding remote node.

3. The method for caching multiple copies of data according to claim 1, characterized in that: The step of writing the local data of the node into the backup space of the corresponding remote node includes: When the number of remote nodes corresponding to the node is one, writing the local data into the backup space of the corresponding remote node in a unicast manner; When the number of remote nodes corresponding to the node is multiple, the local data is written into the backup spaces of the multiple remote nodes in sequence in a unicast manner, or the local data is written into the backup spaces of the multiple remote nodes simultaneously in a multicast manner.

4. The method for caching multiple copies of data according to claim 3, characterized in that: The step of writing the local data into the backup space of the corresponding remote node further includes: When the backup feedback signal of the remote node is received, it is determined that the local data is successfully written into the backup space of the remote node.

5. The method for caching multiple copies of data according to claim 1, characterized in that: The determining of the storage space and the backup space of the plurality of nodes includes: Based on a preset number of nodes and a preset number of replicas, the space of the multiple nodes is divided to determine the initial storage space and the initial backup space of the multiple nodes; After the multi-node cluster is established, obtaining the actual number of nodes and the target number of replicas of the multi-node cluster; Based on the actual number of nodes and the target number of replicas, initial storage space and initial backup space of multiple nodes are adjusted to obtain storage space and backup space of multiple nodes.

6. The method for caching multiple copies of data according to claim 5, characterized in that: The adjusting the initial storage space and the initial backup space of each node based on the actual number of nodes and the target number of replicas includes: Using the initial storage space as the storage space; Determine the backup space requirement of each node according to the actual number of nodes and the target number of replicas; The initial backup space is adjusted in space quantity according to the backup space requirement quantity to determine the backup space of each node.

7. The method for caching multiple copies of data according to claim 1, characterized in that: The determining of the storage space and the backup space of the plurality of nodes includes: After the multi-node cluster is established, obtaining the actual number of nodes and the target number of replicas of the multi-node cluster; The multiple nodes are spatially divided according to the actual number of nodes and the target number of replicas to obtain storage space and backup space of the multiple nodes.

8. The method for caching multiple copies of data according to any one of claims 1 to 7, characterized in that: The storage space includes a first storage subspace and a second storage subspace, the backup space includes a first backup subspace and a second backup subspace, and the method further includes: When the remaining space of the first storage subspace and / or the first backup subspace of a node is less than a first preset threshold, the local data is stored in the second storage subspace of the corresponding node, and the backup data is stored in the second backup subspace of the corresponding node.

9. The method for caching multiple copies of data according to claim 8, characterized in that: The method further comprises: Based on the flush operation, the local data in the first storage subspace is flushed to the first target storage unit, and the backup data in the first backup subspace is flushed to the second target storage unit, and after the flush operation is completed, the data in the first storage subspace and the first backup subspace are cleared.

10. The method for caching multiple copies of data according to claim 9, characterized in that: The method further comprises: When the remaining space of the second storage subspace and / or the second backup subspace corresponding to the node is less than a second preset threshold and the first storage subspace and the first backup subspace corresponding to the node complete the flush operation, the local data is stored in the first storage subspace of the corresponding node, and the backup data is stored in the first backup subspace of the corresponding node.

11. The method for caching multiple copies of data according to claim 1, characterized in that: The method further comprises: In the event of a node failure, determining a target remote node based on the relationship between the failed node and the preset replica; The local data of the failed node is stored based on the backup space of the target remote node.

12. The method for caching multiple copies of data according to claim 11, characterized in that: The determining of the target remote node based on the relationship between the faulty node and the preset replica includes: Determine the number of remote nodes corresponding to the faulty node based on the relationship between the faulty node and the preset replica; When the number of the remote node is one, taking the remote node as the target remote node; When there are multiple remote nodes, the target remote node is determined based on a preset priority.

13. The method for caching multiple copies of data according to claim 11, characterized in that: The local data includes master data and metadata, and the backup space based on the target remote node stores the local data of the failed node, including: An index structure of the faulty node is established based on the master data and metadata of the faulty node stored in the backup space of the target remote node, so that the master data of the faulty node is in an activated state, which facilitates the reading of the local data of the faulty node.

14. The method for caching multiple copies of data according to claim 13, characterized in that: In the event of a node failure, the method further comprises: If the actual number of nodes is greater than the target number of replicas, a replacement node for the failed node is determined based on the preset replica relationship; The backup data of the remote node corresponding to the failed node is stored based on the backup space of the replacement node.

15. The method for caching multiple copies of data according to claim 14, characterized in that: The determining a replacement node for the failed node based on the preset replica relationship includes: Determine, based on the relationship between the failed node and the preset replica, a set of replacement nodes that do not store the remote node corresponding to the failed node among a plurality of nodes; The replacement node is determined based on a preset rule and the replacement node set.

16. The method for caching multiple copies of data according to claim 13, characterized in that: The method further comprises: Determine a set of nodes that do not store backup data of the failed node among the multiple nodes based on the failed node and the preset replica relationship; Determine a target node based on a preset rule and the node set, and establish a replica relationship between the target node and the target remote node; The local data of the failed node stored in the target remote node is backed up based on the replica relationship and the backup space of the target node.

17. A multi-copy data cache device, characterized in that: Applied to a multi-node cluster, the device comprises: A first determination module, used to determine storage spaces of multiple nodes and at least one backup space before data caching, wherein the storage space is used to store local data of the corresponding node; A second determination module is used to determine the backup space of the remote node corresponding to each node based on the preset replica relationship, wherein the backup space of the remote node is used to store the backup data of the current node; A third determination module is used to perform information pre-interaction based on the preset replica relationship to determine the backup space address of the remote node corresponding to each node; The data caching module is used to write the local data of the node into the backup space of the corresponding remote node based on the backup space address of the corresponding remote node during the data caching process when the local data of the node is stored in the corresponding storage space, so as to cache multiple copies of the local data of the node.

18. A computer-readable storage medium, characterized in that: A multiple-copy cache program for data is stored thereon, and when the multiple-copy cache program for data is executed by a processor, a multiple-copy cache method for data according to any one of claims 1-16 is implemented.

19. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the method for caching multiple copies of data as claimed in any one of claims 1 to 16 when executing the computer program.

20. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for caching multiple copies of data as claimed in any one of claims 1 to 16 are implemented.

Images