Data storage method, device, storage medium, and program product

Abstract

Embodiments of the present application provide a data storage method, device, storage medium and program product. In the embodiments of the present application, for the first EC data (i.e. inventory EC data) stored in the EC folding manner by the storage cluster, it can be detected whether the current storage cluster supports the unfolded storage of the inventory EC data. If the current storage cluster supports the unfolded storage of the inventory EC data, from the current storage cluster, a target storage node equal to the number of encoded blocks contained in the inventory EC data is determined, and the inventory EC data is stored in the target storage node in the EC unfolding manner, so as to realize the automatic unfolded storage of the inventory EC data. In this way, the fault tolerance of the inventory EC data stored in the EC unfolding manner can be restored to the original fault tolerance of the inventory EC data, and the fault tolerance of the inventory EC data can be improved.

Description

Technical Field

[0001] This application relates to the field of storage technology, and in particular to a data storage method, device, storage medium, and program product. Background Technology

[0002] Erasure coding (EC) is a data protection technology that divides data into multiple blocks and generates redundant check blocks, enabling the recovery of original data in the event of loss or corruption. Erasure coding is widely used in distributed storage clusters to improve data reliability and availability. A common erasure coding configuration is N+M, which means dividing the data into N blocks, encoding these N blocks using an erasure coding algorithm, generating M check blocks, for a total of N+M encoded blocks. This configuration can support up to M data blocks failing.

[0003] In distributed storage clusters, N+M coded blocks are typically stored across N+M storage nodes. However, in some storage scenarios, such as private cloud storage, the storage cluster is smaller and has fewer storage nodes. To reduce data redundancy and thus lower data storage costs, multiple coded blocks can be stored on a single storage node, achieving EC folding storage. However, traditional EC folding storage solutions cannot adaptively change the storage method for existing erasure coding data, affecting the fault tolerance of existing EC data. Summary of the Invention

[0004] This application provides a data storage method, device, storage medium, and program product to achieve adaptive expansion of existing erasure coding data and improve the fault tolerance of existing EC data.

[0005] This application provides a data storage method, wherein a storage cluster stores first erasure coding data stored in an erasure coding folding manner; the method includes:

[0006] For the first erasure coding data, check whether the current storage cluster supports storing the first erasure coding data in an expanded form;

[0007] If the current storage cluster supports the expanded storage of the first erasure coding data, then target storage nodes are determined from the current storage cluster; the number of target storage nodes is equal to the target number of the first coding blocks contained in the first erasure coding data;

[0008] The erasure coding expansion method is used to store the first erasure code data in the target storage node.

[0009] In some embodiments, the method further includes:

[0010] For the second erasure coding data to be stored, check whether the current storage cluster supports storing the second erasure coding data in an expanded manner.

[0011] If the current storage cluster supports storing the second erasure coding data in an expanded manner, then the second erasure coding data is stored in the current storage cluster in an expanded erasure coding manner.

[0012] Accordingly, if the current storage cluster does not support the expanded storage of the second erasure coding data, the number of second coding blocks allocated to each of the multiple third storage nodes is determined based on the storage resource information of the multiple third storage nodes in the current storage cluster and the fault tolerance supported by the second erasure coding data; wherein, the second coding block is the coding block contained in the second erasure coding data; multiple third storage nodes refer to storage nodes that support the storage of the second coding blocks; the number of the multiple third storage nodes is less than the total number of the second coding blocks;

[0013] The second erasure code data is stored in the plurality of third storage nodes according to the number of second coding blocks allocated to each of the plurality of third storage nodes.

[0014] This application embodiment also provides a data storage method, wherein a storage cluster stores first erasure coding data stored in an erasure coding folding manner; the method includes:

[0015] Obtain storage resource information of other storage nodes in the current storage cluster; the other storage nodes are storage nodes other than the multiple first storage nodes that store the first erasure coding data;

[0016] Based on the storage resource information of other storage nodes in the current storage cluster, detect whether there is a second storage node among the other storage nodes that supports storing at least one first coded block;

[0017] If at least one second storage node is detected, the number of first coding blocks allocated to each of the plurality of first storage nodes and the at least one second storage node is determined based on the sum of the number of second storage nodes and the number of first storage nodes and the total number of first coding blocks;

[0018] The first fault tolerance of the first erasure coding data is determined based on the configuration of the data blocks and the check blocks of the first erasure coding data, and the number of first coding blocks allocated to each of the plurality of first storage nodes and the at least one second storage node.

[0019] If the first fault tolerance is higher than the second fault tolerance of the first erasure code data stored in the erasure code folding manner, based on the configuration of the data blocks and check blocks of the first erasure code data, and the number of first coding blocks allocated to each of the plurality of first storage nodes and the at least one second storage node, a portion of the first coding blocks stored in the first storage node are migrated to the at least one second storage node.

[0020] This application also provides an electronic device, including: a memory and a processor; wherein the memory is used to store computer programs;

[0021] The processor is coupled to the memory and is used to execute the computer program for performing the steps in the aforementioned data storage methods.

[0022] This application also provides a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps in the aforementioned data storage methods.

[0023] This application also provides a computer program product, including a computer program that, when executed by one or more processors, causes the one or more processors to perform the steps in the aforementioned data storage methods.

[0024] In this embodiment, for the first EC data (i.e., existing EC data) stored in the storage cluster using EC folding, it can be detected whether the current storage cluster supports expanding the existing EC data for storage. If the current storage cluster supports expanding the existing EC data for storage, a target storage node with the same number of coded blocks as the existing EC data is determined from the current storage cluster, and the existing EC data is stored in the target storage node using the EC expansion method, thereby achieving automatic expansion storage of the existing EC data. In this way, the fault tolerance capability of the existing EC data stored in the EC expansion method can be restored to the original fault tolerance capability of the existing EC data, which can improve the fault tolerance capability of the existing EC data. Attached Figure Description

[0025] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0026] Figure 1 A flowchart illustrating the data storage method provided in an embodiment of this application;

[0027] Figure 2a A flowchart illustrating another data storage method provided in an embodiment of this application;

[0028] Figure 2b A flowchart illustrating yet another data storage method provided in an embodiment of this application;

[0029] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0032] The concepts and terms involved in the embodiments of this application will be explained below.

[0033] A data block (Chunk) is a block in which the data of a file is divided into multiple data blocks, each defined as a data block. Each data block can be copied multiple times.

[0034] Multi-replica storage: Data blocks are replicated into multiple copies and distributed across different storage nodes. These copies back up each other and can be accessed and recovered independently. Multiple replicas ensure high data availability; even if one storage node fails, replicas on other nodes can still provide data access. However, each replica requires the same amount of storage space, resulting in higher storage overhead. The redundancy factor in multi-replica storage is equal to the total number of replicas corresponding to a data block. For example, three-replica storage requires three times the storage space, resulting in a redundancy factor of 3.

[0035] Erasure Coding (EC): EC is a data protection technology that divides data into multiple blocks and encodes these blocks using an erasure coding algorithm to generate parity blocks. These data blocks and parity blocks are then distributed and stored across different storage nodes. In the event of a certain amount of data loss or corruption, the original data can be recovered using the remaining data. Erasure coding configurations are typically represented as N+M. N+M EC means that the data is divided into N blocks, and each of these N blocks is encoded using an erasure coding algorithm to generate M parity blocks, for a total of N+M encoded blocks. N ≥ 2 and is an integer. For N+M erasure coding, the recovery of data from M encoded blocks is supported.

[0036] Erasure coding storage has a lower redundancy factor compared to multi-replica storage, saving storage space. Specifically, for erasure coding, the redundancy factor equals the total number of coded blocks divided by the total number of data blocks, i.e., (N+M) / N. Assuming the original data is divided into N data blocks with a fault tolerance of M, multi-replica storage requires (M+1) copies of each data block, necessitating the storage space of N*(M+1) data blocks. EC storage requires the storage space of N+M coded blocks, saving the storage space of (N-1)*M data blocks compared to multi-replica storage.

[0037] EC Deployed Storage: EC deployed storage is a traditional EC storage method that stores each coded block of erasure coding data on a separate storage node. For N+M erasure coding data, it can support the recovery of M coded blocks and also supports the failure of M storage nodes.

[0038] EC (Extended Encoding) folding storage: EC folding storage allows a single storage node to hold multiple coded blocks, which are located on different disks within the same storage node. For example, in a private cloud scenario, where the user's storage cluster is relatively small and contains fewer storage nodes, data security requirements can be lowered to reduce data redundancy (lower data storage costs) and allow a single storage node to hold multiple coded blocks, thus enabling wider ECs. The width of an EC refers to the total number of coded blocks contained in the erasure coding data, i.e., the size (N+M). A larger (N+M) indicates a wider EC.

[0039] Employing EC folding storage can reduce the redundancy factor of data, thereby lowering data storage costs. For example, a cluster with 6 data nodes can store a maximum of 4+2 EC data using EC expanded storage, with a redundancy factor of 1.5. Using EC folding storage, it can store 8+3 EC data, or even 12+3 EC data, with redundancy factors of 1.375 and 1.25 respectively.

[0040] The following provides an exemplary description of a traditional EC folding storage solution.

[0041] Traditional EC folding schemes, for N+M erasure-coded data, randomly select (N+M) disks to store N data blocks and M parity blocks. Specifically, the original data is divided into N data blocks, and these blocks are encoded using an erasure coding algorithm to obtain M parity blocks, resulting in (N+M) encoded blocks. These (N+M) encoded blocks are then randomly distributed across (N+M) disks on storage nodes, with each storage node holding a maximum of R encoded blocks. R is determined by the user's fault tolerance requirements.

[0042] In practical use, storage clusters may be expanded. For example, an initial storage cluster may have 6 storage nodes, which may expand to 20 storage nodes as actual needs require. In the aforementioned traditional EC folding storage solution, the existing data stored in the storage cluster using the EC folding method cannot be automatically expanded after the storage cluster is expanded. This results in this portion of existing data not being able to fully utilize the expanded storage resources, and its fault tolerance cannot be improved.

[0043] This is primarily because EC folding storage stores multiple coded blocks on the same storage node. While this reduces data storage costs, it also reduces the fault tolerance of erasure coding, thus lowering data security. For example, if an initial storage cluster has 6 storage nodes, and the user configures 8+3 EC, expecting each storage node to hold a maximum of 2 replicas, then 5 storage nodes each store 2 coded blocks, and 1 storage node stores 1 coded block. This EC folding storage method can support 3 disk failures. For 8+3 EC, using EC unfolding storage, it can support 3 storage node failures. Therefore, traditional EC folding storage solutions cannot automatically unfold existing data stored in EC folding mode, thus failing to improve the fault tolerance of existing data.

[0044] To achieve adaptive expansion of existing data stored in the EC folding mode, in some embodiments of this application, for the first EC data (i.e., existing EC data) stored in the storage cluster in the EC folding mode, it can be detected whether the current storage cluster supports expanding the existing EC data for storage. If the current storage cluster supports expanding the existing EC data for storage, then a target storage node with the same number of coded blocks as the existing EC data is determined from the current storage cluster, and the existing EC data is stored in the target storage node using the EC expansion mode, thereby realizing automatic expansion storage of the existing EC data. In this way, the fault tolerance capability of existing EC data stored in the EC expansion mode can be restored to the original fault tolerance capability of the existing EC data, which can improve the fault tolerance capability of the existing EC data.

[0045] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0046] It should be noted that the same reference numerals in the following figures and embodiments denote the same object or the same step. Therefore, once an object or step is defined in one figure or embodiment, it does not need to be discussed further in subsequent figures and embodiments.

[0047] Figure 1 This is a flowchart illustrating a data storage method provided in an embodiment of this application. This method can be applied to resource scheduling nodes or metadata nodes in a storage cluster. In this embodiment, the storage cluster stores first erasure coding data (i.e., first EC data) stored in an EC folding manner. The data storage method mainly includes:

[0048] 101. For the first EC data stored in the current storage system in EC folding mode, check whether the current storage cluster supports expanding the storage of the first EC data.

[0049] 102. If the current storage cluster supports the expansion storage of the first EC data, then determine the target storage nodes from the current storage cluster; the number of target storage nodes is equal to the target number of the first coded blocks contained in the first EC data.

[0050] 103. Using the EC deployment method, the first EC data is stored on the target storage node.

[0051] In this embodiment, the first EC data stored in the current storage cluster using EC folding contains a target number of coded blocks. The target number refers to the total number of coded blocks contained in the first EC data, denoted as K. Where K = N + M. That is, the first EC data contains (N + M) coded blocks, namely N data blocks and M parity blocks.

[0052] In this embodiment, to achieve adaptive unpacking of erasure coding data stored in EC folding mode, in step 101, for the first EC data stored in EC folding mode, it is detected whether the current storage cluster supports unpacking and storing the first EC data. Unpacking and storing the first erasure coding data means storing the first EC data using EC unpacking mode.

[0053] In some embodiments, the current storage cluster may periodically detect whether it supports expanding and storing the first EC data according to a set detection period. The current storage cluster is the storage cluster within the current detection period.

[0054] In some embodiments, storage resource information of each storage node in the current storage cluster can be obtained. This storage resource information refers to information reflecting the storage resource space capacity and free storage space of the storage node, and may include, but is not limited to, the total storage capacity of the storage node, the total free storage space of the storage node, the number of disks, the storage capacity of each disk, and the free storage space of each disk. Furthermore, storage resource information of other storage nodes besides those storing the aforementioned first EC data can be obtained from the storage resource information of each storage node in the current storage cluster.

[0055] Furthermore, based on the storage resource information of other storage nodes in the current storage cluster, it can be detected whether there are any storage nodes among these other storage nodes that can support storing a single coded block contained in the first EC data. For ease of description and distinction, the storage node currently storing the first EC data is defined as the first storage node; and the storage node among the other storage nodes in the current storage cluster that supports storing a single coded block of the first EC data is defined as the second storage node. There are multiple first storage nodes, denoted as Q0, where Q0 < K.

[0056] Specifically, for any other storage node A in the current storage cluster besides the first storage node, the free storage space of each disk in storage node A can be determined based on the storage resource information of storage node A. If there is a disk in storage node A with free storage space greater than or equal to the data size of a single coded block of the first EC data, then storage node A is determined to be able to support storing a single coded block contained in the first EC data, and storage node A is recorded as a second storage node among the other storage nodes in the current storage cluster. Conversely, if there is no disk in storage node A with free storage space greater than or equal to the data size of a single coded block of the first EC data, then storage node A is determined not to be a second storage node.

[0057] Furthermore, the number of second storage nodes among the other storage nodes in the current storage cluster (excluding the first storage node) can be counted. If the sum of the number of second storage nodes and the number of multiple first storage nodes is greater than or equal to K, then the current storage cluster is determined to support the expanded storage of the first EC data. If the sum of the number of second storage nodes and the number of multiple first storage nodes is less than K, or if there is no second storage node among the other storage nodes in the current storage cluster (excluding the first storage node) that supports storing a single coded block of the first EC data, then the current storage cluster is determined to not support the expanded storage of the first EC data.

[0058] Furthermore, if the current storage cluster supports expanding the storage of the first EC data, then K target storage nodes can be determined from the current storage cluster.

[0059] In some embodiments, K target storage nodes capable of supporting the storage of a single coded block contained in the first EC data can be determined from the current storage cluster. For example, K target storage nodes capable of supporting the storage of a single coded block contained in the first EC data can be randomly selected from the current storage cluster. Alternatively, K target storage nodes can be selected from the storage nodes capable of supporting the storage of a single coded block contained in the first EC data, based on the order of their free storage space information from largest to smallest in the current storage cluster, and so on.

[0060] In other embodiments, to reduce data migration and improve the expansion and storage speed of the first EC data, for multiple coded blocks (assuming P blocks, 2≤P≤R0) stored in the same first storage node in the first EC data, one coded block is retained in the first storage node, and only (P-1) of the P coded blocks are migrated to the second storage node. R0 is the maximum number of coded blocks that can be stored in each storage node of the first EC data. R0 is determined by the fault tolerance supported by the first EC data. The fault tolerance supported by the first EC data is determined by the configuration of data blocks and parity blocks in the first EC data and the fault tolerance requirements of the users of the first EC data. For example, if the ratio of data blocks to parity blocks in the first EC data is N+M, and the fault tolerance requirement of the users of the second EC data is W0 storage nodes, then the fault tolerance supported by the first EC data is either M coded block failures or W0 storage node failures.

[0061] In this embodiment of the application, for ease of description and distinction, multiple coded blocks stored in the same storage node in the first EC data are defined as target coded blocks.

[0062] The first number P0 of target coded blocks contained in the first erasure coding data and the second number Q1 of the first storage nodes storing the target coded blocks can be determined. Wherein, Q1 is less than or equal to the total number of first storage nodes Q0, i.e., Q1 ≤ Q0. The first number P0 of target coded blocks satisfies: Where i represents the i-th first storage node containing the target encoded block. P i This indicates the number of target encoded blocks stored in the i-th first storage node, which is the number of encoded blocks of the first EC data stored in that first storage node.

[0063] Furthermore, a third number of S second storage nodes can be selected from the aforementioned second storage nodes. Where S = (P0 - Q0). In some embodiments, S second storage nodes can be randomly selected from the second storage nodes. Alternatively, S second storage nodes can be selected based on their storage resource information. For example, a storage resource fragmentation minimization strategy can be used to select S second storage nodes. Specifically, S second storage nodes can be selected sequentially from the second storage nodes in ascending order of total free storage space. This allows the first EC-sized number of encoded blocks to be migrated to these S second storage nodes, reducing the overall storage resource fragmentation of the storage cluster. Alternatively, a load balancing strategy can be used to select S second storage nodes. Specifically, S second storage nodes can be selected sequentially from the second storage nodes in descending order of total free storage space. This allows the first EC-sized number of encoded blocks to be migrated to these S second storage nodes, reducing the difference in storage resources occupied by these S second storage nodes compared to other storage nodes, thus contributing to load balancing.

[0064] Furthermore, the S second storage nodes and Q0 first storage nodes can be used as K target storage nodes. Here, S + Q0 = K. This is mainly because S = P0 - Q1, and P0 + (Q0 - Q1) = K. Q0-Q1 represents the number of first storage nodes storing a single coded block of the first EC data, and each of the (Q0-Q1)Q0-Q1 first storage nodes stores a total of (Q0-Q1) coded blocks of the first EC data.

[0065] After the target storage nodes are determined, in step 103, erasure coding expansion can be used to store the first EC data on K target storage nodes.

[0066] Specifically, with the goal of storing one encoded block of the first EC data in each target storage node, S target encoded blocks are migrated from the first storage node to S second storage nodes, and each target storage node stores one encoded block, thereby storing the first EC data in K target storage nodes and realizing the expanded storage of the first EC data.

[0067] Specifically, for a first storage node storing multiple (i.e., P) target encoded blocks of the first EC data, (P-1) target encoded blocks from the first storage node are migrated to (P-1) second storage nodes. Each second storage node stores one target encoded block, while the other target encoded block remains on the first storage node. For a second storage node that has acquired a target encoded block, the target encoded block can be stored on a target disk with free storage space greater than or equal to the target encoded block's size. If there are multiple target disks for a second storage node, the target encoded block can be randomly stored on one target disk, or stored on the target disk with the smallest free storage space to minimize storage fragmentation; or stored on the target disk with the largest free storage space to achieve load balancing. Using the same method, S target encoded blocks are migrated to S second storage nodes. Other first storage nodes storing one encoded block of the first EC data remain on the first storage node, thus expanding and storing the K encoded blocks contained in the first EC data across K target storage nodes, achieving expanded storage of the first EC data.

[0068] In this embodiment, for the first EC data (i.e., existing EC data) stored in the storage cluster using EC folding, it can be detected whether the current storage cluster supports expanding the existing EC data for storage. If the current storage cluster supports expanding the existing EC data for storage, a target storage node with the same number of coded blocks as the existing EC data is determined from the current storage cluster, and the existing EC data is stored in the target storage node using the EC expansion method, thus realizing automatic expansion storage of the existing EC data. The fault tolerance capability of existing EC data stored using the EC expansion method is improved, thereby enhancing the fault tolerance capability of the existing EC data.

[0069] For example, if the initial storage cluster has 6 storage nodes, and the existing EC data is 8+3 ECs, the 8+3 ECs can be stored using an EC folding method where 5 storage nodes each store 2 coded blocks, and 1 storage node stores 1 coded block. This EC folding storage method can support 3 disk failures, meaning its fault tolerance capability is 3 disk failures. By using the solution provided in this application embodiment to expand the 8+3 EC storage, it can support 3 storage node failures, thus improving the fault tolerance capability of the 8+3 ECs and enhancing its data security.

[0070] In some embodiments of this application, in addition to adaptively expanding and storing existing EC data stored in an EC folding manner, when the storage cluster is expanded or a new storage node supports storing at least one coded block of EC data, some coded blocks in the storage node storing coded blocks contained in multiple EC data can be migrated to the new storage node. This changes the EC folding method of the existing EC data stored in an EC folding manner, thereby improving the fault tolerance of the EC data. The following is in conjunction with... Figure 2a Please provide a detailed explanation.

[0071] Figure 2a This is a flowchart illustrating another data storage method provided in an embodiment of this application. The storage cluster stores first erasure coding data (i.e., first EC data) stored in an EC folding manner. Figure 2a As shown, this data storage method mainly includes:

[0072] 21. Obtain storage resource information of other storage nodes in the current storage cluster; other storage nodes are storage nodes other than the multiple first storage nodes storing the first erasure coding data.

[0073] 22. Based on the storage resource information of other storage nodes in the current storage cluster, detect whether there is a second storage node among the other storage nodes that supports storing at least one first coded block.

[0074] 23. If at least one second storage node is detected, the number of first coding blocks allocated to each of the multiple first storage nodes and at least one second storage node shall be determined based on the sum of the number of second storage nodes and the number of first storage nodes and the total number of first coding blocks.

[0075] 24. Determine the first fault tolerance of the first erasure code data based on the configuration of the data blocks and check blocks of the first erasure code data, and the number of first coding blocks allocated to each of the multiple first storage nodes and at least one second storage node.

[0076] 25. If the first fault tolerance is higher than the second fault tolerance of the first erasure code data stored in the erasure code folding manner, based on the configuration of the data blocks and check blocks of the first erasure code data, and the number of first coding blocks allocated to each of the multiple first storage nodes and at least one second storage node, a portion of the first coding blocks stored in the first storage node shall be migrated to at least one second storage node.

[0077] In this embodiment, the first EC data is stored in the storage cluster using an EC folding method. The method for determining the fault tolerance of the first EC data stored using the current EC folding method can be found in the relevant content of the foregoing embodiments, and will not be repeated here.

[0078] When a new storage node emerges in the storage cluster capable of storing at least one coded block of the first EC data, whether to migrate a portion of the coded blocks stored on the first storage node (containing multiple (e.g., P) coded blocks of the first EC data) to the new storage node (i.e., the second storage node) that supports storing at least one coded block of the first EC data depends primarily on whether migrating the coded blocks to the second storage node will improve the fault tolerance of the first EC data. If the fault tolerance of the first EC data remains unchanged after the coded blocks are migrated, migrating the coded blocks to the second storage node will increase the resource consumption of data migration. Only when the fault tolerance of the first EC data is improved after the coded blocks are migrated to the second storage node should the migration of a portion of the coded blocks stored on the first storage node be executed.

[0079] For example, assuming the initial storage cluster has 4 storage nodes, and the existing EC data is 8+4EC, each of the 4 storage nodes can store 3 coded blocks. This EC folding storage method can support 4 disk failures or 1 storage node failure, meaning its fault tolerance is 4 disk failures or 1 storage node failure. If a second storage node appears in the current storage cluster, and some coded blocks stored on the original storage node are migrated to this second storage node, the existing EC data can be stored as follows: 5 storage nodes each store 2 coded blocks, and 1 storage node stores 3 coded blocks. The fault tolerance of the EC data stored in this EC folding storage method remains unchanged, supporting either 4 disk failures or 1 storage node failure. The fault tolerance of the EC data remains the same. If two second storage nodes appear in the current storage cluster, and some coded blocks stored on the original storage nodes are migrated to these two second storage nodes, the existing EC data can be stored as follows: 6 storage nodes each store 2 coded blocks. The fault tolerance of the EC data stored in this EC folding storage method remains unchanged, supporting either 4 disk failures or 2 storage node failures. The fault tolerance of the EC data is improved. Therefore, if a second storage node is detected in steps 21 and 22, it is also necessary to determine whether the fault tolerance of the first EC data is improved after migrating a portion of the coded blocks stored in the first storage node containing multiple coded blocks of the first EC data to the second storage node. Then, it is determined whether to migrate a portion of the coded blocks stored in the first storage node containing multiple coded blocks of the first EC data to the second storage node. For the specific implementation of step 22, please refer to the relevant content of the foregoing embodiments, which will not be repeated here.

[0080] In the following embodiments, for ease of description, the encoded blocks contained in the first EC data are defined as the first encoded block. To determine the fault tolerance capability of the first EC data after migrating some encoded blocks stored on the first storage node containing multiple encoded blocks of the first EC data to the second storage node, in step 23, the number of first encoded blocks allocated to each of the S0 second storage nodes and Q0 first storage nodes can be determined based on the sum of the number of second storage nodes S0 and the number of first storage nodes Q0 (i.e., S0 + Q0), and the total number of first encoded blocks K. Wherein, S0 ≥ 1 and is an integer.

[0081] Specifically, if the total number K of the first coded blocks is an integer multiple of the sum of the number of second storage nodes S0 and the number of first storage nodes Q0 (i.e., S0+Q0), that is, K is an integer multiple of (S0+Q0), then the number of first coded blocks allocated to each of the S0 second storage nodes and the Q0 first storage nodes is determined to be equal to: K / (S0+Q0).

[0082] If the total number K of the first coded blocks is not an integer multiple of the sum of the number of second storage nodes S0 and the number of first storage nodes Q0 (i.e., S0+Q0), meaning K / (S0+Q0) is not divisible, then the quotient of K / (S0+Q0) is taken as the fourth quantity (denoted as Z), and Z first coded blocks are allocated to each of the S0 second storage nodes and Q0 first storage nodes, with one remaining first coded block allocated to some or all of the first storage nodes. If the remaining first coded blocks (i.e., KZ*(S0+Q0) first coded blocks) are allocated to some or all of the first storage nodes and the allocation is completed, then the number of first coded blocks allocated to each of the current S0 second storage nodes and Q0 first storage nodes is determined. Specifically, each of the S0 second storage nodes is allocated Z first coded blocks; each of the Q2 first storage nodes is allocated (Z+1) first coded blocks. Where 1≤Q2≤Q0. If Q2 = Q0, then the first storage node is allocated (Z+1) first coding blocks; if Q2 < Q0, then each of the Q2 first storage nodes is allocated (Z+1) first coding blocks, and each of the (Q0-Q1) first storage nodes is allocated Z first coding blocks.

[0083] For example, assuming the initial storage cluster has 6 first storage nodes, and the existing EC data is 8+4EC, each of the 6 first storage nodes can store 2 encoded blocks. If the current storage cluster has 3 second storage nodes, firstly, allocate 1 encoded block to each first storage node and each second storage node. If there are 3 encoded blocks remaining, allocate these 3 encoded blocks to the 3 first storage nodes. After allocating the remaining encoded blocks to the first storage nodes, the allocation is complete, meaning there are no blocks left. Thus, each of the 3 first storage nodes stores 2 encoded blocks, each of the 3 first storage nodes stores 1 encoded block, and each of the 3 second storage nodes stores 1 encoded block.

[0084] If the remaining KZ*(S0+Q0) first-coded blocks are not fully allocated after allocating them to all first-storage nodes, then the unallocated first-coded blocks are allocated to some second-storage nodes, with each second-storage node receiving one unallocated first-coded block. The number of unallocated first-coded blocks is equal to KZ*(S0+Q0)-Q0. Further, the number of first-coded blocks allocated to each of the current S0 second-storage nodes and Q0 first-storage nodes can be determined. Specifically, each of the Q0 first-storage nodes receives (Z+1) first-coded blocks, each of the [KZ*(S0+Q0)-Q0] second-storage nodes receives (Z+1) first-coded blocks, and the remaining second-storage nodes each receive Z first-coded blocks.

[0085] For example, assuming the initial storage cluster has 4 first storage nodes, and the existing EC data is 8+4EC, each of the 4 first storage nodes can store 3 coded blocks. If the current storage cluster has 3 second storage nodes, firstly, allocate 1 coded block to each first storage node and each second storage node. If there are 5 coded blocks remaining, allocate the remaining 5 coded blocks to the 4 first storage nodes. If there are 1 coded block remaining after allocating the remaining coded blocks to the first storage nodes, allocate this 1 coded block to one second storage node. Thus, each of the 4 first storage nodes stores 2 coded blocks, each of the 1 first storage node stores 2 coded blocks, and the remaining 2 second storage nodes each store 1 coded block.

[0086] After determining the number of first encoded blocks stored by each first storage node and each second storage node, in step 24, the fault tolerance of the first EC data can be determined based on the configuration of the data blocks and check blocks of the first EC data and the number of first encoded blocks stored by each first storage node and each second storage node.

[0087] Specifically, the configuration of data blocks and parity blocks for the first EC data is (N+M), that is, N data blocks and M parity blocks. The maximum number X of first code blocks stored in the same storage node can be determined based on the number of first code blocks stored in each first storage node and each second storage node. It is determined that the first EC data can support M / X storage node failures or M disk failures. That is, the fault tolerance of the first EC data is: M / X storage node failures or M disk failures.

[0088] For example, assuming the configuration of the first EC data is 8+4, and the storage node that stores the most first coding blocks among the first and second storage nodes stores 3 first coding blocks, then the first EC data can support 1 storage node failure or 4 disk failures. That is, the fault tolerance of the first EC data is: 1 storage node or 4 disks.

[0089] In this embodiment, for ease of description and distinction, the fault tolerance of the first EC data determined after migrating a portion of the first coded blocks to the second storage node is defined as the first fault tolerance; and the fault tolerance of the first EC data originally stored by the first storage node in an EC folding manner is defined as the second fault tolerance. The method for determining the second fault tolerance can refer to the method for determining the first fault tolerance described above. Accordingly, the magnitudes of the first fault tolerance and the second fault tolerance can be compared. If the first fault tolerance is greater than the second fault tolerance, then based on the number of first coded blocks allocated to each of the S0 second storage nodes and Q0 first storage nodes, a portion of the first coded blocks are migrated from the first storage node to the second storage node. If the first fault tolerance is equal to the second fault tolerance, then the storage method of the first EC data remains unchanged.

[0090] In this embodiment of the application, for existing data originally stored in the EC folding method, when a new second storage node appears in the storage cluster that can accommodate the coded blocks of existing EC data, the fault tolerance of the existing EC data is first determined after assuming that the coded blocks of the existing EC data are migrated to the second storage node. If the fault tolerance of the existing EC data is improved, then some coded blocks of the existing EC data are migrated to the second storage node, which can improve the fault tolerance capability of the existing EC data.

[0091] For a storage system, in addition to existing data, new data will also be written. In this embodiment, for ease of description and distinction, the aforementioned existing EC data is defined as first EC data, and the newly written EC data is defined as second EC data. Accordingly, the coded blocks contained in the first EC data are defined as first coded blocks; the coded blocks contained in the second EC data are defined as second coded blocks.

[0092] For the aforementioned traditional EC folding storage scheme, even if the scaled-up storage cluster can support the unfolded storage of newly written second EC data, there is still a possibility that multiple second coded blocks may be folded and stored on the same storage node. For example, if the initial storage cluster has 6 storage nodes, and the user configures 8+3 ECs, expecting each storage node to hold a maximum of 2 replicas, then 5 storage nodes will each store 2 coded blocks, and 1 storage node will store 1 coded block. After the storage cluster is expanded to 20 storage nodes, the traditional EC folding storage scheme still randomly selects 11 disks from the disks of the 20 storage nodes to store 11 coded blocks, with the constraint that each storage node can hold a maximum of 2 coded blocks. It is possible that 2 of the selected 11 disks belong to the same storage node, which means that the newly written EC data is still folded and cannot be guaranteed to be unfolded.

[0093] To address this technical problem, embodiments of this application also provide adaptive expansion or collapse storage for newly written EC data. The following, in conjunction with... Figure 2b An example is provided. Figure 2b This is a flowchart illustrating yet another data storage method provided in an embodiment of this application. Figure 2b As shown, this data storage method mainly includes the following steps:

[0094] 201. For the second EC data to be stored, check whether the current storage cluster supports expanding and storing the second EC data.

[0095] 202. If the current storage cluster supports expanding the storage of the second EC data, then the EC expansion method will be used to store the second EC data in the current storage cluster.

[0096] In this embodiment, the second EC data is the EC data to be written (i.e., to be stored) to the storage cluster. In this embodiment, the current storage cluster refers to the storage cluster at the time the second EC data is to be written to, as described above. Figure 1 The current storage cluster in this embodiment may or may not be the same storage cluster, depending on the time when the second EC data is to be written to the storage cluster and the time when the first EC data, which was previously detected, can be expanded for storage. If the two are at the same time, then the current storage cluster in this embodiment is the same as... Figure 1 The current storage cluster in the two instances is the same storage cluster; if they are not at the same time, then the current storage cluster in this embodiment is the same as the current storage cluster in the two instances. Figure 1 The current storage cluster may not be the same storage cluster.

[0097] In this embodiment, in order to prevent the EC data that can be expanded and stored from being folded, in step 201, for the second EC data to be stored, it can be detected whether the current storage cluster supports expanding and storing the second EC data.

[0098] Specifically, storage resource information of each storage node in the current storage cluster can be obtained. The description of the storage resource information of the storage nodes can be found in the relevant content of the aforementioned embodiments, and will not be repeated here. Furthermore, based on the storage resource information of multiple storage nodes in the current storage cluster, multiple third storage nodes can be determined from among the multiple storage nodes, where the free storage resources of a single disk are greater than or equal to the data size of a single second coded block.

[0099] If the number of third storage nodes is less than the total number of second coded blocks, it is determined that the current storage cluster does not support expanding the storage of the second EC data. Conversely, if the number of third storage nodes is greater than or equal to the total number of second coded blocks, it is determined that the current storage cluster supports expanding the storage of the second EC data.

[0100] For embodiments where the current storage cluster supports expanding the storage of the second EC data, the EC expansion method can be used to store the second EC data in the current storage cluster. Specifically, if the number of the aforementioned third storage nodes is equal to the total number of second coded blocks, then the second coded blocks contained in the second EC coded block are randomly stored on multiple third storage nodes, with each third storage node storing one second coded block.

[0101] Specifically, if the number of the aforementioned third storage nodes is greater than the total number of the second coded blocks, a number of third storage nodes equal to the total number of the second coded blocks can be selected from the aforementioned plurality of third storage nodes. Assume the configuration of the second EC data is N1+M1, containing a total of K1 coded blocks, where K1 = N1+M1. That is, the second EC data includes N1 data blocks and M1 parity blocks. Then, K1 third storage nodes can be selected from the plurality of third storage nodes.

[0102] Optionally, K1 third-party storage nodes can be randomly selected from multiple third-party storage nodes. Alternatively, K1 third-party storage nodes can be selected sequentially from multiple third-party storage nodes in descending order of free storage space to achieve load balancing among storage nodes. Alternatively, K1 third-party storage nodes can be selected sequentially from multiple third-party storage nodes in ascending order of free storage space to reduce resource fragmentation among storage nodes.

[0103] Furthermore, K1 second-coded blocks can be randomly stored in K1 third-level storage nodes. Each third-level storage node stores one second-coded block. For any of the K1 third-level storage nodes, a target disk with free storage space greater than or equal to that of the second-coded block can be obtained from that third-level storage node; and the second-coded block can be stored on the target disk. If there are multiple target disks in the same third-level storage node, how to store the second-coded blocks can be found in the aforementioned section on how to store the encoded blocks of the first EC data when there are multiple target disks in the same second-level storage node, which will not be repeated here.

[0104] In this embodiment, the EC data to be written is not stored directly according to the EC expansion or EC folding method. Instead, it is first determined whether the current storage cluster supports the expansion storage of the EC data to be written. If the current storage cluster supports the expansion storage of the EC data to be written, the EC data is expanded and stored. This achieves adaptive expansion of the newly written EC data, which avoids the situation in the traditional EC folding storage scheme where the storage cluster can expand and store EC data, but still folds the newly written EC data for storage. This helps to ensure the fault tolerance of the newly written EC data.

[0105] For embodiments where the current storage cluster does not support the expansion and storage of the second EC data, the number of second coding blocks allocated to each of the multiple third storage nodes can be determined based on the storage resource information of the multiple third storage nodes in the current storage cluster and the fault tolerance supported by the second EC data.

[0106] The fault tolerance supported by the second EC data is determined by the configuration of data blocks and parity blocks in the second EC data and the fault tolerance requirements of the users of the second EC data. For example, if the ratio of data blocks to parity blocks in the second EC data is N1+M1, and the fault tolerance requirement of the users of the second EC data is W storage nodes, then the fault tolerance supported by the second EC data is either M1 failed coding blocks or W failed storage nodes. The fault tolerance supported by the second EC data determines the maximum number of second coding blocks that a single third storage node can store. Since the fault tolerance supported by the second EC data is either M1 failed coding blocks or W failed storage nodes, the total number of second coding blocks stored on W storage nodes must be less than or equal to M1. This way, when W storage nodes fail, the second coding blocks on other non-failed storage nodes can be used to recover the second coding blocks stored on the W failed storage nodes. Accordingly, the maximum number of second coding blocks that a single third storage node can store can be the floor value of M1 / W. If W is 0, then the maximum number of second coding blocks that a single third storage node can store is M1.

[0107] Since the fault tolerance supported by the second EC data determines the maximum number of second coded blocks that a single second storage node can store, and the storage resource information of multiple third storage nodes (specifically, the disk data and free storage space information of each third storage node) determines the number of second coded blocks that each third storage node can accommodate, the number of second coded blocks allocated to each of the multiple third storage nodes can be determined based on the storage resource information of the multiple third storage nodes in the current storage cluster and the fault tolerance supported by the second EC data.

[0108] Specifically, based on the fault tolerance supported by the second EC data and the number of disks and free storage space information of each of the multiple third storage nodes, the maximum number R that each of the multiple third storage nodes can support storing the second coding block is determined.

[0109] Optionally, the maximum number of second coded blocks that a single third storage node can store can be determined based on the fault tolerance supported by the second erasure coding data. For specific implementation details regarding determining the maximum number R of second coded blocks that a single third storage node can store, please refer to the relevant content in the foregoing embodiments, which will not be repeated here.

[0110] For any storage node X, the number of disks that the third storage node X can accommodate a single second coding block can be determined based on the number of disks corresponding to any third storage node X and the information on the free storage space of the disks.

[0111] Furthermore, if the number of disks that the third storage node X can hold for a single second coded block is greater than or equal to the maximum number of second coded blocks that a single third storage node can store, then the maximum number R corresponding to the third storage node X is determined to be the maximum number of second coded blocks that the third storage node X can store. If the number of disks that the third storage node X can hold for a single second coded block is less than the maximum number of second coded blocks that a single third storage node can store, then the maximum number R corresponding to the third storage node X is determined to be the number of disks that the third storage node X can hold for a single second coded block.

[0112] Furthermore, the number of second-coded blocks allocated to each of the multiple third storage nodes, Yi, can be determined by the constraint that the number of second-coded blocks allocated to each of the multiple third storage nodes is less than or equal to the maximum number of supported storage nodes, and the sum of the number of second-coded blocks allocated to the multiple third storage nodes is equal to the total number of second-coded blocks. Here, i represents the i-th third storage node, i = 1, 2, ..., U. U is the total number of third storage nodes.

[0113] After determining the number of second coding blocks allocated to each of the multiple third storage nodes, the second EC data can be stored on the multiple third storage nodes according to the number of second coding blocks allocated to each of the multiple third storage nodes.

[0114] Specifically, for any third storage node i, Yi second-coded blocks can be stored in third storage node i according to the number Yi of second-coded blocks allocated to third storage node i. Within third storage node i, Yi target disks capable of accommodating a single second-coded block can be determined. If the number of target disks in third storage node i is greater than Yi, Yi target disks can be randomly selected from the target disks; or, Yi target disks can be selected sequentially according to their free storage space in descending order; or, Yi target disks can be selected sequentially according to their free storage space in ascending order. Further, Yi second-coded blocks can be randomly stored on Yi target disks. Each target disk stores one second-coded block. Using the same method, second EC data can be folded and stored across multiple third storage nodes.

[0115] In this embodiment, for the EC data to be written, the system can automatically detect whether the current storage cluster supports expanding the EC data for storage. If it does, the EC data is expanded for storage, achieving adaptive expansion of the newly written EC data. This avoids the situation in traditional EC folding storage schemes where the storage cluster can expand the EC data for storage, but the newly written EC data is still folded for storage, thus helping to ensure the fault tolerance of the newly written EC data. If the current storage cluster does not support expanding the EC data for storage, the EC data can be adaptively folded for storage, achieving adaptive folding of the newly written EC data.

[0116] In addition, this EC data storage method has minimal intrusion into the original EC expandable storage logic code. In other words, this solution does not concern itself with how the original EC expandable storage logic is implemented. It only needs to add a step to detect whether the current storage cluster supports expanding EC data storage, and a step to fold EC data storage logic when the current storage cluster does not support expanding EC data storage, so as to achieve adaptive folding of EC data.

[0117] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 101 and 102 can be device A; or the execution subject of step 101 can be device A, and the execution subject of step 102 can be device B; and so on.

[0118] Furthermore, some processes described in the above embodiments and accompanying drawings include multiple operations that appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel.

[0119] Accordingly, embodiments of this application also provide a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause one or more processors to perform the steps in the above-described data storage method.

[0120] This application also provides a computer program product, including a computer program that, when executed by one or more processors, causes the one or more processors to perform the steps in the data storage method described above. In this application, the specific implementation of the computer program product is not limited. In some embodiments, the computer program product may be implemented as an application (APP), a mini-program, a PC client, a program module, a plug-in, an installation package, a software development kit (SDK), an optical disc image file (such as an ISO file), a plug-in, or software in the form of Software as a Service (SaaS), etc., but is not limited thereto.

[0121] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 3 As shown, the electronic device includes a memory 30a and a processor 30b. The memory 30a is used to store computer programs.

[0122] The processor 30b is coupled to the memory 30a and is used to execute a computer program to perform the steps in the data storage methods provided in the foregoing embodiments. Specific implementation details of each step can be found in the relevant descriptions of the foregoing embodiments, and will not be repeated here.

[0123] In some alternative implementations, such as Figure 3 As shown, the electronic device may also include optional components such as a communication component 30c, a power supply component 30d, a display component 30e, and an audio component 30f. Figure 3 The diagram only shows some components and does not mean that the electronic device must contain them. Figure 3 The inclusion of all components does not imply that an electronic device can only include... Figure 3 The components shown.

[0124] in addition, Figure 3 The components within the dashed box are optional, not mandatory, and their specific requirements depend on the form factor of the electronic device. The electronic device in this embodiment can be a desktop computer, laptop computer, mobile phone, or IoT device; it can also be a traditional server, cloud server, or server cluster, or other server equipment.

[0125] In this embodiment, the memory is used to store computer programs and can be configured to store various other data to support operation on its host device. The processor can execute the computer programs stored in the memory to implement corresponding control logic. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Electrically Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0126] In the embodiments of this application, the processor can be any hardware processing device capable of executing the above-described method logic. Optionally, the processor can be a central processing unit (CPU), a graphics processing unit (GPU), or a microcontroller unit (MCU); it can also be a field-programmable gate array (FPGA), a programmable array logic (PAL), a general array logic (GAL), a complex programmable logic device (CPLD), or other programmable devices; or it can be an advanced RISC processor (ARM) or a system on chip (SoC), etc., but is not limited thereto.

[0127] In this embodiment, the communication component is configured to facilitate wired or wireless communication between its host device and other devices. The device housing the communication component can access wireless networks based on communication standards, such as Wireless Fidelity (WiFi), 2G or 3G, 4G, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In another exemplary embodiment, the communication component may also be implemented based on Near Field Communication (NFC), Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wide Band (UWB), Bluetooth (BT), or other technologies.

[0128] In embodiments of this application, the display component may include a liquid crystal display (LCD) and a touch panel (TP). If the display component includes a touch panel, the display component can be implemented as a touchscreen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation.

[0129] In this embodiment, a power supply component is configured to provide power to various components of the device in which it resides. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component resides.

[0130] In embodiments of this application, the audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), which is configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals. For example, in devices with voice interaction capabilities, voice interaction with the user can be achieved through the audio component.

[0131] It should be noted that the terms "first" and "second" in this article are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.

[0132] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, compact disc read-only memory (CD-ROM), optical storage, etc.) containing computer-usable program code.

[0133] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0134] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0135] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0136] In a typical configuration, a computing device includes one or more processors (CPU, etc.), input / output interfaces, network interfaces, and memory.

[0137] Memory may include non-persistent storage in computer-readable media, such as random-access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0138] Computer storage media are readable storage media, also known as removable media. Removable and non-removable media can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media do not include transient computer-readable media, such as modulated data signals and carrier waves.

[0139] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the aforementioned element.

[0140] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A data storage method, characterized in that, The storage cluster stores first erasure coding data stored in an erasure coding folding manner; the method includes: For the first erasure coding data, check whether the current storage cluster supports storing the first erasure coding data in an expanded form; If the current storage cluster supports the expanded storage of the first erasure coding data, then target storage nodes are determined from the current storage cluster; the number of target storage nodes is equal to the target number of the first coding blocks contained in the first erasure coding data; The erasure coding expansion method is used to store the first erasure code data in the target storage node.

2. The method according to claim 1, characterized in that, The current storage cluster includes multiple first storage nodes that store the first erasure coding data; The step of detecting whether the current storage cluster supports expanding and storing the first erasure coding data includes: Based on the storage resource information of other storage nodes in the current storage cluster, detect whether there is a second storage node among the other storage nodes that supports storing a single first coding block contained in the first erasure coding data; the other storage nodes refer to storage nodes other than the plurality of first storage nodes; If a second storage node supporting the storage of the single first code block is detected among the other storage nodes, and the sum of the number of the second storage nodes and the number of the plurality of first storage nodes is greater than or equal to the target number, then it is determined that the current storage cluster supports the expansion and storage of the first erasure coding data.

3. The method according to claim 2, characterized in that, The step of determining the target storage node from the current storage cluster includes: Determine a first number of target coding blocks contained in the first erasure coding data and a second number of first storage nodes storing the target coding blocks; the target coding block refers to a first coding block that is stored in the same first storage node as other first coding blocks in the first erasure coding data. From the second storage nodes, select a third number of second storage nodes; the third number is equal to the first number minus the second number. The third number of second storage nodes and the plurality of first storage nodes are determined as the target storage nodes.

4. The method according to claim 3, characterized in that, The step of storing the first erasure code data in the target storage node using erasure code expansion includes: With the goal of storing one first coding block at each target storage node, a third number of target coding blocks are migrated from the first storage node to the third number of second storage nodes so that the first erasure coding data is stored at the target storage node; each second storage node stores one target coding block.

5. The method according to any one of claims 1-4, characterized in that, Also includes: For the second erasure coding data to be stored, check whether the current storage cluster supports storing the second erasure coding data in an expanded manner. If the current storage cluster supports storing the second erasure coding data in an expanded manner, then the second erasure coding data is stored in the current storage cluster in an expanded erasure coding manner.

6. The method according to claim 5, characterized in that, Also includes: If the current storage cluster does not support storing the second erasure coding data in an expanded manner, then based on the storage resource information of the multiple third storage nodes in the current storage cluster and the fault tolerance supported by the second erasure coding data, the number of second coding blocks allocated to each of the multiple third storage nodes is determined; wherein, the second coding block is the coding block contained in the second erasure coding data; multiple third storage nodes refer to storage nodes that support storing the second coding blocks; the number of the multiple third storage nodes is less than the total number of the second coding blocks; The second erasure code data is stored in the plurality of third storage nodes according to the number of second coding blocks allocated to each of the plurality of third storage nodes.

7. The method according to claim 6, characterized in that, The storage resource information for each third storage node includes: the number of disks in the third storage node and the free storage space information of each disk; determining the number of second coding blocks allocated to each of the multiple third storage nodes based on the storage resource information of the multiple third storage nodes in the current storage cluster and the fault tolerance supported by the second erasure coding data includes: Based on the fault tolerance supported by the second erasure coding data and the number of disks and free storage space information of each of the multiple third storage nodes, the maximum number of second coding blocks that each of the third storage nodes can support for storage is determined. The number of second coding blocks allocated to each of the plurality of third storage nodes is determined by the constraint that the number of second coding blocks allocated to each of the plurality of third storage nodes is less than or equal to the maximum number of corresponding supported storage, and the sum of the number of second coding blocks allocated to the plurality of third storage nodes is equal to the total number of second coding blocks.

8. The method according to claim 7, characterized in that, The step of determining the maximum number of second code blocks that a single third storage node can support for storage, based on the fault tolerance supported by the second erasure coding data and the number of disks and free storage space information of each of the plurality of third storage nodes, includes: Based on the fault tolerance supported by the second erasure coding data, determine the maximum number of second coding blocks that a single third storage node can store; Based on the number of disks corresponding to any third storage node and the free storage space information of the disks, determine the number of disks that any third storage node can accommodate a single second coding block; If the number of disks that any third storage node can hold for a single second coded block is greater than or equal to the maximum number of second coded blocks that a single third storage node can store, then the maximum number corresponding to any third storage node is determined to be the maximum number of second coded blocks that a single third storage node can store; or, if the number of disks that any third storage node can hold for a single second coded block is less than the maximum number of second coded blocks that a single third storage node can store, then the maximum number corresponding to any third storage node is determined to be the number of disks that any third storage node can hold for a single second coded block.

9. A data storage method, characterized in that, The storage cluster stores first erasure coding data stored in an erasure coding folding manner; the method includes: Obtain storage resource information of other storage nodes in the current storage cluster; the other storage nodes are storage nodes other than the multiple first storage nodes that store the first erasure coding data; Based on the storage resource information of other storage nodes in the current storage cluster, detect whether there is a second storage node among the other storage nodes that supports storing at least one first coded block; If at least one second storage node is detected, the number of first coding blocks allocated to each of the plurality of first storage nodes and the at least one second storage node is determined based on the sum of the number of second storage nodes and the number of first storage nodes and the total number of first coding blocks; The first fault tolerance of the first erasure coding data is determined based on the configuration of the data blocks and the check blocks of the first erasure coding data, and the number of first coding blocks allocated to each of the plurality of first storage nodes and the at least one second storage node. If the first fault tolerance is higher than the second fault tolerance of the first erasure code data stored in the erasure code folding manner, based on the configuration of the data blocks and check blocks of the first erasure code data, and the number of first coding blocks allocated to each of the plurality of first storage nodes and the at least one second storage node, a portion of the first coding blocks stored in the first storage node are migrated to the at least one second storage node.

10. The method according to claim 9, characterized in that, The step of determining the number of first coding blocks allocated to each of the plurality of first storage nodes and the at least one second storage node based on the sum of the number of second storage nodes and the number of first storage nodes and the total number of first coding blocks includes: If the total number of the first coded blocks is an integer multiple of the sum of the number of the second storage nodes and the number of the first storage nodes, then the number of first coded blocks allocated to each of the plurality of first storage nodes and the at least one second storage node is equal to the integer multiple. or, If the total number of the first encoded blocks is a non-integer multiple of the sum of the number of the second storage nodes and the number of the first storage nodes, then the quotient of the total number of the first encoded blocks divided by the sum of the number of the second storage nodes and the number of the first storage nodes is used as the fourth number. The fourth number of first coding blocks are allocated to each of the plurality of first storage nodes and the at least one second storage node; Allocate one remaining first coding block to some or all of the first storage nodes; If the remaining first coding blocks are allocated to some or all of the first storage nodes and the allocation is completed, then the number of first coding blocks allocated to each of the plurality of first storage nodes and the at least one second storage node is determined; or, if the remaining first coding blocks are not yet allocated after each of the plurality of first storage nodes is allocated to one remaining first coding block, then the unallocated first coding blocks are allocated to some of the second storage nodes, and the number of first coding blocks allocated to each of the plurality of first storage nodes and the at least one second storage node is determined.

11. An electronic device, characterized in that, include: A memory and a processor; wherein the memory is used to store computer programs; The processor is coupled to the memory for executing the computer program to perform the steps of the method according to any one of claims 1-10.

12. A computer-readable storage medium storing computer instructions, characterized in that, When the computer instructions are executed by one or more processors, the one or more processors are caused to perform the steps of the method according to any one of claims 1-10.

13. A computer program product, characterized in that, Includes a computer program that, when executed by one or more processors, causes the one or more processors to perform the steps of the method according to any one of claims 1-10.

Images