Configuration method and device of data storage system, equipment and storage medium

Abstract

The invention discloses a configuration method, device and equipment of a data storage system and a storage medium, and belongs to the technical field of data storage, any node of the data storage system comprises at least one capacity disk and at least one cache disk, and the at least one capacity disk comprises a plurality of capacity blocks. And constructing a redundancy group from a capacity block respectively selected from different capacity disks of the at least one node, comprising the following steps: respectively constructing a cache acceleration relationship between the at least one capacity disk and the at least one cache disk based on the disk physical properties of the at least one capacity disk and the at least one cache disk; according to the cache acceleration relation between the capacity disk corresponding to each capacity block in the redundancy group and at least one cache disk, at least one enabling cache capacity block in the redundancy group is determined, and the cache disks corresponding to the enabling cache capacity blocks in the redundancy group are different. It can be guaranteed that user data of different capacity blocks in any redundancy group cannot be cached to the same cache disk, and the reliability of data storage is improved.

Description

Technical Field

[0001] The present application relates to the technical field of data storage, and in particular to a configuration method, device, equipment and storage medium of a data storage system. Background Art

[0002] With the development of emerging technologies such as big data and cloud computing, the amount of data generated by related applications has increased dramatically. Distributed storage, as the underlying data storage supporting related applications, has brought new challenges to storage capacity and performance. Although high-performance physical media are becoming more and more mature, the capacity of high-performance storage media is generally small and cannot store all data.

[0003] In the design and implementation of distributed storage products, the general design uses high-performance storage media as cache disks to implement hot data caching, such as Solid State Drives (SSDs), and uses large-capacity low-end storage media as capacity disks to implement data persistence storage, such as Hard Disk Drives (HDDs), in order to achieve a balance between storage capacity and performance.

[0004] In traditional technology, in order to improve the input / output (IO) performance of the system, two storage pools are usually built for the cache disk and the capacity disk respectively. However, this method usually causes data to be stored separately in the two storage pools without data correspondence. Therefore, both the cache disk and the capacity disk need to consider the reliability storage of user data, especially the cache disk. Due to the limited capacity of the cache disk, it may lead to redundant design exceeding the cache disk, and thus the reliability of the data cannot be guaranteed.

[0005] Therefore, how to effectively configure the data storage system and improve the reliability of data storage is a technical problem that needs to be solved urgently. Summary of the invention

[0006] The embodiments of the present application provide a data storage system configuration method, apparatus, device and storage medium, which are used to effectively configure the data storage system and improve the reliability of data storage.

[0007] In a first aspect, an embodiment of the present application provides a configuration method for a data storage system, wherein any node of the data storage system includes at least one capacity disk and at least one cache disk, the at least one capacity disk includes a plurality of capacity blocks, and a capacity block is selected from different capacity disks of at least one node to construct a redundancy group, the method comprising:

[0008] Based on the disk physical performances of the at least one capacity disk and the at least one cache disk, respectively constructing a cache acceleration relationship between the at least one capacity disk and the at least one cache disk; and

[0009] At least one enabled cache capacity block in the redundant group is determined based on the cache acceleration relationship between the capacity disk corresponding to each capacity block in the redundant group and the at least one cache disk, wherein the cache disks corresponding to the enabled cache capacity blocks in the redundant group are different.

[0010] In some embodiments, the step of respectively establishing a cache acceleration relationship between the at least one capacity disk and the at least one cache disk based on the disk physical performance of the at least one capacity disk and the at least one cache disk includes:

[0011] Acquiring the disk physical performance of the at least one capacity disk and the at least one cache disk; and

[0012] According to the disk physical performance of the at least one cache disk and the at least one capacity disk, a first cache disk is selected from the at least one cache disk to establish a cache acceleration relationship with a first capacity disk among the at least one capacity disk, wherein the first cache disk meets the cache requirements of the first capacity disk.

[0013] In some embodiments, the disk physical performance of the first cache disk is higher than the disk physical performance of other cache disks in the at least one cache disk.

[0014] In some embodiments, the disk physical performance includes IOPS and disk capacity, and the cache requirements of the first capacity disk satisfied by the first cache disk include:

[0015] The IOPS of the first cache disk is greater than the IOPS of the first capacity disk; and

[0016] A ratio of the capacity of the first cache disk to the capacity of the first capacity disk is greater than a preset threshold.

[0017] In some embodiments, before determining at least one enabled cache capacity block in the redundant group according to the cache acceleration relationship between the capacity disk corresponding to each capacity block in the redundant group and the at least one cache disk, the method further includes:

[0018] The corresponding priority is determined based on the capacity disk information corresponding to each capacity block in the redundancy group.

[0019] In some embodiments, determining the corresponding priority based on the capacity disk information corresponding to each capacity block in the redundancy group includes:

[0020] For any capacity block of any redundant group, based on the relative starting address of the capacity block in the capacity disk information corresponding to the capacity block, confirm the offset value of the capacity block;

[0021] Calculating a hash value of the capacity block according to the offset value of the capacity block; and

[0022] The priority of each capacity block is determined according to the hash value of each capacity block.

[0023] In some embodiments, the step of confirming the priority of each capacity block according to the hash value of each capacity block includes:

[0024] If the hash values ​​of any two capacity blocks are the same, the priorities of the two capacity blocks are determined according to the identifiers of the capacity disks in the capacity disk information corresponding to the capacity blocks.

[0025] In some embodiments, determining at least one enabled cache capacity block in the redundant group according to the cache acceleration relationship between the capacity disk corresponding to each capacity block in the redundant group and the at least one cache disk includes:

[0026] If the capacity disks corresponding to at least two capacity blocks in the redundant group correspond to the same cache disk, a target capacity block is selected from the at least two capacity blocks as the cache-enabled capacity block according to the priority of each capacity block.

[0027] In some embodiments, according to a preset priority configuration rule, a first part of priorities is determined for each enabled cache capacity block, and a second part of priorities is determined for each disabled cache capacity block, wherein the lowest priority of the first part of priorities is higher than the highest priority of the second part of priorities.

[0028] In some embodiments, the cache acceleration relationship includes a relationship in which one capacity disk corresponds to one cache disk, and a relationship in which multiple capacity disks correspond to one cache disk.

[0029] In some embodiments, it also includes:

[0030] In response to a write data request, determining a first redundancy group from a plurality of pre-built redundancy groups;

[0031] Writing the user data into different capacity blocks of the first redundancy group in a backup manner;

[0032] If the first redundancy group includes at least one enabled cache capacity block, at least part of the user data in the enabled cache capacity block is moved to a corresponding target cache disk.

[0033] In some embodiments, the step of moving at least part of the user data in the cache-enabled capacity block to a corresponding target cache disk includes:

[0034] Based on a plurality of capacity slices obtained by pre-dividing each capacity block according to the minimum storage unit, writing the user data into each capacity slice;

[0035] Based on the cache lines pre-divided into each cache disk, the user data stored in at least one capacity slice in the capacity block configured to enable caching is moved to at least one cache line of the corresponding target cache disk, and a data mapping relationship between the cache line and the capacity slice is established, and the size of any cache line is the same as the size of any capacity slice.

[0036] In some embodiments, it also includes:

[0037] In response to a data read request, determining a second redundancy group corresponding to user data to be read in the request;

[0038] Based on the priorities of the capacity blocks in the second redundancy group, a reading order of the capacity blocks in the second redundancy group is determined, and user data is read according to the reading order.

[0039] In some embodiments, reading the user data in the reading order includes:

[0040] According to the reading order, respectively determine the target capacity block in each round of reading process;

[0041] For any reading process, the user data is read from the cache disk corresponding to the target capacity block of this round. If the reading fails, the user data is read from the capacity disk corresponding to the target capacity block; and

[0042] Determine the next round of target capacity blocks and execute the reading process;

[0043] When it is detected that the user data is read successfully from any cache disk or any capacity disk, the reading of the user data is terminated.

[0044] In some embodiments, the reading of the user data from the cache disk corresponding to the target capacity block of the current round, after the reading fails, further includes:

[0045] The access information of the user data is recorded. When the access information of the user data meets the preset cache condition, a data mapping relationship between the capacity slice corresponding to the user data and the free cache line of the cache disk is established, and the user data is stored in the free cache line of the corresponding cache disk.

[0046] In some embodiments, it also includes:

[0047] Based on the user data access information lists pre-created for each enabled cache capacity block in each redundancy group, when it is determined that the preset data flushing condition is reached, the user data access information lists are parsed in order of priority from low to high for each enabled cache capacity block;

[0048] Based on the parsing result of any user data access information list, the user data whose access information meets the flushing condition is flushed from the corresponding cache disk to the corresponding capacity disk.

[0049] In a second aspect, an embodiment of the present application provides a configuration device for a data storage system, wherein any node of the data storage system includes at least one capacity disk and at least one cache disk, wherein the at least one capacity disk includes a plurality of capacity blocks, and a capacity block selected from different capacity disks of at least one node is used to construct a redundancy group, including:

[0050] A construction module, configured to respectively construct a cache acceleration relationship between the at least one capacity disk and the at least one cache disk based on the disk physical performances of the at least one capacity disk and the at least one cache disk; and

[0051] A determination module is used to determine at least one enabled cache capacity block in the redundant group based on the cache acceleration relationship between the capacity disk corresponding to each capacity block in the redundant group and the at least one cache disk, wherein the cache disks corresponding to the enabled cache capacity blocks in the redundant group are different.

[0052] In a third aspect, an embodiment of the present application provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein:

[0053] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute the configuration method of the data storage system.

[0054] In a fourth aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a computer program, and the computer program is used to execute the configuration method of the above-mentioned data storage system.

[0055] In an embodiment of the present application, any node of the data storage system includes at least one capacity disk and at least one cache disk, at least one capacity disk includes multiple capacity blocks, and a capacity block is selected from different capacity disks of at least one node to form a redundant group, and based on the disk physical performance of at least one capacity disk and at least one cache disk, a cache acceleration relationship between at least one capacity disk and at least one cache disk is respectively constructed; and according to the cache acceleration relationship between the capacity disk corresponding to each capacity block in the redundant group and at least one cache disk, at least one enabled cache capacity block in the redundant group is determined, wherein the cache disks corresponding to each enabled cache capacity block in the redundant group are different, so that it can be ensured that user data of different capacity blocks in any redundant group will not be cached to the same cache disk, thereby realizing data cache acceleration and improving the reliability of data storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0057] Figure 1 A flowchart of a method for configuring a data storage system provided in an embodiment of the present application;

[0058] Figure 2 A schematic diagram of constructing a redundancy group provided in an embodiment of the present application;

[0059] Figure 3 A schematic diagram of a cache acceleration relationship provided in an embodiment of the present application;

[0060] Figure 4 A schematic diagram of a user data storage process provided in an embodiment of the present application;

[0061] Figure 5 A schematic diagram of a data mapping relationship provided in an embodiment of the present application;

[0062] Figure 6 A flow chart of a method for reading user data provided in an embodiment of the present application;

[0063] Figure 7 A schematic diagram of the structure of a configuration device for a data storage system provided in an embodiment of the present application;

[0064] Figure 8 A schematic diagram of the hardware structure of an electronic device for implementing a configuration method of a data storage system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to effectively configure a data storage system and improve the reliability of data storage, embodiments of the present application provide a method, apparatus, device, and storage medium for configuring a data storage system.

[0066] It should be noted that in the description of the present application, "multiple" is understood as "at least two". "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist, and B exists alone. A is connected to B, which can represent two situations: A is directly connected to B and A is connected to B through C. In addition, in the description of the present application, words such as "first" and "second" are only used for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order.

[0067] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0068] Figure 1 A flowchart of a method for configuring a data storage system provided in an embodiment of the present application, the method is executed by the data storage system, and the method includes the following steps.

[0069] In step 101, any node of the data storage system includes at least one capacity disk and at least one cache disk, any capacity disk includes multiple capacity blocks, and a capacity block selected from different capacity disks of at least one node is used to build a redundant group, and based on the disk physical performance of at least one capacity disk and at least one cache disk, a cache acceleration relationship between at least one capacity disk and at least one cache disk is respectively built.

[0070] In a specific implementation, the pre-constructed redundancy group may be determined according to factors such as the importance of the user data to determine the copy requirements of the user data, and then constructed according to the copy requirements of the user data. For example, some user data has a high importance level, and in order to avoid damage or loss, the data needs to be backed up in multiple copies at the same time. For example, when the importance level of the user data is set to high, the copy requirement of the user data is determined to be 3 copies. When the importance level of the user data is low, the copy requirement of the user data is determined to be 2 copies, etc. If the copy requirement of the user data is 2 copies, the constructed redundancy group is a 2-copy redundancy group, that is, one capacity block is selected from two different capacity disks of at least one node respectively. If the copy requirement of the user data is 3 copies, the constructed redundancy group is a 3-copy redundancy group, that is, one capacity block is selected from at least one The three different capacity disks of the node are respectively composed of a capacity block, wherein the capacity block is obtained by dividing each capacity disk in each node in the data storage system according to the preset capacity. It should be noted that the upper limit of the number of capacity blocks in the redundant group is the total number of capacity disks in each node in the data storage system. In this way, it can be ensured that the data is stored in different capacity blocks, thereby falling on different capacity disks, avoiding the situation where the stored data is damaged or lost due to the failure of any capacity disk, and improving the reliability of user data storage. In actual application, the capacity blocks that constitute the redundant group can be set to come from the capacity disks of different nodes. In this way, it can be ensured that when the server of a node fails, the user data stored in the redundant group can still be read from other nodes, further improving the reliability of user data storage.

[0071] Figure 2 A schematic diagram of constructing a redundancy group provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, there are three nodes in the data storage system, namely, node 1, node 2 and node 3. Node 1 has capacity disk 1, capacity disk 2 and capacity disk 3, node 2 has capacity disk 4 and capacity disk 5, and node 3 has capacity disk 6. Before constructing a redundant group, resources can be first divided into blocks for each capacity disk in each node, and each capacity disk can be divided into multiple capacity blocks. For example, each capacity disk can be divided into 5 capacity blocks, and an identification information is determined for each capacity block. Then, according to the copy requirements of user data, a capacity block is selected from different capacity disks to construct a redundant group. Here, the number of capacity blocks in each redundant group can be pre-defined by the distributed storage system, or can be determined by the distributed storage system when responding to a write data request of any user data by analyzing the user data. For example, Figure 2As shown, the constructed redundancy group A is a 3-copy redundancy group, and the 3 capacity blocks in the redundancy group A are respectively from the capacity disk 1 and the capacity disk 2 of the node 1 and the capacity disk 4 of the node 2, and the redundancy group B is a 2-copy redundancy group, and the 2 capacity blocks in the redundancy group B are respectively from the capacity disk 3 of the node 1 and the capacity disk 5 of the node 2. It should be noted that which capacity disks are selected from which nodes and which capacity block is selected from the capacity disks to form a redundancy group can be set by the technical personnel according to actual needs, and this application does not specifically limit this.

[0072] During specific implementation, the disk physical performance of at least one capacity disk and at least one cache disk can be obtained, and based on the disk physical performance of at least one cache disk and at least one capacity disk, a first cache disk is selected from at least one cache disk and a first capacity disk of at least one capacity disk is selected to establish a cache acceleration relationship, wherein the method of obtaining the physical performance of the disk may be simulated according to the IO scenario during actual use, or other acquisition methods may be used, such as using the physical performance indicators given in the product manual or using the same performance data for a unified configuration of a certain type of media, and other methods are not limited here.

[0073] In specific implementation, for any node, a cache disk whose disk physical performance is higher than that of other cache disks can be determined as the first cache disk. When constructing a cache acceleration relationship between the capacity disk and the cache disk, whether the first cache disk meets the cache requirement of the first capacity disk can be determined as a construction condition for the cache acceleration relationship between the first cache disk and the first capacity disk.

[0074] In specific implementation, the physical performance of the disk includes the input / output operations per second (IOPS) and the disk capacity. The cache disk with the highest IOPS performance among the cache disks can be determined as the cache disk with the highest disk physical performance, or the cache disk with the largest capacity among the cache disks can be determined as the cache disk with the highest disk physical performance, or the cache disk with the largest capacity can be determined as the cache disk with the highest disk physical performance, or the IOPS performance and capacity can be comprehensively considered to determine the cache disk with the highest disk physical performance, and the first capacity disk can also be selected as the one with the highest IOPS performance, or the one with the largest disk capacity, or randomly selected, and this application does not limit this herein.

[0075] To determine whether the first cache disk meets the cache requirement of the first capacity disk, it is possible to determine whether the IOPS of the first cache disk is greater than the IOPS of the first capacity disk, and whether the ratio of the capacity of the first cache disk to the capacity of the first capacity disk is greater than a preset threshold. If so, it is determined that the first cache disk meets the cache requirement of the first capacity disk, and a cache acceleration relationship between the first cache disk and the first capacity disk can be established. The cache acceleration relationship includes a relationship in which one capacity disk corresponds to one cache disk, and a relationship in which multiple capacity disks correspond to one cache disk.

[0076] In specific implementation, it is determined whether the first cache disk meets the cache requirement of the first capacity disk. The purpose is to determine whether it is suitable to configure the cache acceleration relationship. The main considerations are the IOPS performance advantage of the cache disk relative to the IOPS performance of the capacity disk and whether the cache disk capacity is sufficient to provide performance acceleration. For example, the difference between the IOPS performance of cache disk 1 and the IOPS performance of capacity disk 1 is greater than a preset performance threshold, such as the preset performance threshold is 1000, and the ratio of the capacity of cache disk 1 to the capacity of capacity disk 1 is greater than the preset capacity threshold, such as the preset capacity threshold is 1 / 5, indicating that cache disk 1 has the ability to provide cache services for capacity disk 1, and it is determined that the cache requirement is met. Otherwise, it means that cache disk 1 does not have the ability to provide cache services for capacity disk 1, and it is determined that the cache requirement is not met.

[0077] Take node 1 as an example, which includes 2 cache disks, namely cache disk 1 and cache disk 2, and 3 capacity disks, namely capacity disk 1, capacity disk 2 and capacity disk 3, and sort the capacity disks and cache disks according to IOPS performance to obtain the disk physical performance data table shown in Table 1.

[0078] Table 1

[0079]

[0080]

[0081] As shown in Table 1 above, the cache disk 1 with the highest IOPS performance among the cache disks is selected as the first cache disk, and the capacity disk 1 with the highest IOPS performance among the capacity disks is selected as the first capacity disk. For the selected cache disk 1, its IOPS performance is 3911 and its capacity is 1T. The selected capacity disk 1 has an IOPS performance of 192 and a capacity of 4T. Since 3911 is greater than 192, and the difference between 3911 and 192 is greater than the preset performance threshold of 1000, and the ratio between the capacity of cache disk 1 and the capacity of capacity disk 1 is 1 / 4, which is greater than the preset capacity threshold of 1 / 5, it is determined that cache disk 1 meets the cache performance of capacity disk 1. Requirements, then a cache acceleration relationship can be built for cache disk 1 and capacity disk 1. After the cache acceleration relationship between cache disk 1 and capacity disk 1 is built, the IOPS performance of cache disk 1 needs to be the value after deducting the IOPS of capacity disk 1, that is, 3719. This can help to identify that cache disk 1 also has the ability to provide cache services for other capacity disks, and capacity disk 1 after the cache acceleration relationship is built does not participate in the subsequent construction of cache acceleration relationships with other cache disks. Therefore, it is necessary to update the IOPS performance of each disk and the disk in Table 1 above to obtain the disk physical performance data table shown in Table 2.

[0082] Table 2

[0083]

[0084] As shown in Table 2 above, at this time, the cache disk with the highest IOPS performance among the cache disks is cache disk 2, and the capacity disk with the highest IOPS performance among the capacity disks is capacity disk 3. Therefore, cache disk 2 is selected as the first cache disk, and capacity disk 3 is selected as the first capacity disk. For the selected cache disk 2, its IOPS performance is 3774 and its capacity is 1T. For the selected capacity disk 3, its IOPS performance is 187 and its capacity is 4T. Since 3774 is greater than 187, and the difference between 3774 and 187 is greater than the preset performance threshold of 1000, and the capacity ratio of cache disk 2 to capacity disk 3 is 1 / 4, which is greater than the capacity preset threshold of 1 / 5, it is determined that cache disk 2 meets the cache requirement of capacity disk 3. Then, a cache acceleration relationship can be established for cache disk 2 and capacity disk 3. Afterwards, the IOPS performance of cache disk 2 is the value after subtracting the IOPS of capacity disk 3. The IOPS performance of each disk and the disk in Table 2 above are updated to obtain the disk performance data table shown in Table 3.

[0085] Table 3

[0086]

[0087] As shown in Table 3 above, at this time, the cache disk with the highest IOPS performance among the cache disks is cache disk 1, and the capacity disk with the highest IOPS performance among the capacity disks is capacity disk 2. Therefore, cache disk 1 is selected as the first cache disk, and capacity disk 2 is selected as the first capacity disk. For the selected cache disk 1, its IOPS performance is 3774 and its capacity is 1T. For the selected capacity disk 3, its IOPS performance is 187 and its capacity is 4T. Since 3774 is greater than 187, and the difference between 3774 and 187 is greater than the preset performance threshold of 1000, and the capacity ratio of cache disk 2 to capacity disk 3 is 1 / 4, which is greater than the preset capacity threshold of 1 / 5, it is determined that cache disk 2 meets the cache requirements of capacity disk 3, then a cache acceleration relationship can be established for cache disk 2 and capacity disk 3. Afterwards, the IOPS performance of each disk and disk in Table 2 above is updated to obtain the disk performance data table shown in Table 3. For the selected cache disk 1, its IOPS performance is 3719 and its capacity is 1T. For the selected capacity disk 2, its IOPS performance is 186 and its capacity is 4T. Since 3719 is greater than 186, and the difference between 3719 and 186 is greater than the preset performance threshold of 1000, and the capacity ratio of cache disk 1 to capacity disk 2 is 1 / 4, which is greater than the preset capacity threshold of 1 / 5, it is determined that cache disk 1 meets the cache requirements of capacity disk 2. Then, a cache acceleration relationship can be constructed for cache disk 1 and capacity disk 2. After the above configuration, a cache acceleration relationship table as shown in Table 4 can be generated.

[0088] Table 4

[0089]

[0090] Similarly, the cache acceleration relationship between the capacity disk and the cache disk can be constructed for each capacity disk of other nodes in the data storage system in the above manner. Assume that the cache acceleration relationship diagram in the data storage system finally generated is as follows: Figure 3 As shown, Figure 3 A cache acceleration relationship schematic diagram is provided for an embodiment of the present application, wherein two cache acceleration relationships are constructed in node 2, namely, cache acceleration relationship 4 constructed between cache disk 3 and capacity disk 4, and cache acceleration relationship 5 constructed between cache disk 3 and capacity disk 5; a cache acceleration relationship constructed in node 3 is cache acceleration relationship 7 constructed between cache disk 4 and capacity disk 6.

[0091] In step 102, at least one enabled cache capacity block in the redundant group is determined based on the cache acceleration relationship between the capacity disk corresponding to each capacity block in the redundant group and at least one cache disk, wherein the cache disks corresponding to the enabled cache capacity blocks in the redundant group are different.

[0092] In specific implementation, according to the cache acceleration relationship between the capacity disk corresponding to each capacity block in the redundant group and at least one cache disk, before determining at least one enabled cache capacity block in the redundant group, the corresponding priority can also be determined based on the capacity disk information corresponding to each capacity block in the redundant group.

[0093] Among them, the capacity disk information corresponding to each capacity block includes the relative starting address of the capacity block and the capacity disk identifier to which the capacity block belongs. According to the capacity disk information corresponding to each capacity block in the redundant group, the priority of each capacity block is determined. Based on the relative starting address of the capacity block in the capacity disk information corresponding to the capacity block, the offset value of the capacity block can be confirmed. According to the offset value of the capacity block, the hash value of the capacity block is calculated, and then the priority corresponding to each capacity block is confirmed according to the hash value of each capacity block.

[0094] Among them, the priority corresponding to each capacity block is confirmed according to the hash value of each capacity block. For example, the smaller the hash value is, the smaller the corresponding priority value is, and the higher the priority is. The larger the hash value is, the larger the corresponding priority value is, and the lower the priority is. If the hash values ​​of any two capacity blocks are the same, the priorities of the two capacity blocks are determined according to the capacity disk identifiers to which the capacity blocks belong. For example, the smaller the capacity disk identifier is, the smaller the priority value is, and the higher the priority is. The larger the capacity disk identifier is, the larger the priority value is, and the lower the priority is. In addition, the priority can also be determined by other methods, which are not limited here.

[0095] For example, by analyzing the capacity disk information of each capacity block of the redundant group M, the redundant group P, and the redundant group Q, the hash value calculation results of each capacity block of the redundant group as shown in Table 5 can be obtained.

[0096] Table 5

[0097]

[0098] As shown in Table 5, taking the capacity block identification information 01001 as an example, "01" represents the capacity disk identification information corresponding to the capacity block, and "001" represents the offset value of the capacity block. For the redundant group M, the hash values ​​of the three capacity blocks are 1, 2 and 3 respectively. According to the above priority determination method, it can be obtained that the priority of capacity block 01001 is 1, the priority of capacity block 02002 is 2, and the priority of capacity block 03003 is 3.

[0099] For redundant group P, the hash values ​​of the three capacity blocks are 4, 4 and 2 respectively. Since the hash value of capacity block 03002 is the smallest, it is determined to have the highest priority, which is 1. The hash values ​​of capacity blocks 01004 and 02004 are the same, but the capacity disk identifier of 01004 is 1, and the capacity disk identifier of 02004 is 2. According to the principle that the smaller the capacity disk identifier, the smaller the priority value, the higher the priority, it can be determined that the priority of capacity block 01004 is higher than that of capacity block 02004, that is, the priority number of capacity block 01004 is 2, and the priority number of capacity block 02004 is 3.

[0100] For redundancy group Q, the hash values ​​of the capacity blocks are not equal, and the hash value of capacity block 04006 is less than the hash value of capacity block 05007. Therefore, the priority of capacity block 04006 is 1 and the priority of capacity block 05007 is 2. The priority calculation results shown in Table 6 are generated according to the above priority setting method.

[0101] Table 6

[0102]

[0103] In a specific implementation, it can be determined by whether the cache resources are sufficient to determine which capacity blocks in the redundant group can be determined as enabled cache capacity blocks. When the cache resources are sufficient, that is, the capacity disks corresponding to each capacity block in the redundant group correspond to different cache disks. At this time, each capacity block in the redundant group can be determined as an enabled cache capacity block. The capacity block configured as an enabled cache capacity block indicates that user data in the capacity block can be stored on the cache disk when the condition that the cache disk with the capacity disk corresponding to the capacity block has established a cache acceleration relationship is met. The capacity block not configured as an enabled cache capacity block indicates that user data in the capacity block cannot be stored on any cache disk. When the cache resources are insufficient, that is, the capacity disks corresponding to at least two capacity blocks in the redundant group correspond to the same cache disk, at this time, a target capacity block can be selected from at least two capacity blocks corresponding to the same cache disk as the enabled cache capacity block according to the priority of each capacity block, wherein the target capacity block is the capacity block with the highest priority, and the remaining other capacity blocks in the at least two capacity blocks are determined as disabled capacity blocks.

[0104] For example, redundant group H includes three capacity blocks, namely capacity block 1, capacity block 2 and capacity block 3, among which capacity block 1 corresponds to capacity disk 1, capacity block 2 corresponds to capacity disk 2, capacity block 3 corresponds to capacity disk 3, and capacity disk 1 corresponds to cache disk 1, capacity disk 2 corresponds to cache disk 2, and capacity disk 3 corresponds to cache disk 3. Then, it can be determined that the cache resources are sufficient, and each capacity block in the redundant group H can be determined as a cache-enabled capacity block.

[0105] For another example, the redundant group K includes three capacity blocks, namely, capacity block 4, capacity block 5 and capacity block 6, wherein capacity block 4 corresponds to capacity disk 1, capacity block 5 corresponds to capacity disk 2, capacity block 6 corresponds to capacity disk 3, and capacity disk 1 corresponds to cache disk 1, capacity disk 2 corresponds to cache disk 1, and capacity disk 3 corresponds to cache disk 2. Then, it can be determined that the cache resources are insufficient. Since the capacity disks corresponding to capacity blocks 4 and 5 in the redundant group K both correspond to cache disk 1, only one of capacity block 4 and capacity block 5 can be determined as a cache-enabled capacity block. If the priority of capacity block 4 is higher than that of capacity block 5, capacity block 4 is determined as a cache-enabled capacity block. If the priority of capacity block 5 is higher than that of capacity block 4, capacity block 5 is determined as a cache-enabled capacity block. Since the cache disk 2 corresponding to the capacity disk corresponding to capacity block 6 does not have a cache acceleration relationship with the capacity disks corresponding to other capacity blocks in the redundant group, the cache resources are sufficient for capacity block 6, so capacity block 6 can also be determined as a cache-enabled capacity block. In this way, capacity block 4 and capacity block 6 in redundant group K are both determined as enabled cache capacity blocks, and capacity block 5 is a disabled capacity block.

[0106] It should be noted that if the capacity disks corresponding to the capacity blocks of any redundant group do not correspond to any cache disk, that is, the capacity disks corresponding to the capacity blocks do not establish a cache acceleration relationship with any cache disk, then all capacity blocks in the redundant group are disabled capacity blocks.

[0107] In specific implementation, since the capacity block determined as the enabled capacity block is, the user data in the capacity block can be stored on the corresponding cache disk when the conditions for storing on the cache disk corresponding to the capacity disk corresponding to the capacity block are met. In order to speed up the cache reading speed, it is necessary to set the priority to read user data from the enabled capacity block. Therefore, the priority of the enabled capacity block is higher than that of the disabled capacity block.

[0108] In specific implementation, after determining the enabled cache capacity blocks and disabled capacity blocks for each capacity block of any redundant group, it is also possible to determine the first part of the priority for each enabled cache capacity block and the second part of the priority for each disabled cache capacity block according to the preset priority configuration rules, wherein the lowest priority of the first part of the priorities is higher than the highest priority of the second part of the priorities.

[0109] For example, the redundant group T includes 3 capacity blocks, and the priority calculation results of each capacity block in the redundant group T are shown in Table 7.

[0110] Table 7

[0111]

[0112]

[0113] As shown in Table 7, if the capacity disks corresponding to capacity block 01001 and capacity block 02002 correspond to the same cache disk a, and the capacity disk corresponding to capacity block 03003 corresponds to cache disk b, then it can be determined that capacity block 01001 with a higher priority among capacity block 01001 and capacity block 02002 is a cache-enabled capacity block, capacity block 02002 is a disabled capacity block, and capacity block 03003 is a cache-enabled capacity block. Afterwards, since capacity block 02002 is a disabled capacity block and capacity block 03003 is a cache-enabled capacity block, but the priority of capacity block 02002 is higher than that of capacity block 03003, it is necessary to perform cache operations on capacity block 01001 and capacity block 02002. 2002 and capacity block 03003 are re-determined the priority. The first part of the priority is determined according to the priority determination principle for capacity block 01001 and capacity block 03003 in the enabled cache capacity block. Since the priority of capacity block 01001 is higher than the priority of capacity block 03003, it can be obtained that the first part of the priority of capacity block 01001 is 1, and the first part of the priority of capacity block 03003 is 2. The second part of the priority is determined according to the priority determination principle for capacity block 02002 in the disabled cache capacity block. It can be obtained that the second part of the priority of capacity block 02002 is 3. Therefore, the priority of each capacity block is updated as shown in Table 8.

[0114] Table 8

[0115]

[0116] In the embodiment of the present application, it is ensured that the user data stored in each capacity block in a redundant group will not be mapped to the same cache disk, which can ensure that the original user data copy redundancy mechanism of the system will not be reduced due to the use of the cache mechanism, and thus ensure that the reliability of the user data is not affected, and finally achieve reliable cache of the system. In this way, whether it is a cache disk failure or a capacity disk failure, the user data still maintains the original fault recovery capability, and the lost user data is reconstructed through the remaining normal redundant copies.

[0117] In an embodiment of the present application, for a write data request for user data, a first redundant group can be determined from a plurality of pre-constructed redundant groups, and the user data can be written in a backup manner into different capacity blocks of the first redundant group. If the first redundant group includes at least one enabled cache capacity block, it will be configured to enable at least part of the user data in the cache capacity block to be moved to the corresponding target cache disk, thereby achieving cache write acceleration.

[0118] In specific implementation, for a redundant group, user data will store multiple identical copies in different capacity blocks. For different user data in the same capacity block, the storage is implemented in the capacity block as multiple fixed-size capacity slices Block, where Block is the minimum storage unit, and the resources of the cache disk are also stored by multiple fixed-size cache lines Cache Line, where cache line is the minimum operation unit, and the size of any cache line is the same as the size of any capacity slice. Therefore, it will be configured to enable at least part of the user data in the cache capacity block to be moved to the corresponding target cache disk. In fact, user data can also be written to each capacity slice based on multiple capacity slices obtained by pre-dividing each capacity block according to the minimum storage unit, and then based on the cache lines pre-divided for each cache disk, the user data stored in at least one capacity slice in the capacity block configured to enable the cache will be moved to at least one cache line of the corresponding target cache disk, and a data mapping relationship between the cache line and the capacity slice will be established.

[0119] Figure 4 A schematic diagram of a user data storage process provided in an embodiment of the present application is shown in FIG. Figure 4 As shown in the figure, take the redundant group G as an example, the redundant group G is a 3-copy redundant group, and the capacity blocks in the redundant group are capacity block 3, capacity block 8 and capacity block 12 respectively. When the user data is written, the user data is first written into the capacity slice of each capacity block in the form of a copy, so as to be stored in the capacity disk corresponding to each capacity block. Afterwards, when the cache disk storage condition is met, for example, the cache disk corresponding to each capacity disk has an idle Cache Line, the user data stored in the capacity slice is moved to the CacheLine of the cache disk to realize cache writing. Figure 5 A data mapping relationship diagram provided in an embodiment of the present application is as follows: Figure 5 As shown, the capacity block 3 of the redundant group G comes from the capacity disk 1, and a Block of the capacity block 3 storing user data is written into the Block of the capacity block 3 in the capacity disk 1. The Block of the capacity block 3 in the capacity disk 1 can establish a data mapping relationship with any Cache Line of the cache disk 1 that has a cache acceleration relationship.

[0120] In some embodiments, since a priority is set for each capacity block, during the data reading process of the stored user data, the data reading process may also be performed in the following manner: Figure 6 A flow chart of a method for reading user data provided in an embodiment of the present application, the method comprising the following steps.

[0121] In step 601, in response to a data read request, a second redundancy group corresponding to user data to be read in the request is determined.

[0122] The data read request may include information such as identification information or a hash value of the user data to be read. The identification information or the hash value of the user data to be read may be used to determine a second redundancy group storing the user data to be read, wherein the second redundancy group may be the same redundancy group as the first redundancy group, or may be a different redundancy group from the first redundancy group.

[0123] In step 602, based on the priority of each capacity block in the second redundancy group, the reading order of each capacity block in the second redundancy group is determined, and the user data is read according to the reading order.

[0124] In specific implementation, due to the equivalence relationship between the capacity blocks storing user data, any capacity block storing user data can be selected to meet the user data reading requirement. In the process of writing user data, the priority of each capacity block is determined based on the cache acceleration relationship between the target capacity disk corresponding to each capacity block and the corresponding target cache disk, and whether to enable cache capacity blocks or disable cache capacity blocks for each capacity block. Therefore, the reading order of the target capacity blocks in the second redundancy group can actually be determined in descending order of priority.

[0125] In specific implementation, after determining the reading order, the target capacity blocks in each round of reading process can be determined according to the reading order. For any reading process, the user data is read from the cache disk corresponding to the target capacity block of this round. If the reading fails, the user data is read from the capacity disk corresponding to the target capacity block. At the same time, the target capacity block for the next round is determined, and the reading process is executed. When it is detected that the user data is read successfully from any cache disk or any capacity disk, the reading of the user data is terminated.

[0126] In specific implementation, when the user data is not read from the cache disk corresponding to the target capacity block of this round, for example, the cache resources of the corresponding cache disk do not meet the demand when the user data to be read is written, and thus the user data to be read cannot be stored in the second target cache disk. At this time, the user data can be read from the capacity disk corresponding to the target capacity block, but generally the speed of reading user data from the capacity disk is slow. Therefore, when it is determined that the reading fails, the process of reading user data can be executed again according to the target capacity block of the next round, and so on, until it is detected that the user data is read successfully from any cache disk or any capacity disk, and the reading of user data is terminated, thereby achieving cache read acceleration.

[0127] In actual applications, after writing user data to a capacity disk or a cache disk, some user data is accessed very frequently, while some user data is accessed very infrequently. The access popularity includes, for example, the number of accesses and the access duration. Therefore, if a certain user data is not stored in the cache disk, but the number of accesses to the user data is relatively frequent, and each time the user data is read, it is read from the corresponding target capacity disk storing the user data, and cache read acceleration is not achieved, for this type of user data, cache read can actually be achieved in the following way to speed up the reading speed.

[0128] In specific implementation, the access information of user data is recorded. When the access information of user data meets the preset cache conditions, a data mapping relationship between the capacity slice corresponding to the user data and the free cache line of the cache disk is established, and the user data is stored in the free cache line of the corresponding cache disk.

[0129] In this way, the next time there is a request to read the user data, the user data can be directly read from the cache disk, thereby achieving read IO acceleration.

[0130] In actual applications, since the access popularity of each user data is different, if a type of user data is written to the cache disk but the access popularity is relatively low, this type of user data will occupy a large amount of cache disk resources, which may cause some user data with relatively high access popularity to be unable to be written to the cache disk to achieve read IO acceleration. Therefore, for this type of situation, it is actually possible to regularly clean up the user data in the cache disk and flush the user data with relatively low access popularity back to the corresponding capacity disk to release cache disk resources.

[0131] In specific implementation, based on the user data access information list created in advance for each enabled cache capacity block in each redundant group, when it is determined that the preset data flushing condition is met, each user data access information list is parsed in order from low to high priority of each enabled cache capacity block. Based on the parsing result of any user data access information list, the user data whose access information meets the flushing condition is flushed from the corresponding cache disk to the corresponding capacity disk.

[0132] For example, the current priority and enabling information of each capacity block in each redundant group is shown in Table 9.

[0133] Table 9

[0134]

[0135]

[0136] As shown in Table 9 above, the enabled capacity blocks in the redundant group X are capacity block 01001 and capacity block 03003, the enabled capacity blocks in the redundant group Y are capacity block 04001 and capacity block 06003, and the enabled capacity block in the redundant group Z is capacity block 08002. A user data access information list is created for each enabled capacity block, and the priority of capacity block 01001 is 1. Each user data access information list is used to record the access information of the user data stored in the corresponding capacity block, such as the number of accesses and the access duration. When it is determined that the preset data flushing condition is reached, such as reaching the preset period, the preset period is such as one week, one month, three months, etc. According to the order of priority of each enabled cache capacity block from low to high, for the redundancy group X, the user data access information list corresponding to the capacity block 03003 with a low priority can be parsed first, and the user data whose access information meets the flushing condition, such as the number of accesses is lower than the threshold, is flushed from the corresponding cache disk to the corresponding capacity disk. After a certain period of time, the user data access information list corresponding to the capacity block 01001 with a high priority is parsed first, and the user data whose access information meets the flushing condition, such as the number of accesses is lower than the threshold, is flushed from the corresponding cache disk to the corresponding capacity disk. The other redundancy groups are similar and will not be repeated here.

[0137] In this way, through the hierarchical control of cache flushing, user data with low access popularity can be flushed back to the capacity disk in batches, so that user data can be stored in the cache disk for the longest time and read first, thereby maximizing the use of cache disk resources and improving the utilization rate of cache disk resources.

[0138] Based on the same technical concept, an embodiment of the present application also provides a configuration device for a data storage system. The principle of solving the problem by the configuration device for the data storage system is similar to that of the configuration method for the data storage system mentioned above. Therefore, the implementation of the configuration device for the data storage system can refer to the implementation of the configuration method for the data storage system, and the repeated parts will not be repeated.

[0139] Figure 7 A structural schematic diagram of a configuration device for a data storage system provided in an embodiment of the present application, wherein any node of the data storage system includes at least one capacity disk and at least one cache disk, the at least one capacity disk includes multiple capacity blocks, and a capacity block selected from different capacity disks of at least one node is used to construct a redundancy group, including a construction module 701 and a first determination module 702.

[0140] A construction module 701 is used to respectively construct a cache acceleration relationship between the at least one capacity disk and the at least one cache disk based on the disk physical performances of the at least one capacity disk and the at least one cache disk; and

[0141] The first determination module 702 is used to determine at least one enabled cache capacity block in the redundant group according to the cache acceleration relationship between the capacity disk corresponding to each capacity block in the redundant group and the at least one cache disk, wherein the cache disks corresponding to the enabled cache capacity blocks in the redundant group are different.

[0142] In some embodiments, the construction module 701 is specifically used to:

[0143] Acquiring the disk physical performance of the at least one capacity disk and the at least one cache disk; and

[0144] According to the disk physical performance of the at least one cache disk and the at least one capacity disk, a first cache disk is selected from the at least one cache disk to establish a cache acceleration relationship with a first capacity disk among the at least one capacity disk, wherein the first cache disk meets the cache requirements of the first capacity disk.

[0145] In some embodiments, the disk physical performance of the first cache disk is higher than the disk physical performance of other cache disks in the at least one cache disk.

[0146] In some embodiments, the disk physical performance includes IOPS and disk capacity, and the cache requirements of the first capacity disk satisfied by the first cache disk include:

[0147] The IOPS of the first cache disk is greater than the IOPS of the first capacity disk; and

[0148] A ratio of the capacity of the first cache disk to the capacity of the first capacity disk is greater than a preset threshold.

[0149] In some embodiments, before the determination module determines at least one enabled cache capacity block in the redundant group according to the cache acceleration relationship between the capacity disk corresponding to each capacity block in the redundant group and the at least one cache disk, the first determination module 702 is further used to:

[0150] The corresponding priority is determined based on the capacity disk information corresponding to each capacity block in the redundancy group.

[0151] In some embodiments, the first determining module 702 is specifically configured to:

[0152] For any capacity block of any redundant group, based on the relative starting address of the capacity block in the capacity disk information corresponding to the capacity block, confirm the offset value of the capacity block;

[0153] Calculating a hash value of the capacity block according to the offset value of the capacity block; and

[0154] The priority of each capacity block is determined according to the hash value of each capacity block.

[0155] In some embodiments, the first determining module 702 is specifically configured to:

[0156] If the hash values ​​of any two capacity blocks are the same, the priorities of the two capacity blocks are determined according to the identifiers of the capacity disks in the capacity disk information corresponding to the capacity blocks.

[0157] In some embodiments, the first determining module 702 is specifically configured to:

[0158] If the capacity disks corresponding to at least two capacity blocks in the redundant group correspond to the same cache disk, a target capacity block is selected from the at least two capacity blocks as the cache-enabled capacity block according to the priority of each capacity block.

[0159] In some embodiments, it also includes:

[0160] The second determination module 703 is used to determine the first part of priorities for each enabled cache capacity block and the second part of priorities for each disabled cache capacity block according to a preset priority configuration rule, wherein the lowest priority of the first part of priorities is higher than the highest priority of the second part of priorities.

[0161] In some embodiments, the cache acceleration relationship includes a relationship in which one capacity disk corresponds to one cache disk, and a relationship in which multiple capacity disks correspond to one cache disk.

[0162] In some embodiments, it also includes:

[0163] A data writing module 704, configured to determine a first redundancy group from a plurality of pre-constructed redundancy groups in response to a data writing request;

[0164] Writing the user data into different capacity blocks of the first redundancy group in a backup manner;

[0165] If the first redundancy group includes at least one enabled cache capacity block, at least part of the user data in the enabled cache capacity block is moved to a corresponding target cache disk.

[0166] In some embodiments, the data writing module 704 is specifically used to:

[0167] Based on a plurality of capacity slices obtained by pre-dividing each capacity block according to the minimum storage unit, writing the user data into each capacity slice;

[0168] Based on the cache lines pre-divided into each cache disk, the user data stored in at least one capacity slice in the capacity block configured to enable caching is moved to at least one cache line of the corresponding target cache disk, and a data mapping relationship between the cache line and the capacity slice is established, and the size of any cache line is the same as the size of any capacity slice.

[0169] In some embodiments, it also includes:

[0170] A data reading module 705, configured to determine, in response to a data reading request, a second redundancy group corresponding to user data to be read in the request;

[0171] Based on the priorities of the capacity blocks in the second redundancy group, a reading order of the capacity blocks in the second redundancy group is determined, and user data is read according to the reading order.

[0172] In some embodiments, the data reading module 705 is specifically used to:

[0173] According to the reading order, respectively determine the target capacity block in each round of reading process;

[0174] For any reading process, the user data is read from the cache disk corresponding to the target capacity block of this round. If the reading fails, the user data is read from the capacity disk corresponding to the target capacity block; and

[0175] Determine the next round of target capacity blocks and execute the reading process;

[0176] When it is detected that the user data is read successfully from any cache disk or any capacity disk, the reading of the user data is terminated.

[0177] In some embodiments, it also includes:

[0178] The data mapping module 706 is used to read the user data from the cache disk corresponding to the target capacity block of this round in the data reading module 705, and record the access information of the user data after the reading fails. When the access information of the user data meets the preset cache condition, a data mapping relationship between the capacity slice corresponding to the user data and the free cache line of the cache disk is established, and the user data is stored in the free cache line of the corresponding cache disk.

[0179] In some embodiments, it also includes:

[0180] The data flush module 707 is used to parse the user data access information lists in order of priority from low to high for each enabled cache capacity block in each redundancy group based on the user data access information lists created in advance for each enabled cache capacity block in each redundancy group when it is determined that the preset data flush condition is met;

[0181] Based on the parsing result of any user data access information list, the user data whose access information meets the flushing condition is flushed from the corresponding cache disk to the corresponding capacity disk.

[0182] The division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, the functional modules in the embodiments of the present application may be integrated in a processor, or may exist physically separately, or two or more modules may be integrated in one module. The coupling between the modules may be achieved through some interfaces, which are usually electrical communication interfaces, but it is not excluded that they may be mechanical interfaces or other forms of interfaces. Therefore, the modules described as separate components may or may not be physically separated, and may be located in one place or distributed to different locations of the same or different devices. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0183] After introducing the configuration method and device of the data storage system according to the exemplary embodiment of the present application, next, an electronic device according to another exemplary embodiment of the present application is introduced.

[0184] Refer to the following Figure 8 The electronic device 130 implemented according to this embodiment of the present application is described. Figure 8 The electronic device 130 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0185] like Figure 8As shown, the electronic device 130 is in the form of a general electronic device. The components of the electronic device 130 may include but are not limited to: the at least one processor 131, the at least one memory 132, and a bus 133 connecting different system components (including the memory 132 and the processor 131).

[0186] Bus 133 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

[0187] The memory 132 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 1321 and / or a cache memory 1322 , and may further include a read-only memory (ROM) 1323 .

[0188] The memory 132 may also include a program / utility 1325 having a set (at least one) of program modules 1324, such program modules 1324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0189] The electronic device 130 may also communicate with one or more external devices 134 (e.g., keyboards, pointing devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 130, and / or communicate with any device that enables the electronic device 130 to communicate with one or more other electronic devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 135. Furthermore, the electronic device 130 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 136. As shown, the network adapter 136 communicates with other modules for the electronic device 130 via a bus 133. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0190] In an exemplary embodiment, a storage medium is also provided, and when a computer program in the storage medium is executed by a processor of an electronic device, the electronic device can perform the configuration method of the data storage system. Optionally, the storage medium can be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0191] In an exemplary embodiment, the electronic device of the present application may include at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor may execute the steps of any data storage system configuration method provided in the embodiments of the present application.

[0192] In an exemplary embodiment, a computer program product is also provided. When the computer program product is executed by an electronic device, the electronic device can implement any exemplary method provided in the present application.

[0193] Furthermore, the computer program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0194] The program product for configuring the data storage system in the embodiment of the present application may adopt a CD-ROM and include program code, and may be run on a computing device. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0195] The readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0196] The program code contained on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, radio frequency (RF), etc., or any suitable combination of the foregoing.

[0197] Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device via any type of network such as a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0198] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0199] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0200] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0201] The present application is described with reference to the flowcharts and / or block diagrams of the methods, apparatus (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0202] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0203] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0204] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0205] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for configuring a data storage system, characterized in that: Any node of the data storage system includes at least one capacity disk and at least one cache disk, the at least one capacity disk includes a plurality of capacity blocks, and a capacity block is selected from different capacity disks of at least one node to form a redundancy group, and the method includes: Based on the disk physical performances of the at least one capacity disk and the at least one cache disk, respectively constructing a cache acceleration relationship between the at least one capacity disk and the at least one cache disk; and At least one enabled cache capacity block in the redundant group is determined based on the cache acceleration relationship between the capacity disk corresponding to each capacity block in the redundant group and the at least one cache disk, wherein the cache disks corresponding to the enabled cache capacity blocks in the redundant group are different.

2. The method according to claim 1, characterized in that The step of respectively establishing a cache acceleration relationship between the at least one capacity disk and the at least one cache disk based on the disk physical performance of the at least one capacity disk and the at least one cache disk comprises: Acquiring the disk physical performance of the at least one capacity disk and the at least one cache disk; and According to the disk physical performance of the at least one cache disk and the at least one capacity disk, a first cache disk is selected from the at least one cache disk to establish a cache acceleration relationship with a first capacity disk among the at least one capacity disk, wherein the first cache disk meets the cache requirements of the first capacity disk.

3. The method according to claim 2, characterized in that The disk physical performance of the first cache disk is higher than the disk physical performance of other cache disks in the at least one cache disk.

4. The method according to claim 2, characterized in that The physical performance of the disk includes IOPS and disk capacity. The cache requirements of the first capacity disk satisfied by the first cache disk include: The IOPS of the first cache disk is greater than the IOPS of the first capacity disk; and A ratio of the capacity of the first cache disk to the capacity of the first capacity disk is greater than a preset threshold.

5. The method according to claim 1, characterized in that Before determining at least one enabled cache capacity block in the redundant group according to the cache acceleration relationship between the capacity disk corresponding to each capacity block in the redundant group and the at least one cache disk, the method further includes: The corresponding priority is determined based on the capacity disk information corresponding to each capacity block in the redundancy group.

6. The method according to claim 5, characterized in that The determining the corresponding priority based on the capacity disk information corresponding to each capacity block in the redundancy group includes: For any capacity block of any redundant group, based on the relative starting address of the capacity block in the capacity disk information corresponding to the capacity block, confirm the offset value of the capacity block; Calculating a hash value of the capacity block according to the offset value of the capacity block; and The priority of each capacity block is determined according to the hash value of each capacity block.

7. The method according to claim 6, characterized in that The step of confirming the priority of each capacity block according to the hash value of each capacity block includes: If the hash values ​​of any two capacity blocks are the same, the priorities of the two capacity blocks are determined according to the identifiers of the capacity disks in the capacity disk information corresponding to the capacity blocks.

8. The method according to claim 6, characterized in that The step of determining at least one enabled cache capacity block in the redundant group according to the cache acceleration relationship between the capacity disk corresponding to each capacity block in the redundant group and the at least one cache disk comprises: If the capacity disks corresponding to at least two capacity blocks in the redundant group correspond to the same cache disk, a target capacity block is selected from the at least two capacity blocks as the cache-enabled capacity block according to the priority of each capacity block.

9. The method according to claim 1, characterized in that Also includes: According to the preset priority configuration rule, a first priority is determined for each enabled cache capacity block, and a second priority is determined for each disabled cache capacity block, wherein the lowest priority of the first priority is higher than the highest priority of the second priority.

10. The method according to claim 1, characterized in that The cache acceleration relationship includes a relationship in which one capacity disk corresponds to one cache disk, and a relationship in which multiple capacity disks correspond to one cache disk.

11. The method according to claim 1, characterized in that Also includes: In response to a write data request, determining a first redundancy group from a plurality of pre-built redundancy groups; Writing the user data into different capacity blocks of the first redundancy group in a backup manner; If the first redundancy group includes at least one enabled cache capacity block, at least part of the user data in the enabled cache capacity block is moved to a corresponding target cache disk.

12. The method according to claim 11, characterized in that The step of moving at least part of the user data in the cache-enabled capacity block to a corresponding target cache disk includes: Based on a plurality of capacity slices obtained by pre-dividing each capacity block according to the minimum storage unit, writing the user data into each capacity slice; Based on the cache lines pre-divided into each cache disk, the user data stored in at least one capacity slice in the capacity block configured to enable caching is moved to at least one cache line of the corresponding target cache disk, and a data mapping relationship between the cache line and the capacity slice is established, and the size of any cache line is the same as the size of any capacity slice.

13. The method according to claim 1, characterized in that Also includes: In response to a data read request, determining a second redundancy group corresponding to user data to be read in the request; Based on the priorities of the capacity blocks in the second redundancy group, a reading order of the capacity blocks in the second redundancy group is determined, and user data is read according to the reading order.

14. The method according to claim 13, characterized in that The reading of user data in the reading order includes: According to the reading order, respectively determine the target capacity block in each round of reading process; For any reading process, the user data is read from the cache disk corresponding to the target capacity block of this round. If the reading fails, the user data is read from the capacity disk corresponding to the target capacity block; and Determine the next round of target capacity blocks and execute the reading process; When it is detected that the user data is read successfully from any cache disk or any capacity disk, the reading of the user data is terminated.

15. The method according to claim 14, characterized in that The step of reading the user data from the cache disk corresponding to the target capacity block of this round, after the reading fails, further includes: The access information of the user data is recorded. When the access information of the user data meets the preset cache condition, a data mapping relationship between the capacity slice corresponding to the user data and the free cache line of the cache disk is established, and the user data is stored in the free cache line of the corresponding cache disk.

16. The method according to any one of claims 1 to 15, characterized in that: Also includes: Based on the user data access information lists pre-created for each enabled cache capacity block in each redundancy group, when it is determined that the preset data flushing condition is reached, the user data access information lists are parsed in order of priority from low to high for each enabled cache capacity block; Based on the parsing result of any user data access information list, the user data whose access information meets the flushing condition is flushed from the corresponding cache disk to the corresponding capacity disk.

17. A configuration device for a data storage system, characterized in that: Any node of the data storage system includes at least one capacity disk and at least one cache disk, the at least one capacity disk includes a plurality of capacity blocks, and a capacity block selected from different capacity disks of at least one node is used to construct a redundancy group, including: A construction module, configured to respectively construct a cache acceleration relationship between the at least one capacity disk and the at least one cache disk based on the disk physical performances of the at least one capacity disk and the at least one cache disk; and The first determination module is used to determine at least one enabled cache capacity block in the redundant group according to the cache acceleration relationship between the capacity disk corresponding to each capacity block in the redundant group and the at least one cache disk, wherein the cache disks corresponding to the enabled cache capacity blocks in the redundant group are different.

18. An electronic device, characterized in that: include: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the method according to any one of claims 1 to 16.

19. A computer storage medium, characterized in that: The computer storage medium stores a computer program for executing the method according to any one of claims 1-16.

Images