A method for determining block coding of a storage system and a storage system

Abstract

The present application discloses a method for determining block coding of a storage system and a storage system, which relates to the technical field of storage systems. The method includes: dividing the spatial distribution of the independent redundant array of the storage system into multiple data groups with the total number of disks as a cycle period, repeating the stripe distribution pattern of each data group within the cycle period, so that the spatial distribution is strictly aligned on the physical storage, and optimizing the spatial distribution of the independent redundant array of the storage system. Regardless of whether the number of disks is odd or even, a unified cycle model is adopted to facilitate continuous read and write operations of the disk array, and query the cache structure according to the data group number. If the cache hits, the block number is returned. If it misses, the block number is dynamically calculated and the calculation result is written to the corresponding position, simplifying the steps of solving the block coding. Through the cycle space distribution structure and cache acceleration mechanism, the problem of complex and time-consuming block number calculation in the prior art is solved, and the performance and versatility of the storage system are significantly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of storage systems, and in particular to a method for determining block coding of a storage system and a storage system. Background Art

[0002] Storage systems are complex theoretical models, often combining a series of independent, redundant disks into a RAID array. Currently, common RAID array models include RAID 0, RAID 1, RAID 5, and RAID 6. RAID 0 uses a data striping strategy to evenly distribute data across the disk array. However, due to its lack of redundancy, a single disk failure can result in complete data loss. This high-risk nature makes RAID 0 less widely used in practical applications. RAID 1 uses mirroring to synchronously back up user data to additional disks, with the two disks acting as mirror copies of each other. If data on one disk is lost, the remaining disk can fully recover the data. In scenarios where the probability of disk failure is low, RAID 1 can significantly waste disk space and require duplicate writes, significantly reducing write performance. RAID 5 offers a redundancy level of 1. Even if any disk in the array fails, the data on the failed disk can be restored through computation using data from the remaining disks. Developed on this basis, RAID 6 boasts a redundancy of 2, further enhancing data security and thus gaining widespread adoption in practical applications. Today, storage system evaluation criteria are no longer limited to security; performance metrics have also become key considerations. IOPS (Input / Output Operations Per Second) is a core parameter for measuring storage system performance. The shorter the time taken for a single IO operation, the higher the number of IO operations completed per unit time (IOPS).

[0003] RAID 6 is an array consisting of 4 to 16 disks, with two hot spares. In the RAID 6 architecture, the concepts of stripe and stripe are crucial. A stripe consists of multiple blocks, each with a corresponding number (stripnum). Blocks with a stripnum of 0 are called p-blocks, and blocks with a stripnum of 1 are called q-blocks. Blocks p and q serve as parity blocks, storing parity information between data blocks. The remaining blocks are data blocks, responsible for actual data storage. It is worth noting that the size of the q-parity block is 1 / 16 larger than the other blocks. For example, when the data block size is 256KB, the q-parity block size is 272KB. This requires spatial alignment. However, existing technologies for achieving spatial alignment require block number calculation for each given disk number and stripe number, while also distinguishing between even and even disks. If the number of disks is even, the p and q blocks need to be swapped. The whole process is complicated and time-consuming, affecting the efficiency of the storage system. Summary of the Invention

[0004] The present application provides a method for determining block codes of a storage system and a storage system to solve the defects of the prior art storage system independent redundant array in that the original block code calculation process is complex and time-consuming, resulting in low operating efficiency and poor performance of the storage system.

[0005] This application provides a method for determining block encoding of a storage system, comprising:

[0006] The space distribution of the independent redundant array of storage systems is divided into N data groups with a cycle period, where N is the total number of disks, each data group includes N consecutive stripes, and the stripe distribution pattern of all data groups is repeated in the cycle period N;

[0007] Create and initialize a cache structure based on the spatial distribution of the redundant array of independent storage systems to store the first stripe distribution of each data group;

[0008] Get the received disk number and stripe number, and calculate the corresponding data group number. Respond to the received disk number and stripe number, and calculate the corresponding data group number. Query the cache structure based on the data group number. If a hit is found, the block number is directly returned. If not, the block number is dynamically calculated and the calculation result is written to the corresponding location in the cache.

[0009] Output the target block number for data reading, writing, verification and recovery.

[0010] The present application also provides a storage system that executes any of the above-mentioned methods for determining block coding of the storage system to perform block coding management.

[0011] The present application also provides a distributed storage system, in which multiple nodes collaborate with any of the above-mentioned methods to execute the block coding determination method of the storage system to perform consistency management of data blocks and parallel data access.

[0012] The present application also provides an electronic device, comprising:

[0013] Memory for storing computer programs;

[0014] A processor is configured to implement the steps of any of the above-mentioned methods for determining block coding of a storage system when executing the computer program.

[0015] The present application also provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the steps of any of the above-mentioned methods for determining block coding of a storage system.

[0016] The present application also provides a computer program product, including computer instructions, which are used to enable a computer to execute the steps of the block coding determination method of the storage system of the first aspect or any corresponding embodiment thereof.

[0017] This application divides the spatial distribution of the independent redundant array of the storage system into multiple data with the total number of disks as the cycle period. The stripe distribution pattern of each data group is repeated within the cycle period, so that the spatial distribution is strictly aligned in the physical storage, and the spatial distribution of the independent redundant array of the storage system is optimized. Regardless of whether the number of disks is odd or even, a unified cycle model is used to facilitate continuous read and write operations of the disk array, and the cache structure is preferentially queried according to the data group number. After the cache hits, the data does not need to be calculated and directly extracted from the cache. If it does not hit, the block number is dynamically calculated and the calculation result is written to the corresponding position of the cache, effectively reducing the complex calculation steps for solving the block code. Through the cycle space distribution structure and cache acceleration mechanism, the problem of complex and time-consuming block number calculation method in the prior art is solved, and the performance and versatility of the storage system are significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A flowchart of a method for determining block coding of a storage system provided in an embodiment of the present application;

[0020] Figure 2 2 is a structural block diagram of a method and apparatus for determining block coding in a storage system according to an embodiment of the present invention;

[0021] Figure 3 FIG. 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In the prior art, as shown in Table 1, a RAID 6 array consists of six components, with each stripe being a stride. In this array, stripe numbers are shown from 0 to 5. Each stride shows a differently coded block number, and a corresponding algorithm is designed for this distribution.

[0023] Table 1

[0024]

[0025] The most common algorithm in existing technologies is address translation, phys to virt, which converts physical addresses to logical addresses. Given componentindex and stride, the algorithm determines the block encoding. The following describes the specific algorithm when the number of disks, componentcount, is an even number:

[0026] (1) First calculate the Rotation according to the strip number:

[0027] packOffset=StrideNumber%Stridesperpack

[0028] Rotation=(ComponentCount-2)-(packOffset 2)

[0029] if Rotation<0,Rotation=Rotation+ComponentCount

[0030] When the number of disks is even (and Rotation < 0), parpityOrder = R6_pARITY_qp (1)

[0031] (2) Secondly, calculate StripNum based on ComponentIndex and Rotation:

[0032] StripNum=ComponentIndex-Rotation (if StripNum<0, add ComponentCount)

[0033] Consider the following conditions: Rotation < 0, the number of disks is an even number

[0034] That is, parityOrder=R6_pARITY_qp

[0035] The check disk StripNum in the Stride changes from 1(0) to 0(1).

[0036] When the number of disks is odd, there's no need to worry about permuting the block encoding. Table 2 below shows an array with an odd number of disks. p rotates from the rightmost edge of the array to the left, rotating two disks at a time. This ensures that the bottom of the pack is aligned.

[0037] Table 2

[0038]

[0039] As can be seen, the existing technology requires calculating the block number every time a disk number and stripe number are given. It also needs to distinguish whether the number of disks is odd or even. If the number is even, it also needs to design a swap between the p and q blocks, which is a complex and time-consuming process. To address this problem, this application designs a new RAID6 spatial distribution model and a corresponding spatial distribution solution algorithm that can accelerate the solution of block codes.

[0040] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0041] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0042] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0043] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the block encoding determination method of the storage system depends, the specific application environment architecture or specific hardware architecture is described herein.

[0044] The present application provides a method for determining chunk encoding of a storage system. Figure 1 It is a flowchart of the method for determining chunk encoding of a storage system according to an embodiment of the present invention. As Figure 1 shown, the process includes the following steps:

[0045] Step S101: Divide the spatial distribution of the independent redundant array of the storage system into a cycle period of N data groups, where N is the total number of disks, each data group includes N consecutive stripes, and the stripe distribution patterns of all data groups are arranged repeatedly according to the cycle period N.

[0046] Specifically, the present application divides the spatial distribution of the independent redundant array of the storage system into N data groups (packs), and uses the total number of disks (N = drivecount) as the cycle period. All data groups are arranged cyclically with the total number of disks as the period, and the stripe distribution patterns of each data group within each period are exactly the same, so that the physical storage boundaries of each data group are strictly aligned, ensuring the continuous distribution of data on the disks.

[0047] In one embodiment, the stripes of each data group in the independent redundant array RAID6 all include: a first parity block (p parity block) and a second parity block (q parity block). The p parity blocks and q parity blocks in each data group are cyclically distributed according to a fixed offset, so that the spatial distribution of the disk array takes N data groups as a complete cycle period. It should be noted that the p parity is a horizontal parity based on exclusive OR (XOR) operation, and the q parity is a polynomial parity based on Galois Field.

[0048] In one embodiment, the stripe distribution of each data group follows the rule of circular left shift, where:

[0049] The first parity block in the (k + 1)-th data group is located at the disk position before the disk where the first parity block in the k-th data group is located, where 0 ≤ k < N; the second parity block in the (k + 1)-th data group is located at the disk position before the disk where the second parity block in the k-th data group is located;

[0050] When the parity block position moves to the first disk, the next position cycles to the last disk. As shown in Table 3, which takes 6 disks as an example for illustration:

[0051] Table 3

[0052]

[0053]

[0054] As can be seen in Table 3, the parity block positions of adjacent data groups are cyclically shifted left by a fixed offset of one disk position. When k = 6, the P / Q block positions are exactly the same as when k = 0, forming a closed loop of six data groups as a complete cycle. This cyclic distribution with a fixed offset enables RAID 6 to achieve a globally balanced distribution of parity blocks. This ensures that when a single disk fails, the parity data is evenly distributed across the other disks, improving reconstruction speed. The staggered distribution of the p and q parity blocks ensures a 100% data recovery success rate when any two disks fail, and only one cycle of the distribution pattern needs to be stored, reducing memory usage and achieving a solution that simultaneously improves performance and resource efficiency. This design does not distinguish between even and odd disks, and uniformly implements dynamic positioning through modular operations, which is the core embodiment of RAID 6's efficiency and versatility.

[0055] Step S102: creating a cache structure according to the spatial distribution of the redundant array of independent storage systems and initializing the structure to store the first stripe distribution of each data group.

[0056] Specifically, the present application creates a cache structure based on the spatial distribution of the independent redundant array of storage systems, including:

[0057] 1. Drivecount, which is used to record the number of disks in RAID 6.

[0058] 2. Cycle period packcyclenum, used to indicate the cycle period of data group distribution;

[0059] 3. The data group distribution array pack[n] is used to store the block number stripofpack[drivecount] of the first stripe of each data group on each disk.

[0060] In one embodiment, according to the number of disks drivecount=n of the RAID 6 type, a corresponding structure is created:

[0061] Struct RD6GEOM{

[0062] Int drivecount;

[0063] Int packcyclenum;

[0064] StructpACKGEOMpack[drivecount]

[0065] }

[0066] The structure struct pACKGEOM{

[0067] Int stripofpack[drivecount]

[0068] Int pack;

[0069] }

[0070] The member pack[drivecount] in the structure RD6GEOM stores the spatial distribution of the first strip of pack for the number of disks. pack[0] stores the spatial distribution of the first strip of the first pack, where packcyclenum represents the cycle period of the data group pack, and numerically it is equal to the number of disks drivecount = n. Regardless of whether the number of disks is odd or even, the spatial distribution of the disk array cycles every n, and n packs are aligned. From the first pack to the bottom of the Nth pack, they are aligned.

[0071] The cache structure of this application only stores the first strip distribution of each data group. Compared with the traditional method that needs to store the complete strip distribution of all data groups, it greatly reduces the memory occupancy. For example, in a RAID6 system with 10 disks, if each data group contains 10 strips, the traditional method needs to store 10×10×N (N is the number of data groups) block numbers; while this application only needs to store 10×10 basic data (the stripofpack array of pack[0]), and the rest of the data groups are dynamically deduced by formulas, significantly saving storage space in large-scale storage scenarios. [[ID=!13]] [[ID=!14]]

[0072] In this application, the fixed cycle period is equal to the total number of disks. Regardless of whether the number of disks is odd or even, the cache structure can use the same logic for data storage and query, avoiding the complex logic of designing the p-block and q-block swapping for even disks in the traditional scheme, reducing the difficulty of algorithm implementation and maintenance costs, and at the same time reducing the computational overhead caused by logical branch judgments.

[0073] The initialization method of the cache structure in this application is as follows: The strip numbers in the data group distribution array of the first data group are numbered in disk order, and the number range is not greater than the total number of disks; the rest of the data group distribution arrays are initialized to invalid values and are dynamically filled during operation.

[0074] In an embodiment, the cache structure variable rd6_cache_geom is initialized as type Struct RD6GEO, and stripofpack[i] = i (0 ≤ i < drivecount) of the data group distribution array pack[0] of the first data group is initialized; the stripofpack of the rest of the data groups is initialized to invalid values and is dynamically filled during operation.

[0075] This application utilizes this simple initialization method, avoiding complex precalculation of all data groups. This significantly reduces initialization time and computing resource consumption during system startup or RAID6 rebuilding, enabling rapid initialization and a usable state. The invalid value initialization strategy ensures the accuracy of cached data. Before data is calculated and populated, it is explicitly marked as invalid, preventing incorrect block numbering caused by misreading undetermined data. This ensures the correctness of data read, write, and verification and recovery operations within the RAID6 system, enhancing system stability and reliability.

[0076] This application enables the storage system to quickly respond to these high-frequency access requests through dynamic filling and cache hits, and for low-frequency access data groups, calculations are only performed when necessary, balancing the use of computing resources and storage resources, bringing efficient and stable performance to the RAID6 storage system.

[0077] Step S103, respond to the received disk number and stripe number, and calculate the corresponding data group number. If the cache structure is hit according to the data group number, the block number is directly returned. If it is not hit, the block number is dynamically calculated and the calculation result is written to the corresponding position of the cache.

[0078] Specifically, the RAID 6 system divides the continuous logical address space into multiple stripes, each of which is further divided into individual physical disks. Based on the disk address in the read and write instructions sent by the server, a logical operation is performed to obtain the disk number componentindex and the stripe number stridenum.

[0079] In one embodiment, LBA is the logical block address (Logical Block Address) sent by the server, stripe_size is the number of logical blocks contained in each stripe, and drive_count is the total number of disks in RAID6; then:

[0080] Stripe number (stridenum) = LBA / stripe_size;

[0081] Disk number (componentindex) = (LBA % stripe_size) / number of blocks per disk.

[0082] Assume that RAID6 consists of 6 disks and each stripe contains 64KB of data (divided into 16 logical blocks): If LBA=23, then the stripe number = 23 / 16 = 1, and the disk number = (23 % 16) / (16 / 6) = 7 / 2.67 ≈ 2 (mapped to disk 2). In this case, the server only needs to provide the logical disk address and does not need to understand the stripe distribution and parity block location within RAID6.

[0083] Furthermore, the data group number is calculated according to the following formula:

[0084] Data group number = strip number ÷ number of strips per data group

[0085] packnum = stridenum / / packstridenum,

[0086] Here, / / represents integer division, rounded down. packnum is the data group number, and stridenum is the stripe number. Packstridenum is the number of stripes per data group, which is equal to the total number of disks. The stripe number is used to calculate the data group (pack) to which it belongs. The periodic distribution of data groups is leveraged to narrow the query scope and improve computational efficiency.

[0087] In one example, for example, if stripe number stridenum = 3 and packstridenum = 6, then packnum = 3 / / 6 = 0, belonging to Pack 0, the data group includes stripe numbers 0~5.

[0088] In one example, stripe number stridenum = 6, packstridenum = 6, then packnum = 6 / / 6 = 1, belonging to Pack 1, the data group includes stripe numbers 6 to 11.

[0089] In one example, stripe number stridenum = 17, packstridenum = 6, then packnum = 17 / / 6 = 2 (integer division, 17 ÷ 6 = 2 remainder 5), then it belongs to Pack 2, and the data group includes stripe numbers 12 to 17.

[0090] This embodiment of the present application divides massive stripes into fixed-size data groups, facilitating batch management and rapid location. Integer division, a fast operation natively supported by the CPU, is used, making it suitable for real-time I / O scenarios (such as database reads and writes). The data group number is strongly correlated with the number of disks (packstridenum = drivecount), ensuring that the stripe distribution pattern is strictly repeated within each cycle, facilitating the caching mechanism to reuse calculation results. This calculation method divides linearly increasing stripe numbers into discrete data groups. Combined with the cyclic distribution rules of the parity blocks, it achieves the dual goals of "fast logical address mapping" and "physical storage load balancing." This not only simplifies the block number calculation logic but also provides a structural foundation for caching mechanisms and parallel access.

[0091] Furthermore, the present application responds to the received disk number and stripe number, calculates the corresponding data group number, queries the cache structure based on the data group number, and directly returns the block number if a hit occurs. If a hit does not occur, dynamically calculates the block number and writes the calculation result to the corresponding location in the cache. The process includes:

[0092] A1, query the cache structure based on the data group number to see if there is valid data; if there is valid data, directly return the block number.

[0093] When querying the block number, this application locates the target data group in the cache structure by taking the remainder of the total number of disks by the data group number, specifically by checking whether rd6_cache_geom.pack[packnum % drivecount]. stripofpack[componentindex] is valid storage data.

[0094] A2: If there is no valid data, the target block code is calculated based on the target data group number, the initial distribution of the first data group, and the disk number, and the calculation result is written to the corresponding location in the cache.

[0095] Specifically, the present application calculates the block number in the following manner: rd6_cache_geom.pack[packnum].stripofpack[componentindex]= (rd6_cache_geom.pack[0].stripofpack[componentindex]- packnum+drivecount)%drivecount.

[0096] Where, pack0.stripofpack[componentindex]: the initial block number of the target disk in the first data group (usually equal to the disk number, i.e. componentindex);

[0097] pack0.stripofpack[componentindex] - packnum: Starting from the initial number, subtract the number of the target data group. The larger the data group number, the larger the offset and the smaller the block number (circle to the left);

[0098] + drivecount: Ensures that the result within the brackets is non-negative to avoid errors in the remainder operation. For example, when packnum is large, the initial number - packnum may be negative, but it becomes positive after adding drivecount.

[0099] % drivecount: Limits the result to the range [0, drivecount-1] to achieve round-robin distribution. For example, if the result exceeds the total number of disks, it automatically returns to the first disk and renumbers the stripes. Using the remainder operation, the stripes are arranged in a round-robin pattern across the disks to ensure load balancing.

[0100] In one application scenario, let's take querying the block number of stripe number 18 as an example:

[0101] 1. Calculate the data group number (packnum):

[0102] Strip number stridenum=18, each data group contains 6 stripes (packstridenum=6);

[0103] packnum = 18 / / 6 = 3 (belongs to data group 3);

[0104] 2. Take the remainder to locate the cache index:

[0105] Total number of disks drivecount=6;

[0106] cache_idx = 3 % 6 = 3;

[0107] Target cache location: rd6_cache_geom.pack[3].

[0108] 3. Determine whether the cache is valid:

[0109] Assuming that the block number of disk 2 (componentindex=2) has been stored as valid data (for example, the value is 5) in the stripofpack array of pack[3], then 5 is returned directly;

[0110] If stripofpack[2] of pack[3] is an invalid value (such as initialized to -1), the dynamic calculation process is entered.

[0111] 1. Dynamically calculate the block number:

[0112] Initial number of disk 2 in the first data pack (pack0): pack0.stripofpack[2] = 2;

[0113] Formula: (2 - 3 + 6) % 6 = 5 % 6 = 5;

[0114] Calculation result: block number is 5.

[0115] 2. Update cache:

[0116] Set pack[3].stripofpack[2] to 5 and mark it as valid data.

[0117] 3. Subsequent query optimization:

[0118] When querying disk 2 of data group 3 again, the value 5 is directly obtained from pack[3].stripofpack[2] without recalculation.

[0119] This application directly reads the block number from the cache, avoiding repeated mathematical operations (such as modulo and subtraction), resulting in low single-query latency. The calculation is performed once the first time a data group is accessed, and the result is reused for subsequent accesses. For example, if the same data group is accessed 100 times, only the first calculation is required, improving overall efficiency by approximately 99%. Cache space is dependent only on the number of disks, not the total number of data groups. There's no need to pre-store the block numbers for all data groups; only the first data group can be used to calculate any position, saving storage space. For example, a RAID 6 system with 100 disks only needs to store the stripe distribution of 100 data groups, while traditional methods require storing all data groups, reducing space complexity from O(n×m) to O(n). The calculation (packnum % drivecount) allows limited cache space to cover an unlimited number of data groups, eliminating the need for complex cache eviction strategies. Combined with a validity verification mechanism, the accuracy of query results is ensured. This design not only significantly reduces computational overhead but also improves cache utilization through round-robin reuse, making it particularly suitable for scenarios where data access is localized.

[0120] Step S104: output the target block number for data reading, writing, verification and recovery.

[0121] Specifically, the target block encoding is output in the following way:

[0122] rd6_cache_geom.pack[packnum % drivecount].stripofpack[componentindex].

[0123] In one application scenario, for example, to query the block number of disk 3 (componentindex=3) in stripe number 23 (stridenum=23):

[0124] 1. Calculate the data group number: Each data group contains 6 stripes (packstridenum=6), so packnum= 23 ÷ 6 = 3 (data group 3).

[0125] 2. Query the cache structure: the cache index is packnum % 6 = 3 % 6 = 3, that is, query rd6_cache_geom.pack[3].

[0126] 3. Assuming that the cache misses during the first query (pack[3] is an invalid value), enter the dynamic calculation stage.

[0127] 4. Dynamically calculate the block number: The initial block number of disk 3 in the first data group (pack0) is 3 (pack0.stripofpack[3] = 3). Substituting into the formula: (3 - 3 + 6) % 6 = 6 % 6 = 0, the block number is 0.

[0128] 5. Update the cache and return the result: store the calculation result in

[0129] rd6_cache_geom.pack[3].stripofpack[3] = 0, marking it as valid data. When querying the same data set again later, the result is directly hit from the cache without recalculation.

[0130] The cache mechanism provided in this application significantly reduces the query delay of RAID6 block numbers through the strategy of "calculating once and reusing multiple times". It is particularly suitable for scenarios where data access is local. High-frequency access data groups always reside in the cache, avoiding fluctuations in request response time due to repeated calculations and reducing waste of system resources.

[0131] The embodiment of the present application uses a programming language to implement the algorithm and designs a new spatial distribution structure of an independent redundant array (RAID6) of storage systems. The spatial distribution rotates according to the number of disks as a cycle, and there is no need to consider the complex situation where the number of disks is odd or even. At the same time, the design combines the cache mechanism with the algorithm, and utilizes the fast and efficient characteristics of the cache to effectively reduce the tedious calculation steps of solving the block code, greatly accelerating the speed of solving the block logical address, thereby improving the performance and versatility of the device.

[0132] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0133] The embodiment of the present application also provides a device for determining block coding of a storage system, such as Figure 2 Shown, including:

[0134] A spatial distribution design module 201 is configured to divide the spatial distribution of the redundant array of independent disks of the storage system into N data group cycles, where N is the total number of disks, each data group includes N consecutive stripes, and the stripe distribution pattern of all data groups is repeated according to the cycle period N;

[0135] A cache structure initialization module 202 is configured to create and initialize a cache structure based on the spatial distribution of the redundant array of independent storage systems, and store the first stripe distribution of each data group;

[0136] The block code acquisition module 203 is used to respond to the received disk number and stripe number, calculate the corresponding data group number, query the cache structure based on the data group number, and directly return the block number if a hit is found. If a hit is found, the block number is dynamically calculated and the result is written to the corresponding location in the cache;

[0137] The block number output module 204 is used to output the target block number for data reading, writing, verification and recovery.

[0138] For the description of the features in the embodiment corresponding to the above-mentioned device for determining block codes of the storage system, reference can be made to the relevant description of the embodiment corresponding to the method for determining block codes of the storage system, which will not be repeated here.

[0139] The present application also provides a storage system that implements any of the above-described methods for determining block codes for a storage system to manage block codes. This system has beneficial effects in terms of computing efficiency, storage performance, system reliability, and scalability. Specifically:

[0140] 1. Block coding improves computational efficiency and eliminates traditional performance bottlenecks

[0141] The caching mechanism achieves "compute once, reuse multiple times." When a data set is first accessed, the block number is dynamically calculated and stored in the cache. Subsequent accesses directly hit the cache, reducing single query times from milliseconds to microseconds. For example, frequently accessed data sets can significantly reduce repeated computation overhead. Cache space is proportional only to the number of disks, eliminating the need to store all block mappings, significantly reducing memory usage compared to traditional methods. Striping is uniformly handled regardless of the number of disks, reducing code complexity and maintenance costs.

[0142] 2. Comprehensively improve storage performance to support high concurrency and large data volume scenarios

[0143] A cache hit directly returns the block number, reducing random I / O latency to sub-microseconds and improving sequential I / O throughput. For example, in a six-disk RAID 6 system, random read and write IOPS can be significantly improved compared to traditional methods. Physical storage strictly aligns data groups, reducing head seeks and improving performance in sequential read and write scenarios. Parity blocks are distributed cyclically with a fixed offset, evenly distributing the load across all disks and eliminating the write bottleneck of traditional parity disks. For example, write operations for dual parity blocks are distributed across different disks, improving bandwidth utilization during multi-threaded writes.

[0144] 3. Enhanced system reliability and more efficient data protection and recovery

[0145] Circularly distributed parity blocks support parallel data recovery, and dual-disk reconstruction time is shortened compared to traditional methods.

[0146] The regular distribution of parity blocks and data blocks ensures that even if any two disks fail, data can be restored using the remaining disks and parity information. Fault tolerance persists even with increasing disk count. A dynamic cache update mechanism ensures that block numbers are consistent with actual storage locations, preventing data read and write errors caused by cache expiration and improving system stability.

[0147] 4. Flexible expansion and low-cost adaptation to adapt to dynamic changes in storage scale

[0148] When the number of disks changes dynamically (e.g., when expanding), only the cache structure needs to be updated, without recalculating all block distributions. This allows the system to quickly respond to hardware changes. For example, when expanding from 8 to 16 disks, initialization time is reduced from hours to seconds. The block encoding calculation logic is independent of storage capacity, supporting petabyte-level data scaling while maintaining a constant cache footprint (dependent only on the number of disks).

[0149] An embodiment of the present application also provides a distributed storage system, in which multiple nodes collaboratively execute any one of the methods for determining block coding of the storage system to perform consistency management of data blocks and parallel data access.

[0150] Specifically, in a distributed storage system, each node maintains an independent cache structure (such as RD6GEOM), but a unified mathematical formula (such as stripofpack[i] = (pack0[i]- packnum + n) % n) is used to ensure that all nodes have consistent block encoding calculation results for the same logical address. The specific process is as follows:

[0151] 1. Address resolution phase: The client or metadata node sends the logical stripe number (stridenum) and disk number (componentindex) to the data node;

[0152] 2. Local calculation stage: Each data node independently calculates the data group number (packnum) and queries the local cache. If there is a miss, the block number is deduced using a formula;

[0153] 3. Result synchronization phase: Nodes synchronize frequently accessed block encodings through lightweight protocols (such as Gossip) to reduce repeated calculations.

[0154] The distributed storage system provided in this application can eliminate data deviations between nodes through consistency management. The block numbering only depends on global parameters such as the total number of disks and the data group number, avoiding access errors caused by inconsistent metadata in traditional distributed systems; even if the cache versions between nodes are temporarily inconsistent, they can be recalculated through a unified formula to achieve final consistency, thereby improving the system's fault tolerance.

[0155] The load balancing of parallel data access in this application can be divided into stripe-level parallelism and data group-level parallelism. Stripe-level parallelism means that multiple stripes within the same data group can be processed in parallel by different nodes. For example, in a 6-node distributed system, the 6 stripes of data group 0 can be accessed by 6 nodes simultaneously. Data group-level parallelism means that access requests for different data groups can be distributed to different node clusters, leveraging cache locality to improve concurrency. The cyclic distribution characteristics of block encoding ensure that data is evenly distributed across all nodes, avoiding hotspot nodes. Highly accessed block encoding is cached by multiple nodes, reducing cross-node communication overhead and lowering hardware and operation and maintenance costs.

[0156] The distributed storage system provided in this application breaks through the metadata management bottleneck of traditional distributed RAID6 by collaborating with a mathematical consistency model and lightweight cache, providing an efficient and reliable distributed storage foundation for cloud computing, edge computing, big data and other scenarios. It is especially suitable for applications that need to process massive amounts of data and have high scalability requirements.

[0157] The embodiment of the present application also provides an electronic device, such as Figure 3 As shown, it includes a memory 10 and a processor 20. The memory 10 stores a computer program. The processor 20 is configured to run the computer program to execute the steps in any of the above-mentioned embodiments of the method for determining block coding of a storage system.

[0158] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps of any of the above-mentioned embodiments of the method for determining block coding of a storage system when running.

[0159] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0160] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned embodiments of the method for determining block coding of a storage system are implemented.

[0161] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned embodiments of the method for determining block coding of a storage system are implemented.

[0162] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0163] The above is a detailed introduction to the block encoding method for a storage system provided by this application. Specific examples are used herein to illustrate the principles and implementation methods of this application. The description of the above embodiments is only intended to help understand the method and core ideas of this application. It should be pointed out that, for those skilled in the art, without departing from the principles of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

Claims

1. A method for determining block coding of a storage system, characterized in that: include: Divide the spatial distribution of the redundant array of independent disks in the storage system into a cycle of N data groups, where N is the total number of disks, each data group includes N consecutive stripes, and the stripe distribution pattern of all data groups is repeated in the cycle N; A cache structure is created and initialized based on the spatial distribution of the redundant array of independent storage systems to store the first stripe distribution of each data group. The cache structure includes: Total number of disks, used to record the number of disks in the independent redundant array; Cycle period, used to represent the cycle period of data group distribution; Data group distribution array, used to store the block number of the first stripe of each data group on each disk; Respond to the received disk number and stripe number and calculate the corresponding data group number. The data group number is calculated according to the following formula: data group number = stripe number / number of stripes contained in each data group. The cache structure is queried based on the data group number. If a hit is found, the block number is directly returned. If a miss is found, the block number is dynamically calculated and the result is written to the corresponding location in the cache. Output the target block number for data reading, writing, verification and recovery.

2. The method according to claim 1, characterized in that The stripe of each data group of the independent redundant array includes: a first check block and a second check block, and the stripe distribution pattern of all data groups is repeatedly arranged according to a cycle period N, including: The first check block and the second check block in each data group are cyclically distributed according to a fixed offset, so that the spatial distribution of the disk array has N data groups as a complete cycle.

3. The method according to claim 2, characterized in that The stripe distribution of each data group follows the cyclic left shift rule, where: The first parity block in the k+1th data group is located on the disk before the first parity block in the kth data group, where 0≤k <N; The second parity block in the k+1th data group is located at the disk position before the disk where the second parity block in the kth data group is located; When the parity block position moves to the first disk, the next position cycles to the last disk.

4. The method according to claim 1, wherein The cache structure is initialized as follows: Initialize the stripe numbers in the data group distribution array of the first data group in disk order, and the number range is no greater than the total number of disks; The remaining data group distribution arrays are initialized to invalid values and are filled dynamically at runtime.

5. The method according to claim 4, characterized in that When querying the cache structure based on the data group number, the target data group in the cache structure is located by taking the modulus of the total number of disks using the data group number.

6. The method according to claim 5, characterized in that The query cache structure according to the data group number directly returns the block number if a hit occurs, and dynamically calculates the block number if a hit does not occur and writes the calculation result to the corresponding location in the cache, including: Check the cache structure for valid data based on the data group number. If valid data exists, the block number is returned directly. If there is no valid data, the target block code is calculated based on the target data group number, the initial distribution of the first data group and the disk number, and the calculation result is written to the corresponding location of the cache.

7. The method according to claim 6, characterized in that The target block number is calculated based on the target data group number, the stripe distribution of the first data group, and the disk number using the following formula: The block number of the specified disk in the target data group = (the initial number of the disk in the first data group - the target data group number + the total number of disks) % the total number of disks.

8. The method according to claim 1, characterized in that The disk number and stripe number are obtained through logical operation based on the disk address in the read and write instructions sent by the server.

9. A storage system, characterized in that: Execute the block coding determination method of the storage system according to any one of claims 1 to 8 to perform block coding management.

10. A distributed storage system, characterized in that: Multiple nodes collaborate to execute the block coding determination method of the storage system described in any one of claims 1 to 8 to perform consistency management of data blocks and parallel data access.

11. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the method for determining block coding of a storage system according to any one of claims 1 to 8 when executing the computer program.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the block coding determination method of the storage system according to any one of claims 1 to 8.

13. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the block coding determination method of the storage system according to any one of claims 1 to 8.

Images