Block code determination method of storage system and storage system

Abstract

The invention discloses a block code determination method of a storage system and the storage system, and relates to the technical field of storage systems.The method comprises the steps that space distribution of a redundant array of the storage system is divided into a plurality of data sets with the total number of disks as the cycle period, and strip distribution modes of all the data sets in the cycle period are repeated; spatial distribution is strictly aligned on physical storage, spatial distribution optimization of the independent redundant array of the storage system is achieved, no matter whether the number of disks is an odd number or an even number, a uniform cycle period model is adopted, continuous read-write operation of the disk array is facilitated, a cache structure is queried according to a data group number, block numbers are returned if a cache is hit, and the cache structure is optimized. If not, the block number is dynamically calculated, and the calculation result is written into the corresponding position, so that the step of solving the block code is simplified. Through a cyclic period space distribution structure and a cache acceleration mechanism, the problem that block numbering calculation is complex and time-consuming in the prior art is solved, and the performance and universality of a storage system are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of storage systems, and particularly relates to a method for determining block coding of a storage system and a storage system. Background Art

[0002] A storage system is a complex theoretical model, and a series of independent and redundant disks are often used to form a RAID array. Currently, common RAID array models include: Raid 0, Raid1, Raid5, Raid6, etc. Among them, Raid 0 adopts a data striping distribution strategy, which evenly disperses and stores data in the disk array. However, due to the lack of a redundancy protection mechanism, once a single disk fails, all user data will be lost. This high-risk characteristic makes Raid 0 less widely used in practical applications. Raid 1 synchronously backs up user data to an additional disk through mirror storage, and the data on the two disks are mirror copies of each other. When the data on one disk is lost, the other disk can completely restore the data. In scenarios where the disk failure probability is relatively low, Raid 1 will cause a great waste of disk space, and the write performance will be significantly reduced because double disk write operations are required when writing data. Raid 5 has a redundancy of 1, and even if any disk in the disk array fails, the data of the failed disk can be restored by calculation based on the data of other disks. Based on this, Raid 6 has the advantage of a redundancy of 2, further improving data security, and thus is widely adopted in practical applications. Nowadays, the evaluation criteria for storage systems are no longer limited to security, and performance indicators have also become key considerations. Among them, IOPS (Input / Output Operations Per Second) is the core parameter for measuring the performance of a storage system. The shorter the time taken for a single IO operation, the higher the number of IO operations completed by the storage system per unit time, that is, the higher the IOPS.

[0003] RAID6 is an array composed of 4 - 16 disks, and there are two hot spare disks in this array. In the RAID 6 architecture, stride and strip are two important concepts. A stride is composed of multiple strips, and each strip has a corresponding number (stripnum). Among them, the strip with stripnum = 0 is called the p - strip, and the strip with stripnum = 1 is called the q - strip. The p and q strips are used as parity strips to store the parity information between data strips; the remaining strips are data strips, which are responsible for actual data storage. It should be noted that the size of the q parity strip is 1 / 16 larger than that of other strips. For example, when the data strip size is 256KB, the size of the q parity strip is 272KB, which makes alignment requirements for space distribution. However, in the prior art, when implementing space distribution alignment, whenever a disk number and a stride number are given, it is necessary to calculate the strip number, and at the same time, it is also necessary to distinguish the parity of the number of disks. If the number of disks is even, it is also necessary to perform the p - strip and q - strip swapping operation. The whole process is complex and time - consuming, affecting the operation efficiency of the storage system. Summary of the Invention This application provides a method for determining the strip encoding of a storage system and a storage system, so as to solve the defects that the original strip encoding calculation process of the independent redundant array of the storage system in the prior art is complex and time - consuming, resulting in low operation efficiency and poor performance of the storage system.

[0004] This application provides a method for determining the strip encoding of a storage system, including: Dividing the space distribution of the independent redundant array of the storage system into a cyclic period of N data groups, where N is the total number of disks, each data group includes N consecutive strides, and the stride distribution patterns of all data groups are arranged repeatedly according to the cyclic period N; Creating a cache structure according to the space distribution of the independent redundant array of the storage system and initializing it, and storing the first stride distribution of each data group; Obtaining the received disk number and stride number, and calculating the corresponding data group number. In response to the received disk number and stride number, calculating the corresponding data group number, querying the cache structure according to the data group number, if it hits, directly returning the strip number, if it does not hit, dynamically calculating the strip number and writing the calculation result to the corresponding position in the cache; Outputting the target strip number for data reading, writing or parity recovery.

[0005] This application also provides a storage system, which performs strip encoding management by executing the method for determining the strip encoding of the storage system described in any one of the above.

[0006] The present application also provides a distributed storage system, in which multiple nodes cooperate to execute the block coding determination method of the above-mentioned storage system, so as to perform consistency management of data blocks and parallel data access.

[0007] The present application also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of the block coding determination method of the above-mentioned storage system when executing the computer program.

[0008] The present application also provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to cause a computer to execute the steps of the block coding determination method of the above-mentioned storage system.

[0009] The present application also provides a computer program product, including computer instructions, and the computer instructions are used to cause a computer to execute the steps of the block coding determination method of the storage system in the first aspect or any corresponding embodiment thereof.

[0010] In the present application, the spatial distribution of the independent redundant array of the storage system is divided into multiple data with the total number of disks as the cycle period. The stripe distribution patterns of each data group within the cycle period are repeated, so that the spatial distribution is strictly aligned physically, realizing the optimization of the spatial distribution of the independent redundant array of the storage system. Whether the number of disks is odd or even, a unified cycle period model is adopted, which is convenient for the disk array to perform continuous read and write operations. And the cache structure is preferentially queried according to the data group number. After the cache hit, the data is directly extracted from the cache without calculation. If not hit, the block number is dynamically calculated and the calculation result is written into the corresponding position of the cache, effectively weakening the complex calculation steps for solving the block coding. Through the cycle period spatial distribution structure and the cache acceleration mechanism, the problems of complex and time-consuming block number calculation methods in the prior art are solved, and the performance and versatility of the storage system are significantly improved. Description of the Drawings

[0011] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0012] Figure 1 It is a schematic flowchart of a method for determining the block coding of a storage system provided by an embodiment of the present application; Figure 2 It is a structural block diagram of a device for determining the block coding of a storage system according to an embodiment of the present invention; Figure 3 It is a schematic diagram of the hardware structure of the electronic device according to an embodiment of the present invention. Detailed implementation manners

[0013] In the prior art, as shown in Table 1, RAID6 composed of 6 components, each horizontal bar is a stride. In this array, stripe numbers stride 0 to 5 can be seen. Different encoded block numbers can be seen in each stride, and corresponding algorithms are designed for this distribution.

[0014] Table 1

[0015] The most common algorithm in the prior art is address translation, phys To Virt, that is, converting the physical address to the logical address, that is, given component index and stride, to find the determined block encoding. The following introduces the specific algorithm when the number of disks component count is even: (1) First calculate Rotation according to the stripe number: packOffset = StrideNumber % Stridesperpack Rotation = (ComponentCount - 2) - (packOffset 2) if Rotation < 0, Rotation = Rotation + ComponentCount When the number of disks is even (and Rotation < 0), parityOrder = R6_pARITY_qp (1) (2) Secondly, calculate StripNum according to ComponentIndex and Rotation: StripNum = ComponentIndex - Rotation (if StripNum < 0, then add ComponentCount) Consider simultaneously satisfying the following: Rotation < 0, the number of disks is even That is, satisfying parityOrder = R6_pARITY_qp The parity disk StripNum in the Stride changes from 1(0) to 0(1), When the number of disks is odd, there is no need to consider the problem of block coding swapping. Table 2 below shows an array with an odd number of disks. p rotates from the rightmost to the leftmost of the array, rotating 2 disks each time. This aligns the bottom of the entire pack.

[0016] Table 2

[0017] It can be seen that in the prior art, whenever a disk number and a stripe number are given, the process of calculating a block number needs to be performed once, and it is also necessary to distinguish whether the number of disks is odd or even. If it is even, it is also necessary to design the p and q block swaps, and the process is both complex and time-consuming. To address this problem, the present application designs a new raid6 space distribution model and a corresponding space distribution solving algorithm, which can accelerate the solution of block coding.

[0018] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0019] It should be noted that in the description of the present application, the terms "including", "comprising" or any other variant thereof are intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0020] To enable those skilled in the art of the present technology to better understand the solution of the present application, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0021] Combined with the specific application environment architecture or specific hardware architecture on which the execution of the block coding determination method of the storage system depends, the specific application environment architecture or specific hardware architecture is described herein.

[0022] The present application provides a method for determining block coding of a storage system, Figure 1 which is a flowchart of the method for determining block coding of a storage system according to an embodiment of the present invention. As Figure 1 shown, the process includes the following steps: Step S101: Divide the space distribution of the independent redundant array of the storage system into a cycle period of N data groups, where N is the total number of disks. Each data group includes N consecutive stripes, and the stripe distribution patterns of all data groups are arranged repeatedly according to the cycle period N.

[0023] Specifically, in this application, the space distribution of the independent redundant array of the storage system is divided into N data groups (packs), and the total number of disks (N = drivecount) is used as the cycle period. All data groups are arranged in a cycle with the total number of disks as the period. The stripe distribution patterns of each data group within each period are exactly the same, so that the physical storage boundaries of each data group are strictly aligned, ensuring the continuous distribution of data on the disks.

[0024] In one embodiment, each stripe of each data group in the independent redundant array RAID6 includes: a first parity block (p parity block) and a second parity block (q parity block). The p parity blocks and q parity blocks in each data group are distributed cyclically according to a fixed offset, so that the space distribution of the disk array takes N data groups as a complete cycle period. It should be noted that the p parity is a horizontal parity based on the exclusive OR (XOR) operation, and the q parity is a polynomial parity based on the Galois Field.

[0025] In one embodiment, the stripe distribution of each data group follows the rule of circular left shift, where: The first parity block in the (k + 1)-th data group is located at the disk position before the disk where the first parity block in the k-th data group is located, where 0 ≤ k < N; the second parity block in the (k + 1)-th data group is located at the disk position before the disk where the second parity block in the k-th data group is located; When the parity block position moves to the first disk, the next position cycles to the last disk. As shown in Table 3, an example is given with 6 disks: Table 3

[0026] It can be seen from Table 3 that for adjacent data groups, the positions of the parity blocks are circularly shifted to the left by a fixed offset. The offset is 1 disk position. When k = 6, the positions of the P / Q blocks are exactly the same as when k = 0, forming a closed loop of 6 data groups as a complete cycle. Through the above circular distribution with a fixed offset, RAID6 achieves a global balanced distribution of parity blocks, which can ensure that when a single disk fails, the parity data is evenly distributed on other disks, improving the reconstruction speed. The staggered distribution of the p parity block and the q parity block enables the data recovery success rate to remain 100% when any two disks fail, and only one cycle of the distribution pattern needs to be stored, reducing the memory and realizing a scheme for synchronously improving performance and resource efficiency. This design does not need to distinguish between odd and even numbers of disks, and uniformly realizes dynamic positioning through modulo arithmetic, which is the core embodiment of the high efficiency and generality of RAID6.

[0027] Step S102, create a cache structure according to the spatial distribution of the independent redundant array of the storage system and initialize it, and store the first strip distribution of each data group.

[0028] Specifically, the present application creates a cache structure according to the spatial distribution of the independent redundant array of the storage system, including: 1. The total number of disks drivecount, which is used to record the number of disks in RAID6; 2. The cycle period packcyclenum, which is used to represent the cycle period of data group distribution; 3. The data group distribution array pack[n], which is used to store the block numbers stripofpack[drivecount] of the first strip of each data group on each disk.

[0029] In an embodiment, according to the number of disks drivecount = n of the created RAID6 type, the corresponding structure is created: Struct RD6GEOM{ Int drivecount; Int packcyclenum; StructpACKGEOMpack[drivecount] } Where the structure struct pACKGEOM{ Int stripofpack[drivecount] Int pack; } The member pack[drivecount] in the structure RD6GEOM stores the spatial distribution of the first stripe of drivecount number of packs. pack[0] stores the spatial distribution of the first stripe of the first pack, where packcyclenum represents the cycle period of the data group pack, and numerically it is equal to the number of disks drivecount = n. Regardless of whether the number of disks is odd or even, the spatial distribution of the disk array cycles every n, and n packs are aligned. From the first pack to the bottom of the Nth pack, they are aligned.

[0030] The cache structure of this application only stores the first stripe distribution of each data group. Compared with the traditional method that needs to store the complete stripe distribution of all data groups, it greatly reduces the memory occupation. For example, in a RAID6 system with 10 disks, if each data group contains 10 stripes, the traditional method needs to store 10×10×N (N is the number of data groups) block numbers; while this application only needs to store 10×10 basic data (the stripofpack array of pack [0]), and the rest of the data groups are dynamically deduced by formulas, significantly saving storage space in large-scale storage scenarios.

[0031] In this application, the fixed cycle period is equal to the total number of disks. Regardless of whether the number of disks is odd or even, the cache structure can use the same logic for data storage and query, avoiding the complex logic of designing the p-block and q-block swapping for even disks in the traditional scheme, reducing the algorithm implementation difficulty and maintenance cost, and at the same time reducing the computational overhead caused by logical branch judgments.

[0032] The initialization method of the cache structure in this application is as follows: The stripe numbers in the data group distribution array of the first data group are numbered in disk order during initialization, and the number range is no greater than the total number of disks; the rest of the data group distribution arrays are initialized to invalid values and are dynamically filled during runtime.

[0033] In one embodiment, the cache structure variable rd6_cache_geom is initialized as the type Struct RD6GEO, and stripofpack[i] = i (0 ≤ i < drivecount) of the data group distribution array pack[0] of the first data group is initialized; the stripofpack of the rest of the data groups is initialized to invalid values and is dynamically filled during runtime.

[0034] This application uses this simple initialization method to avoid complex pre-computations for all data groups, greatly reducing the initialization time and computational resource consumption when the system starts up or rebuilds RAID6. It can complete the initialization in a short time and quickly enter the available state. The invalid value initialization strategy ensures the accuracy of the cached data. Before the data is computed and filled, its state is clearly invalid, avoiding the chunk numbering error caused by misreading undetermined data, ensuring the correctness of the data read / write and parity recovery operations in the RAID6 system, and enhancing the stability and reliability of the system.

[0035] Through dynamic filling and cache hits, this application enables the storage system to quickly respond to these high-frequency access requests. For low-frequency access data groups, computations are only performed when necessary, balancing the use of computational resources and storage resources, and bringing efficient and stable performance to the RAID6 storage system.

[0036] Step S103: Respond to the received disk number and stripe number, calculate the corresponding data group number, query the cache structure according to the data group number. If a hit occurs, directly return the chunk number; if not, dynamically calculate the chunk number and write the calculation result to the corresponding position in the cache.

[0037] Specifically, the RAID6 system divides the continuous logical address space into multiple stripes, and each stripe is further subdivided onto each physical disk. The disk number componentindex and stripe number stridenum are obtained through logical operations on the disk address in the read / write instruction sent by the server side.

[0038] In an embodiment, LBA is the logical block address (Logical Block Address) sent by the server, stripe_size is the number of logical blocks included in each stripe, and drive_count is the total number of disks in RAID6. Then: Stripe number (stridenum) = LBA / stripe_size; Disk number (componentindex) = (LBA % stripe_size) / number of blocks per disk.

[0039] Assume that RAID6 consists of 6 disks, and each stripe contains 64KB of data (divided into 16 logical blocks). If LBA = 23, then the stripe number = 23 / 16 = 1, and the disk number = (23 % 16) / (16 / 6) = 7 / 2.67 ≈ 2 (mapped to disk 2). In this way, the server side only needs to provide the logical disk address without knowing the internal stripe distribution and parity block position of RAID6.

[0040] Further, the data group number is calculated according to the following formula: Data group number = stripe number ÷ number of stripes per data group packnum = stridenum / / packstridenum, where / / represents integer division, rounding down, packnum is the data group number, stridenum is the stripe number, and packstridenum is the number of stripes included in each data group, which is numerically equal to the total number of disks. Calculate the data group (pack) to which it belongs through the stripe number, and use the periodic distribution characteristics of the data group to narrow the query range and improve the operation efficiency.

[0041] In an example, for example, the stripe number stridenum = 3 and packstridenum = 6, then packnum = 3 / / 6 = 0, belonging to Pack 0, and this data group contains stripe numbers 0 to 5.

[0042] In an example, the stripe number stridenum = 6 and packstridenum = 6, then packnum = 6 / / 6 = 1, belonging to Pack 1, and this data group contains stripe numbers 6 to 11.

[0043] In an example, the stripe number stridenum = 17 and packstridenum = 6, then packnum = 17 / / 6 = 2 (integer division, 17 ÷ 6 = 2 remainder 5), then it belongs to Pack 2, and this data group contains stripe numbers 12 to 17.

[0044] The embodiments of this application divide a large number of stripes into data groups of a fixed size, which is convenient for batch management and quick positioning. Using integer division is a fast operation natively supported by the CPU and is suitable for I / O scenarios with high real-time requirements (such as database reading and writing). The data group number is strongly related to the number of disks (packstridenum = drivecount), ensuring that the stripe distribution pattern within each cycle is strictly repeated, which is convenient for the cache mechanism to reuse the calculation results. Through this calculation method, the linearly increasing stripe numbers are divided into discrete data groups, combined with the cyclic distribution rule of the check blocks, achieving the dual goals of "fast logical address mapping" and "physical storage load balancing". It not only simplifies the calculation logic of the block numbers but also provides a structured basis for the cache mechanism and parallel access.

[0045] Further, in response to the received disk number and stripe number, the present application calculates the corresponding data group number, queries the cache structure according to the data group number, if a hit occurs, directly returns the block number, and if a miss occurs, dynamically calculates the block number and writes the calculation result to the corresponding position in the cache. The process includes: A1. Query whether there is valid data in the cache structure according to the data group number; if there is valid data, directly return the block number.

[0046] When the present application queries the block number, it takes the remainder of the data group number divided by the total number of disks to locate the target data group in the cache structure. Specifically, it determines whether rd6_cache_geom.pack[packnum % drivecount]. stripofpack[componentindex] stores valid data.

[0047] A2. If there is no valid data, calculate the target block code based on the target data group number, the initial distribution of the first data group, and the disk number, and write the calculation result to the corresponding position in the cache.

[0048] Specifically, the present application calculates the block number in the following way: rd6_cache_geom.pack[packnum].stripofpack[componentindex]= (rd6_cache_geom.pack[0].stripofpack[componentindex]- packnum+drivecount)%drivecount.

[0049] Among them, pack0.stripofpack[componentindex]: the initial block number of the target disk in the first data group (usually equal to the disk number, that is, componentindex); pack0.stripofpack[componentindex] - packnum: starting from the initial number, subtract the number of the target data group. The larger the data group number, the larger the offset, and the smaller the block number (circular left); + drivecount: ensure that the result in the parentheses is non - negative to avoid errors in the modulo operation. For example, when packnum is large, the initial number - packnum may be negative, and it becomes positive after adding drivecount.

[0050] % drivecount: Limit the result within the range of [0, drivecount - 1] to achieve circular distribution. For example, when the result exceeds the total number of disks, it automatically returns to the first disk to be renumbered, and the strip is circularly arranged among the disks through modulo operation to ensure load balancing.

[0051] In an application scenario, take the example of querying the block number of strip number 18: 1. Calculate the data group number (packnum): Strip number stridenum = 18, and each data group contains 6 strips (packstridenum = 6); packnum = 18 / / 6 = 3 (belonging to data group 3); 2. Use modulo operation to locate the cache index: Total number of disks drivecount = 6; cache_idx = 3 % 6 = 3; Target cache location: rd6_cache_geom.pack[3].

[0052] 3. Determine whether the cache is valid: Assume that in the stripofpack array of pack[3], the block number of disk 2 (componentindex = 2) has been stored as valid data (for example, the value is 5), then directly return 5; If stripofpack[2] of pack[3] is an invalid value (such as initialized to -1), then enter the dynamic calculation process.

[0053] 1. Dynamically calculate the block number: Initial number of disk 2 in the first data group (pack0): pack0.stripofpack[2] = 2; Formula: (2 - 3 + 6) % 6 = 5 % 6 = 5; Calculation result: The block number is 5.

[0054] 2. Update the cache: Set pack[3].stripofpack[2] to 5 and mark it as valid data.

[0055] 3. Subsequent query optimization: When querying disk 2 of data group 3 again, directly obtain the value 5 from pack[3].stripofpack[2] without recalculation.

[0056] This application directly reads the chunk number from the cache, avoiding repeated execution of mathematical operations (such as modulo operation, subtraction), and the time consumption of a single query is low. It is calculated once when a certain data group is first accessed, and the result is directly reused for subsequent accesses. For example, if the same data group is accessed 100 times, only the first calculation is required, and the overall efficiency is increased by about 99%. The cache space is only related to the number of disks and has nothing to do with the total number of data groups. There is no need to pre-store the chunk numbers of all data groups. Any position can be calculated only relying on the first data group, saving storage space. For example, a RAID6 system with 100 disks only needs to store the stripe distribution of 100 data groups, while the traditional method needs to store all data groups, and the space complexity is reduced from O(n×m) to O(n). The coverage of the infinite data groups by the finite cache space is realized through the operation (packnum % drivecount), and there is no need for a complex cache replacement strategy. Combining with the validity verification mechanism, the accuracy of the query result is ensured. This design not only greatly reduces the calculation overhead, but also improves the cache utilization rate through cyclic reuse, especially suitable for scenarios where data access has locality.

[0057] Step S104, output the target chunk number for data reading, writing, or verification and recovery.

[0058] Specifically, the target chunk code is output in the following way: rd6_cache_geom.pack[packnum % drivecount].stripofpack[componentindex].

[0059] In an application scenario, for example: query the chunk number of disk 3 (componentindex = 3) at stripe number 23 (stridenum = 23): 1. Calculate the data group number: Each data group contains 6 stripes (packstridenum = 6), so packnum = 23 ÷ 6 = 3 (data group 3).

[0060] 2. Query the cache structure: The cache index is packnum % 6 = 3 % 6 = 3, that is, query rd6_cache_geom.pack[3].

[0061] 3. Assume that the cache is not hit during the first query (pack[3] is an invalid value), and enter the dynamic calculation stage.

[0062] 4. Dynamically calculate the block number: The initial block number of disk 3 in the first data group (pack0) is 3 (pack0.stripofpack[3] = 3). Substitute it into the formula: (3 - 3 + 6) % 6 = 6 % 6 = 0, so the block number is 0.

[0063] 5. Update the cache and return the result: Store the calculation result into rd6_cache_geom.pack[3].stripofpack[3]= 0, and mark it as valid data. When querying the same data group again later, directly hit the result from the cache without recalculation.

[0064] The cache mechanism provided by this application significantly reduces the query latency of RAID6 block numbers through the strategy of "calculate once, reuse multiple times". It is especially suitable for scenarios where data access has locality. Frequently accessed data groups always reside in the cache, avoiding fluctuations in request response time caused by repeated calculations and reducing system resource waste.

[0065] The embodiments of this application use a programming language to implement the algorithm, design a new spatial distribution structure of the independent redundant array of disks (RAID6) in the storage system. The spatial distribution rotates in a cycle according to the number of disks, without considering the complex situation of whether the number of disks is odd or even. At the same time, the cache mechanism is combined with the algorithm, and the fast and efficient characteristics of the cache are used to effectively weaken the cumbersome calculation steps for solving the block encoding, and well accelerate the solving speed of the block logical address, thereby improving the device performance and versatility.

[0066] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method.

[0067] The embodiments of this application also provide a device for determining the block encoding of a storage system, as Figure 2 shown, including: A spatial distribution design module 201, configured to use a cycle of N divided data groups for the spatial distribution of the independent redundant array of disks in the storage system, where N is the total number of disks, each data group includes N consecutive stripes, and the stripe distribution patterns of all data groups are arranged repeatedly according to the cycle N; A cache structure initialization module 202, configured to create and initialize a cache structure according to the spatial distribution of the independent redundant array of disks in the storage system, and store the first stripe distribution of each data group; The block coding acquisition module 203 is configured to respond to the received disk number and stripe number, calculate the corresponding data group number, query the cache structure according to the data group number, directly return the block number if a hit occurs, and dynamically calculate the block number and write the calculation result to the corresponding position in the cache if a miss occurs; The block number output module 204 is configured to output the target block number for data reading, writing, or verification and recovery.

[0068] For the descriptions of the features in the corresponding embodiments of the block coding determination device of the above storage system, reference can be made to the relevant descriptions of the corresponding embodiments of the block coding determination method of the storage system, which will not be elaborated here one by one.

[0069] The embodiment of the present application further provides a storage system that performs block coding management by executing any one of the above-mentioned block coding determination methods of the storage system. It has beneficial effects in terms of computing efficiency, storage performance, system reliability, and expansion ability. Specifically: 1. The block coding calculation efficiency is improved, eliminating the traditional performance bottleneck The cache mechanism realizes "one calculation, multiple reuses". When accessing a data group for the first time, the block number is dynamically calculated and stored in the cache. Subsequent accesses directly hit the cache, and the single query time-consuming is reduced from the millisecond level to the microsecond level. For example, the data groups with high-frequency access can greatly reduce the repetitive calculation overhead. The cache space is only proportional to the number of disks, without the need to store all block mappings, and the memory occupancy is greatly reduced compared with the traditional method. Whether the number of disks is odd or even, the stripe distribution is uniformly processed, reducing the code complexity and maintenance cost.

[0070] 2. The storage performance is comprehensively improved, supporting high-concurrency and large-data-volume scenarios When the cache is hit, the block number is directly returned, the random I / O latency is reduced to sub-microseconds, and the sequential I / O throughput is increased. For example, in a RAID6 system with 6 disks, the random read and write IOPS can be greatly improved compared with the traditional method. The physical storage strictly aligns the data groups, reducing the number of head seek operations and improving the performance in continuous read and write scenarios. The parity blocks are cyclically distributed at a fixed offset, and the load is evenly distributed to all disks, eliminating the write bottleneck of the traditional parity disk. For example, the write operations of dual parity blocks are dispersed to different disks, and the bandwidth utilization rate is increased during multi-threaded writing.

[0071] 3. The system reliability is enhanced, and data protection and recovery are more efficient The cyclically distributed parity blocks support parallel data recovery, and the dual-disk reconstruction time is shortened compared with the traditional method.

[0072] The distribution rule of the parity block and the data block ensures that data can be recovered from the remaining disks and the parity information when any two disks fail, and the fault tolerance does not decrease as the number of disks increases. The dynamic update cache mechanism ensures that the block number is consistent with the actual storage location, avoiding data read and write errors caused by cache expiration and improving the system stability.

[0073] 4. Flexible expansion and low-cost adaptation to adapt to dynamic changes in storage scale When the number of disks changes dynamically (such as expansion), only the cache structure needs to be updated, and there is no need to recalculate the distribution of all blocks. The system can quickly respond to hardware changes. For example, when expanding from 8 disks to 16 disks, the initialization time is shortened from the hour level to the second level. The block coding calculation logic is independent of the storage capacity, supports the expansion of PB-level data scale, and the cache space occupancy is constant (only related to the number of disks).

[0074] The embodiment of the present application also provides a distributed storage system, in which multiple nodes cooperate to execute any of the block coding determination methods of the storage system to perform consistency management of data blocks and parallel data access.

[0075] Specifically, in the distributed storage system, each node maintains an independent cache structure (such as RD6GEOM), but ensures that the block coding calculation results of all nodes for the same logical address are consistent through a unified mathematical formula (such as stripofpack[i] = (pack0[i]- packnum + n) % n). The specific process is as follows: 1. Address resolution stage: The client or the metadata node sends the logical stripe number (stridenum) and the disk number (componentindex) to the data node; 2. Local calculation stage: Each data node independently calculates the data group number (packnum) and queries the local cache. If not hit, the block number is deduced through the formula; 3. Result synchronization stage: The nodes synchronize the block coding of frequently accessed data through a lightweight protocol (such as Gossip) to reduce duplicate calculations.

[0076] The distributed storage system provided by the present application can eliminate data deviation between nodes through consistency management. The block number only depends on global parameters such as the total number of disks and the data group number, avoiding access errors caused by metadata inconsistency in traditional distributed systems; even if the cache versions between nodes are briefly inconsistent, the final consistency can be achieved through recalculation using a unified formula, improving the system fault tolerance.

[0077] The load balancing of parallel data access in this application can be divided into strip-level parallelism and data group-level parallelism. Among them, strip-level parallelism means that multiple strips within the same data group can be processed in parallel by different nodes. For example, in a 6-node distributed system, 6 strips of data group 0 can be accessed by 6 nodes simultaneously; data group-level parallelism means that access requests for different data groups can be assigned to different node clusters to improve concurrency using cache locality. The cyclic distribution characteristic of block coding evenly distributes data across all nodes, avoiding hot nodes. Frequently accessed block coding is cached by multiple nodes, reducing cross-node communication overhead and lowering hardware and operation and maintenance costs.

[0078] The distributed storage system provided by this application collaborates with a mathematical consistency model and lightweight caching, breaking through the metadata management bottleneck of traditional distributed RAID6, providing an efficient and reliable distributed storage foundation for scenarios such as cloud computing, edge computing, and big data, and is particularly suitable for applications that need to process massive data and have high scalability requirements.

[0079] An embodiment of this application also provides an electronic device, such as Figure 3 shown, including a memory 10 and a processor 20. A computer program is stored in the memory 10, and the processor 20 is configured to run the computer program to execute the steps in any of the above embodiments of the method for determining block coding of the storage system.

[0080] An embodiment of this application also provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps in any of the above embodiments of the method for determining block coding of the storage system when running.

[0081] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: USB flash drives, read-only memory (ROM for short), random access memory (RAM for short), external hard drives, magnetic disks, or optical discs and other media that can store computer programs.

[0082] An embodiment of this application also provides a computer program product. The above computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above embodiments of the method for determining block coding of the storage system.

[0083] An embodiment of this application also provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above embodiments of the method for determining block coding of the storage system.

[0084] Those skilled in the art may further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0085] The above has introduced in detail a block coding method for a storage system provided by this application. Specific examples are used herein to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of this application, several improvements and modifications can still be made to this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for determining block coding of a storage system, characterized in that, Including: The space distribution of the independent redundant array of the storage system is cycled in N divided data groups, where N is the total number of disks. Each data group includes N consecutive stripes, and the stripe distribution patterns of all data groups are arranged repeatedly according to the cycle period N; Create a cache structure according to the space distribution of the independent redundant array of the storage system and initialize it, and store the first stripe distribution of each data group; Respond to the received disk number and stripe number, calculate the corresponding data group number, query the cache structure according to the data group number. If it hits, directly return the block number. If it does not hit, dynamically calculate the block number and write the calculation result to the corresponding position in the cache; Output the target block number for data reading, writing, or verification and recovery.

2. The method according to claim 1, characterized in that The stripes of each data group of the independent redundant array all include: a first parity block and a second parity block. The stripe distribution patterns of all data groups are arranged repeatedly according to the cycle period N, including: The first parity block and the second parity block in each data group are cyclically distributed according to a fixed offset, so that the space distribution of the disk array takes N data groups as a complete cycle period.

3. The method according to claim 2, characterized in that, The stripe distribution of each data group follows the cyclic left shift rule, where: The first parity block in the (k + 1)-th data group is located at the previous disk position of the disk where the first parity block in the k-th data group is located, where 0 ≤ k < N; The second parity block in the (k + 1)-th data group is located at the previous disk position of the disk where the second parity block in the k-th data group is located; When the parity block position moves to the first disk, the next position cycles to the last disk.

4. The method according to claim 1, wherein The cache structure includes: The total number of disks, which is used to record the number of disks in the independent redundant array; The cycle period, which is used to represent the cycle period of data group distribution; The data group distribution array, which is used to store the block numbers of the first stripe of each data group on each disk.

5. The method according to claim 4, characterized in that, The data group number is calculated according to the following formula: Data group number = stripe number / number of stripes included in each data group.

6. The method according to claim 4 or 5, characterized in that, The initialization method of the cache structure is: Initialize the stripe numbers in the data group distribution array of the first data group to be numbered in disk order, and the number range does not exceed the total number of disks; The remaining data group distribution arrays are initialized to invalid values and are dynamically filled during operation.

7. The method according to claim 6, wherein When querying the cache structure according to the data group number, take the remainder of the data group number by the total number of disks to locate the target data group in the cache structure.

8. The method according to claim 7, wherein When querying the cache structure according to the data group number and hitting, directly return the block number. If it does not hit, dynamically calculate the block number and write the calculation result to the corresponding position in the cache, including: Query whether there is valid data in the cache structure according to the data group number. If there is valid data, directly return the block number; If there is no valid data, calculate the target block code based on the target data group number, the initial distribution of the first data group, and the disk number, and write the calculation result to the corresponding position in the cache.

9. The method according to claim 8, wherein Calculate the target block code through the following formula based on the target data group number, the stripe distribution of the first data group, and the disk number: The block number of the specified disk in the target data group = (the initial number of this disk in the first data group - the target data group number + the total number of disks) % the total number of disks.

10. The method according to claim 1, wherein The disk number and stripe number are obtained through logical operations based on the disk address in the read / write instruction sent by the server side.

11. A storage system, characterized in that, Execute the block coding determination method of the storage system according to any one of claims 1 to 10 for block coding management.

12. A distributed storage system, characterized in that, Multiple nodes cooperate to execute the block coding determination method of the storage system according to any one of claims 1 to 10 for consistency management of data blocks and parallel data access.

13. An electronic device, characterized in that, Comprising: A memory for storing a computer program; A processor for implementing the block coding determination method of the storage system according to any one of claims 1 to 10 when executing the computer program.

14. A computer-readable storage medium, characterized in that, Computer instructions are stored on the computer-readable storage medium, and the computer instructions are used to cause a computer to execute the block coding determination method of the storage system according to any one of claims 1 to 10.

15. A computer program product, characterized in that, Comprising computer instructions for causing a computer to execute the block coding determination method of the storage system according to any one of claims 1 to 10.

Images