A storage array data dynamic storage method, device, medium and product

Abstract

The application discloses a kind of storage array data dynamic depositing method, equipment, medium and product, it is related to data storage technical field, comprising: target compression ratio is compressed to the data to be written, based on historical writing data dynamically updating compression ratio maximum, minimum and sliding average, according to the current compression rate value range is divided into interval and is distributed unique mapping mark, determine target mapping mark corresponding to the interval where target compression ratio is located, determine target recyclable unit handle and recycling group based on the corresponding relationship between mapping mark and deposit location mark, write the data to be written and new check data with corresponding mark into storage device.It makes the data with similar compression characteristics stored in the same physical area, reduces garbage collection data migration, and suppresses write amplification effect.

Description

Technical Field

[0001] This invention relates to the field of data storage technology, and in particular to a method, device, medium, and product for dynamically storing data in a storage array. Background Technology

[0002] Flexible data placement is a feature defined in the NVMe protocol specification (Non-Volatile Memory Express), allowing the host to participate in the management of the physical layout of data within the SSD. Its mechanisms include: reclaimable cell handles, representing a reusable storage area within the SSD; and reclaim groups, used to associate data with similar lifecycles, with the SSD prioritizing the reclamation of space within the same reclaim group during garbage collection. By classifying and storing data by type, the effective data migration during garbage collection can be reduced, mitigating write amplification.

[0003] In scenarios where a storage array controller is introduced between the host and the solid-state drives (SSDs) for redundancy protection, the striping write strategy of the storage array controller conflicts with the optimization goal of flexible data placement. Striping writes divide host data and distribute it across multiple SSDs. This process is unaware of the differences in data content characteristics, resulting in data with similar lifecycles being stored separately and data with different lifecycles being stored together. This disrupts the physical continuity upon which flexible data placement depends, requiring SSDs to still migrate a large amount of valid data during garbage collection, and the write amplification effect cannot be effectively suppressed.

[0004] To address this issue, one approach is to establish a static mapping within the storage array controller, which permanently binds data types to reclaimable cell handles and reclaimable groups. This approach does not rely on hardware modifications, but the mapping relationship is fixed after initialization. When the compression characteristics of written data change with the workload, the mapping relationship cannot be adjusted, and the effectiveness of data classification and storage deteriorates over time.

[0005] In summary, how to achieve dynamic adaptive adjustment of data storage location in the context of storage array controllers to effectively suppress write amplification is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a method, device, medium, and product for dynamically storing data in a storage array, which can solve the problems of static mapping being unable to adapt to load changes and poor write amplification effect suppression.

[0007] To address the aforementioned technical problems, embodiments of the present invention provide a method for dynamically storing data in a storage array, applied to a storage array controller, comprising: Receive the data to be written from the host and cache the data to be written in the local data cache; Perform compression processing on the data to be written to obtain the target compression ratio of the data to be written; The maximum and minimum compression ratios are continuously updated using historical write data. The moving average of the compression ratio is also updated based on the target compression ratio and the size of the data to be written, resulting in an updated moving average of the compression ratio. The current compression ratio range is then divided into intervals based on the updated maximum, minimum, and moving average of the compression ratio, and a unique mapping identifier is assigned to each current interval. Historical write data refers to data that precedes the data to be written in chronological order. The unique mapping identifier corresponding to the current interval where the target compression ratio is located is determined as the target mapping identifier; The target recyclable unit handle and target recycling group corresponding to the target mapping identifier are determined based on the correspondence between the preset mapping identifier and the storage location identifier; wherein, the storage location identifier includes the recyclable unit handle and the recycling group; Read the original data and original check data corresponding to the data to be written, and perform array check calculation based on the data to be written, the original data and the original check data to generate new check data; Determine the handle to the reclaimable cell and the reclaim group corresponding to the new verification data; Write the data to be written, carrying the target reclaimable cell handle and the target reclaim group, to the storage device, and write the new verification data, carrying the corresponding reclaimable cell handle and reclaim group, to the storage device.

[0008] Optionally, based on the updated maximum compression ratio, the updated minimum compression ratio, and the updated moving average compression ratio, the current compression ratio range is divided into intervals, and a unique mapping identifier is assigned to each current interval, including: The minimum value of the updated compression ratio is used as the lower bound of the interval, and the maximum value of the updated compression ratio is used as the upper bound of the interval to determine the current compression ratio range. The current compression ratio range is divided into multiple intervals using the updated compression ratio moving average and the preset interval division strategy, and a unique mapping identifier is assigned to each current interval.

[0009] Optionally, the preset interval division strategy is the average division strategy; Accordingly, the current compression ratio range is divided into multiple intervals using a preset interval division strategy, and a unique mapping identifier is assigned to each current interval, including: The current compression ratio range is evenly divided into multiple current intervals of equal length, and a unique mapping identifier is assigned to each current interval.

[0010] Optionally, the preset interval division strategy is a mean-center two-sided segmentation strategy; Accordingly, the current compression ratio range is divided into multiple intervals using the updated compression ratio moving average and a preset interval division strategy, and a unique mapping identifier is assigned to each current interval, including: Using the updated compression ratio moving average as the boundary, the current compression ratio range is divided into a preset high compression ratio region and a preset low compression ratio region; The preset high compression rate region is evenly divided into multiple intervals, and the preset low compression rate region is evenly divided into multiple intervals, and a unique mapping identifier is assigned to each current interval.

[0011] Optionally, the preset interval division strategy is a mean center density decreasing segmentation strategy; Accordingly, the current compression ratio range is divided into multiple intervals using the updated compression ratio moving average and a preset interval division strategy, and a unique mapping identifier is assigned to each current interval, including: Centered on the updated compression ratio moving average, the current compression ratio range is divided into multiple intervals, and a unique mapping identifier is assigned to each current interval; the length of each current interval increases as the distance from the updated compression ratio moving average increases.

[0012] Optionally, the moving average of the compression ratio is updated based on the target compression ratio and the size of the data to be written, resulting in an updated moving average of the compression ratio, including: The moving average of the previous compression ratio is updated using the target compression ratio, the size of the data to be written, the moving average of the previous compression ratio, and the preset moving average statistical window size to obtain the updated moving average of the compression ratio; wherein, the preset moving average statistical window size is the preset number of statistical bytes.

[0013] Optionally, before determining the target recyclable unit handle and target recycling group corresponding to the target mapping identifier based on the correspondence between the preset mapping identifier and the storage location identifier, the method further includes: Construct a matrix with recyclable unit handles as rows and recyclable groups as columns, and use the matrix as a storage location identifier; Each unique mapping identifier is mapped to a storage location identifier in a preset order to obtain the correspondence between the preset mapping identifier and the storage location identifier.

[0014] Optionally, determine the reclaimable cell handle and reclaim group corresponding to the new verification data, including: The target reclaimable cell handle and target reclaimable group are determined as the reclaimable cell handle and reclaimable group corresponding to the new verification data.

[0015] Optionally, determine the reclaimable cell handle and reclaim group corresponding to the new verification data, including: Check whether the write operation type corresponding to the data to be written is random write or sequential write; When the write operation type is random write, the first reclaimable unit handle and the first reclaimable group are determined as the reclaimable unit handle and reclaimable group corresponding to the new verification data; When the write operation type is sequential write, the second reclaimable unit handle and the second reclaimable group are determined as the reclaimable unit handle and reclaimable group corresponding to the new verification data.

[0016] Optionally, determine the reclaimable cell handle and reclaim group corresponding to the new verification data, including: Retrieve the handle of the reclaimable unit and the reclaimable group corresponding to the new verification data from the preset storage location identifier library.

[0017] Optionally, the storage array controller is a standalone disk redundant array controller.

[0018] Optionally, the method for dynamically storing data in the storage array also includes: A first processing thread and a second processing thread are constructed and started in parallel. The first processing thread is used to perform compression processing on the data to be written to obtain the target compression ratio of the data to be written. The second processing thread is used to read the original data and original verification data corresponding to the data to be written.

[0019] In a second aspect, the present invention discloses an electronic device, comprising: Memory, used to store computer programs; A processor for executing a computer program to implement any of the aforementioned methods for dynamically storing data in a storage array.

[0020] Thirdly, the present invention discloses a computer-readable storage medium on which a computer program is stored, wherein when the computer program is executed by a processor, it implements the steps of any of the aforementioned methods for dynamically storing data in a storage array.

[0021] Fourthly, the present invention discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of any of the aforementioned methods for dynamically storing data in a storage array.

[0022] This invention provides a method for receiving data to be written from a host and caching the data in a local data cache; performing compression processing on the data to be written to obtain a target compression ratio; continuously updating the maximum and minimum compression ratios using historical write data, and updating the moving average of the compression ratio based on the target compression ratio and the size of the data to be written to obtain an updated moving average of the compression ratio; dividing the current compression ratio range into intervals based on the updated maximum, minimum, and moving average of the compression ratio, and assigning a unique mapping identifier to each current interval; wherein, historical write data refers to data that precedes the data to be written in chronological order; and setting the target compression ratio at the specified location... The unique mapping identifier corresponding to the current interval is determined as the target mapping identifier; based on the correspondence between the preset mapping identifier and the storage location identifier, the target recyclable cell handle and target recyclable group corresponding to the target mapping identifier are determined; wherein, the storage location identifier includes the recyclable cell handle and the recyclable group; the original data and original verification data corresponding to the data to be written are read, and array verification calculation is performed based on the data to be written, the original data, and the original verification data to generate new verification data; the recyclable cell handle and recyclable group corresponding to the new verification data are determined; the data to be written carrying the target recyclable cell handle and the target recyclable group are written to the storage device, and the new verification data carrying the corresponding recyclable cell handle and recyclable group are written to the storage device.

[0023] As can be seen from the above technical solution, by compressing the data to be written in real time and classifying it using the compression ratio as a data feature, the storage array controller first compresses the data to be written to obtain the target compression ratio during each write operation. Simultaneously, it continuously maintains the maximum, minimum, and moving average values ​​of the compression ratio based on the data already written. Then, based on these three dynamically updated statistical values, the compression ratio range is divided into intervals. Each interval corresponds to a storage location identifier combination, namely a reclaimable unit handle and a reclaimable group. Data is written to the physical storage area pointed to by the reclaimable unit handle and reclaimable group corresponding to the target compression ratio within that interval. In this way, data with similar compression characteristics are grouped and stored in the same physical location, solving the problem of data dispersion caused by striped writing not being aware of data characteristics. Furthermore, the statistical values ​​upon which the interval division depends are continuously updated during the write process, and the interval boundaries are dynamically adjusted, solving the problem that static mapping cannot adapt to load changes. When the solid-state drive (SSD) performs garbage collection, the data lifecycles within the same reclaimable group are similar, the effective data migration amount is reduced, the write amplification effect is suppressed, and the lifespan of the SSD is extended. Attached Figure Description

[0024] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart of a method for dynamically storing data in a storage array, provided in an embodiment of the present invention; Figure 2 A data flow diagram of write processing for a RAID5 storage array provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a uniform, equal-length mapping segmentation interval provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of interval division with bilateral segmentation based on the mean center, provided in an embodiment of the present invention. Figure 5 This is a schematic diagram of interval division with decreasing mean center density provided in an embodiment of the present invention; Figure 6 A functional block diagram of compression rate to mapping identifier provided in an embodiment of the present invention; Figure 7 A diagram illustrating several mapping methods from mapping identifiers to a (RUH, RG) matrix, provided by an embodiment of the present invention; Figure 8 A control flowchart for a write operation of a RAID controller is provided in an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0027] The terms "comprising" and "having," and any variations thereof, in the specification and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may include steps or units not listed.

[0028] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] A storage array controller is an intermediate layer device deployed between the host and storage devices, responsible for redundancy protection of written data. Taking a Redundant Array of Independent Disks (RAID) controller as an example, after receiving a write command from the host, it divides the data into stripes according to the configured RAID level, calculates parity data, and distributes the data blocks and parity blocks to multiple solid-state drives. In scenarios with redundant power supplies, the RAID controller can be configured in write-back mode. After the written data has been moved from the host to the local data cache, it can return a write completion acknowledgment to the host, resulting in lower write latency on the host side.

[0030] Flexible data placement is a feature defined in the NVMe protocol specification, and its mechanism includes reclaimable cell handles and reclaim groups. A reclaimable cell handle represents a reusable storage area within the SSD, such as a flash block or superblock; reclaim groups are used to associate data with similar lifecycles or characteristics, allowing the SSD to prioritize reclaiming space within the same reclaim group during garbage collection. Categorizing and storing boot data by type reduces the amount of valid data migrated during garbage collection, mitigating write amplification.

[0031] However, when the storage array controller performs striped writes, it divides the host data and distributes it across multiple solid-state drives. This process is unaware of the differences in data content characteristics, resulting in data with similar lifecycles being stored separately, while data with different lifecycles may be mixed together, disrupting the physical continuity upon which flexible data placement depends. Existing solutions establish static mappings within the storage array controller, permanently binding data types to reclaimable cell handles and reclaimable groups. However, these mappings are fixed after initialization and cannot be adjusted with load changes, causing the effectiveness of data classification and storage to deteriorate over time.

[0032] To address this, the present invention provides a dynamic data storage scheme for storage arrays, which achieves dynamic adaptive adjustment of data storage location by introducing a compression rate sensing mechanism inside the storage array controller.

[0033] like Figure 1 As shown, the present invention provides a method for dynamically storing data in a storage array, applied to a storage array controller, comprising: Step S11: Receive the data to be written from the host and cache the data to be written in the local data cache.

[0034] In this embodiment, the host sends a write command to the storage array controller through the bus interface. The storage array controller is a RAID controller. The write command carries the data to be written and logical address information. The front-end interface of the storage array controller receives the write command, parses out the data to be written, and temporarily stores the data to be written in the local data cache. The local data cache can be a high-speed cache device such as dynamic random access memory or static random access memory, and there is no specific limitation on it.

[0035] Step S12: Perform compression processing on the data to be written to obtain the target compression ratio of the data to be written.

[0036] In this embodiment, the compression engine inside the storage array controller performs real-time compression processing on the data to be written. After compression, the compression engine outputs the target compression ratio of the data to be written, which is defined as the ratio of the original size of the data to be written to the compressed size. Figure 2 The diagram shows the main data flow of the RAID controller when the host writes to a RAID5 storage array. First, for the newly written data D', the compression ratio feature needs to be extracted by the compression engine.

[0037] Step S13: Continuously update the maximum and minimum compression ratios using historical write data, and update the moving average of the compression ratio based on the target compression ratio and the size of the data to be written, to obtain the updated moving average of the compression ratio. Based on the updated maximum, minimum, and moving average of the compression ratio, divide the current compression ratio range into intervals and assign a unique mapping identifier to each current interval; where historical write data refers to data that precedes the data to be written in chronological order.

[0038] In this embodiment, the storage array controller maintains three statistical values: the maximum compression ratio, the minimum compression ratio, and the moving average compression ratio. After each write operation is completed, the storage array controller compares the target compression ratio of that data with the currently maintained maximum and minimum compression ratios. If the target compression ratio is greater than the current maximum compression ratio, the maximum compression ratio is updated accordingly; if the target compression ratio is less than the current minimum compression ratio, the minimum compression ratio is updated accordingly. Simultaneously, the storage array controller updates the moving average compression ratio using the target compression ratio and the size of the data to obtain the updated moving average compression ratio.

[0039] Specifically, the moving average of the previous compression ratio is updated using the target compression ratio, the size of the data to be written, the moving average of the previous compression ratio, and the preset moving average window size to obtain the updated moving average of the compression ratio. The preset moving average window size is the preset number of bytes to be counted. In essence, the updated moving average of the compression ratio = (size of data to be written × target compression ratio + (preset moving average window size - size of data to be written) × moving average of the previous compression ratio) / preset moving average window size.

[0040] The sliding window mechanism allows the compression ratio moving average to reflect the compression characteristics of recently written data, rather than the average value over the entire historical period, thus responding more quickly to load changes.

[0041] Furthermore, a preset time decay factor can be obtained, which decreases as the time interval between the writing time of historical data and the current time increases. Using the compression ratio corresponding to each historical write data, the size of each historical write data, and its corresponding time decay factor, a weighted average of the compression ratios of each historical write data is calculated to obtain the updated moving average of the compression ratio. It can be understood that, based on the above sliding window statistical model, a time decay weighting mechanism is introduced. The storage array controller maintains a time decay factor for each historical write data, which decreases as the time interval between the data writing time and the current time increases, for example, using an exponential decay function or a linear decay function. When updating the moving average of the compression ratio, the compression ratio of each historical write data is no longer calculated with equal weight, but is multiplied by its respective time decay factor before being weighted and averaged. Data with longer time intervals contribute less to the current moving average; recently written data receives higher weight. Compared to the moving average method with a fixed window size, this method can discard outdated statistical information and respond to recent changes in data characteristics more quickly in scenarios where load characteristics change rapidly. This makes the moving average of the compression ratio closer to the current actual load, thereby improving the real-time performance and accuracy of interval division and mapping decisions.

[0042] After obtaining the three updated statistical values, the storage array controller divides the current compression ratio range into intervals based on the updated maximum compression ratio, the updated minimum compression ratio, and the updated moving average compression ratio. The current compression ratio range refers to the numerical range from the updated minimum compression ratio to the updated maximum compression ratio. Since the statistical values ​​are continuously updated during the write process, the range of the current compression ratio range also changes dynamically, and the interval boundaries are dynamically adjusted accordingly.

[0043] Specifically, the minimum value of the updated compression ratio is used as the lower bound of the interval, and the maximum value of the updated compression ratio is used as the upper bound of the interval to determine the current compression ratio range. The current compression ratio range is divided into multiple intervals using the moving average of the updated compression ratio and the preset interval division strategy, and a unique mapping identifier is assigned to each current interval.

[0044] This invention provides the following three specific implementation methods for interval division.

[0045] In one specific implementation, the preset interval division strategy is an average division strategy; correspondingly, the current compression ratio value range is divided into multiple intervals using the preset interval division strategy, and a unique mapping identifier is assigned to each current interval, including: uniformly dividing the current compression ratio value range into multiple current intervals of equal length, and assigning a unique mapping identifier to each current interval. For example... Figure 3 As shown, let the current compression ratio CR t If the value range is [min(CR), max(CR)], and the total number of mapping identifiers is N, then mapID is represented as: ; in, This represents the minimum compression ratio of history. This represents the maximum compression ratio in history. This indicates the number of combinations of RUH (Reclaim Unit Handle, host-allocable physical write unit handle) and RG (Reclaim Group, logical data reclamation group).

[0046] The average partitioning strategy is simple to calculate, directly dividing the compression ratio range into sub-intervals of equal length without complex calculations, making it suitable for rapid implementation; it does not make any pre-set assumptions about the compression ratio distribution, avoiding the introduction of distribution preferences; and it ensures that all compression ratio value ranges (such as from low compression ratio to high compression ratio) are covered by the same number of sub-intervals.

[0047] In another specific implementation, the preset interval division strategy is a mean-center-based two-sided segmentation strategy. Correspondingly, the current compression ratio range is divided into multiple intervals using the updated compression ratio moving average and the preset interval division strategy, and each current interval is assigned a unique mapping identifier. This includes: using the updated compression ratio moving average as the boundary, dividing the current compression ratio range into a preset high compression ratio region and a preset low compression ratio region; uniformly dividing the preset high compression ratio region into multiple intervals, and uniformly dividing the preset low compression ratio region into multiple intervals, and assigning a unique mapping identifier to each current interval. If the total number of mapping identifiers is N, then N / 2 mapping identifiers are assigned to the low compression ratio region, and another N / 2 mapping identifiers are assigned to the high compression ratio region. Each region is then further uniformly divided. This strategy, using the moving average as the boundary, is suitable for scenarios where there is a clear high-low boundary in the compression ratio distribution.

[0048] like Figure 4 As shown, the interval is divided into two regions, high and low, by the mean. Each region is then further divided evenly. The mapID under the corresponding strategy is then represented as follows: exist hour: ; otherwise: ; and The historical moving average of compression ratio is calculated using the following formula: ; in, This indicates the size of the data currently being written, in bytes. This indicates the size of the sliding statistics window, in bytes.

[0049] The mean-center bilateral segmentation method uses the mean compression ratio as a boundary to independently divide regions into low compression ratio (poor compression effect) and high compression ratio (good compression effect) regions, facilitating comparative analysis. If there is a clear high-low boundary in the compression ratio distribution (e.g., the mean represents the compression performance threshold), the density of sub-intervals in the high and low regions can be optimized accordingly. Each region occupies half of the sub-intervals, avoiding over-coverage of sparse regions by the uniform segmentation method.

[0050] In another specific implementation, the preset interval division strategy is a mean center density decreasing segmentation strategy. Correspondingly, the current compression ratio range is divided into multiple intervals using the updated compression ratio moving average and the preset interval division strategy, and a unique mapping identifier is assigned to each current interval. This includes: dividing the current compression ratio range into multiple intervals centered on the updated compression ratio moving average, and assigning a unique mapping identifier to each current interval; wherein the length of each current interval increases with the distance from the updated compression ratio moving average. That is, the closer the region is to the moving average, the finer the interval division and the shorter the interval length; the farther the region is from the moving average, the sparser the interval division and the longer the interval length. This strategy sets finer intervals near the moving average, suitable for application scenarios where the compression ratio is concentrated near the mean.

[0051] like Figure 5 As shown, when the region is symmetrically divided around the mean, the length of the sub-intervals increases with distance from the mean, and the central region has a higher resolution. Therefore, the mapID representation under the corresponding strategy is: like , ; otherwise: ; The advantage of the mean-center density decreasing segmentation method is that it sets finer sub-intervals near the mean (the area of ​​high compression ratio), which is suitable for capturing the business impact of small fluctuations in compression ratio. Compression ratio and business metrics (such as storage costs) often have a non-linear relationship, and densely dividing the core area can better match actual needs. Sparse areas (such as extremely high / extremely low compression ratios) are covered with fewer sub-intervals to avoid resource waste.

[0052] Step S14: Determine the unique mapping identifier corresponding to the current interval where the target compression rate is located as the target mapping identifier.

[0053] In this embodiment, after dividing the current compression ratio range into intervals, the storage array controller determines the interval into which the target compression ratio falls, and uses the unique mapping identifier corresponding to that interval as the target mapping identifier for the data to be written. Figure 6 As shown, the compression ratio to mapping identifier module takes the target compression ratio of the data to be written, D', as input and outputs a mapping identifier. Internally, this module records and calculates historical compression ratio data: maximum compression ratio, minimum compression ratio, and moving average compression ratio. After the target compression ratio is input for each piece of data to be written, the module first updates three statistical values, then divides the value range based on the updated statistical values, and finally determines the mapping identifier corresponding to the range in which the target compression ratio falls.

[0054] Step S15: Determine the target recyclable unit handle and target recycling group corresponding to the target mapping identifier based on the correspondence between the preset mapping identifier and the storage location identifier; wherein, the storage location identifier includes the recyclable unit handle and the recycling group.

[0055] In this embodiment, a matrix is ​​constructed with recyclable unit handles as rows and recyclable groups as columns, and this matrix serves as a storage location identifier. Each unique mapping identifier is mapped to a storage location identifier in a preset order to obtain the correspondence between the preset mapping identifier and the storage location identifier. The recyclable unit handles and recyclable groups form a matrix, where each element is a storage location identifier, representing a specific (recyclable unit handle, recyclable group) combination. The mapping identifier mapID, from 0 to 11, is mapped sequentially to each element of the matrix. For example, mapID 0 corresponds to (RUH0, RG0), mapID 1 corresponds to (RUH0, RG1), and so on. Different mapping orders can adapt to different application characteristics. The mapping from mapID to a specific (RUH, RG) combination can be defined according to specific application characteristics. Figure 7 Here are some typical mapping methods from mapID: 0~11 to the (RUH, RG) matrix.

[0056] In this embodiment, based on the correspondence constructed above, the target recyclable unit handle and target recycling group corresponding to the target mapping identifier are determined.

[0057] Step S16: Read the original data and original verification data corresponding to the data to be written, and perform array verification calculation based on the data to be written, the original data and the original verification data to generate new verification data.

[0058] In this embodiment, the storage array controller determines the physical location of the corresponding original data and original parity data in the storage device based on the logical address of the data to be written, and issues a read command to the storage device to read the original data and original parity data into the local data cache. Then, the calculation engine performs array parity calculation based on the data to be written, the original data, and the original parity data to generate new parity data. Taking RAID5 as an example, the calculation method for the new parity data P' is: P' = D' XOR D XOR P, where D' is the data to be written, D is the original data, and P is the original parity data. Taking RAID6 as an example, additional Q parity data needs to be calculated.

[0059] like Figure 2 As shown, the original data D and the old parity data P are read from the disk side into the data cache within the RAID, and the new parity data P' is generated by the RAID calculation engine.

[0060] Step S17: Determine the reclaimable cell handle and reclaim group corresponding to the new verification data.

[0061] In this embodiment, the new verification data also needs to be written to the storage device carrying the handle of the reclaimable unit and the reclaimable group tag, such as... Figure 2 As shown, the P' data also needs to be selected and determined according to certain rules for RUH / RG. This invention provides the following three methods for determining the storage location of the verification data.

[0062] In one specific implementation, the target reclaimable unit handle and the target reclaimable group are determined as the reclaimable unit handle and reclaimable group corresponding to the new verification data. The verification data inherits the storage location identifier of the data to be written and is written to the same physical storage area as the data to be written, thus maintaining the correlation between the verification data and the data block in physical location.

[0063] In another specific implementation, the write operation type corresponding to the data to be written is checked to see if it is random write or sequential write. When the write operation type is random write, the first reclaimable unit handle and the first reclaimable group are determined as the reclaimable unit handle and reclaimable group corresponding to the new verification data. When the write operation type is sequential write, the second reclaimable unit handle and the second reclaimable group are determined as the reclaimable unit handle and reclaimable group corresponding to the new verification data. Random write corresponds to read-modify-write operations, and sequential write corresponds to full-strip write operations. By distinguishing the write operation type and matching different storage locations, the storage efficiency under different write modes can be further optimized.

[0064] In another specific implementation, the reclaimable cell handle and reclaimable group corresponding to the new verification data are queried from a preset storage location identifier library. During RAID group initialization, at least one dedicated reclaimable cell handle and reclaimable group are pre-allocated for the verification data and stored in the preset storage location identifier library. When verification data is written, the dedicated storage location identifier is directly retrieved from this library. This method isolates the verification data from the data blocks, avoiding mutual interference.

[0065] Step S18: Write the data to be written, carrying the target reclaimable unit handle and the target reclaim group, to the storage device, and write the new verification data, carrying the corresponding reclaimable unit handle and reclaim group, to the storage device.

[0066] In this embodiment, the storage array controller generates a write command, encapsulating the target reclaimable cell handle and the target reclaim group within the write command, and sends the write command to the storage device to complete the disk write operation of the data to be written and the new verification data. After receiving the write command, the storage device writes the data to the corresponding physical storage area according to the reclaimable cell handle and the reclaim group tag.

[0067] like Figure 2 The diagram shows the data flow of the write process in a RAID 5 storage array. New data D' written by the host first enters the compression engine for compression ratio feature extraction. After obtaining the compression ratio data, a mapping from the compression ratio to reclaimable cell handles and reclaimable groups is determined. Simultaneously, the original data D and the old parity data P are read from the disk side into the data cache within the RAID controller. The RAID calculation engine generates new parity data P', and P' data is also used to select reclaimable cell handles and reclaimable groups according to preset rules. Finally, D' and P', carrying the specified reclaimable cell handles and reclaimable groups, are written to the NVMe solid-state drive.

[0068] like Figure 8 The diagram shows the control flow of a RAID controller write operation. When the RAID controller is in write-back mode, it can perform write I / O (Input / Output) on the host to complete the transfer of written data from the host to the RAID controller's data buffer, thus achieving lower write latency from the host side. Figure 8 The document also demonstrates that the compression ratio feature extraction process of data D' is executed concurrently with the normal RAID processing flow to achieve lower backend I / O processing latency, while requiring an additional concurrent synchronization mechanism to ensure this. There are three RUH / RG mapping methods selected during the writing of the newly generated parity data P' to the SSD (Solid State Drive), which have been described in the aforementioned embodiments and will not be repeated here.

[0069] In this embodiment, the operations of reading the original data and the original verification data in steps S12 and S16 can be executed concurrently. Specifically, a first processing thread and a second processing thread are constructed and started in parallel. The first processing thread is used to perform compression processing on the data to be written to obtain the target compression ratio, and the second processing thread is used to read the original data and the original verification data corresponding to the data to be written. The two threads execute concurrently without blocking each other, effectively reducing the overall processing latency.

[0070] Therefore, by dynamically mapping the compression ratio to the RUH / RG cells of FDP (Flexible Data Placement), data with similar compression characteristics is ensured to be stored centrally, optimizing garbage collection efficiency and reducing the amount of effective data migration. RG cells isolate data with different compression ratios, avoiding mixed load interference, improving I / O performance stability, reducing NAND wear, and extending SSD lifespan. Compression ratio feature extraction and dynamic mapping are implemented within the RAID controller, requiring no driver modification or additional metadata from the host, seamlessly supporting existing FDP SSDs. Based on a sliding window to statistically analyze historical compression ratios (maximum, minimum, and average), the mapping strategy is adjusted in real time, overcoming the limitation of static mapping in adapting to changes in data characteristics. Three dedicated mapping strategies are proposed for parity data (P' / Q'): inheriting the D' tag, pre-allocating dedicated RUH / RG cells, and distinguishing between random and sequential writes, to prevent parity blocks from compromising the FDP optimization effect.

[0071] Furthermore, embodiments of the present invention also disclose an electronic device, Figure 9 This is a schematic diagram of an electronic device structure according to an exemplary embodiment. Figure 9 The content herein should not be construed as limiting the scope of the invention. Specifically, the electronic device may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the dynamic data storage method for the storage array disclosed in any of the foregoing embodiments. Furthermore, the electronic device in this embodiment may specifically be an electronic computer.

[0072] In this embodiment, the power supply 23 is used to provide operating voltage for various hardware devices on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this invention, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0073] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0074] The operating system 221 is used to manage and control the various hardware devices on the electronic device and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the dynamic storage method for storage array data executed by the electronic device as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0075] Furthermore, the present invention also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned method for dynamically storing data in a storage array. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0076] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0077] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0078] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0079] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0080] The technical solution provided by the present invention has been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for dynamically storing data in a storage array, characterized in that, Applied to storage array controllers, including: Receive the data to be written from the host and cache the data to be written in the local data cache; Perform compression processing on the data to be written to obtain the target compression ratio of the data to be written; The maximum and minimum compression ratios are continuously updated using historical write data. The moving average compression ratio is also updated based on the target compression ratio and the size of the data to be written, resulting in an updated moving average. The current compression ratio range is then divided into intervals based on these updated maximum, minimum, and moving average values, and a unique mapping identifier is assigned to each interval. The historical write data refers to data that precedes the data to be written in chronological order. The unique mapping identifier corresponding to the current interval where the target compression ratio is located is determined as the target mapping identifier; Based on the correspondence between the preset mapping identifier and the storage location identifier, the target recyclable unit handle and the target recycling group corresponding to the target mapping identifier are determined; wherein, the storage location identifier includes the recyclable unit handle and the recycling group; Read the original data and original verification data corresponding to the data to be written, and perform array verification calculation based on the data to be written, the original data and the original verification data to generate new verification data; Determine the reclaimable unit handle and reclaim group corresponding to the new verification data; The data to be written, carrying the handle of the target recyclable unit and the target recyclable group, is written to the storage device, and new verification data carrying the corresponding recyclable unit handle and recyclable group is written to the storage device.

2. The method for dynamically storing data in a storage array according to claim 1, characterized in that, The process of dividing the current compression ratio range into intervals based on the updated maximum compression ratio, the updated minimum compression ratio, and the updated moving average compression ratio, and assigning a unique mapping identifier to each current interval, includes: The minimum value of the updated compression ratio is used as the lower bound of the interval, and the maximum value of the updated compression ratio is used as the upper bound of the interval to determine the current compression ratio range. The current compression ratio range is divided into multiple intervals using the updated compression ratio moving average and the preset interval division strategy, and a unique mapping identifier is assigned to each current interval.

3. The method for dynamically storing data in a storage array according to claim 2, characterized in that, The preset interval division strategy is an average division strategy; Accordingly, the current compression ratio range is divided into multiple intervals using a preset interval division strategy, and a unique mapping identifier is assigned to each current interval, including: The current compression ratio range is evenly divided into multiple current intervals of equal length, and a unique mapping identifier is assigned to each current interval.

4. The method for dynamically storing data in a storage array according to claim 2, characterized in that, The preset interval division strategy is a mean center-based two-sided segmentation strategy; Accordingly, the step of dividing the current compression ratio range into multiple intervals using the updated compression ratio moving average and the preset interval division strategy, and assigning a unique mapping identifier to each current interval, includes: Using the updated compression ratio moving average as the boundary, the current compression ratio range is divided into a preset high compression ratio region and a preset low compression ratio region; The preset high compression ratio region is evenly divided into multiple intervals, and the preset low compression ratio region is evenly divided into multiple intervals, and a unique mapping identifier is assigned to each current interval.

5. The method for dynamically storing data in a storage array according to claim 2, characterized in that, The preset interval division strategy is a mean center density decreasing segmentation strategy; Accordingly, the step of dividing the current compression ratio range into multiple intervals using the updated compression ratio moving average and the preset interval division strategy, and assigning a unique mapping identifier to each current interval, includes: Centered on the updated compression ratio moving average, the current compression ratio value range is divided into multiple intervals, and a unique mapping identifier is assigned to each current interval; wherein, the interval length of each current interval increases as the distance from the updated compression ratio moving average increases.

6. The method for dynamically storing data in a storage array according to claim 1, characterized in that, The step of updating the moving average of the compression ratio based on the target compression ratio and the size of the data to be written, to obtain the updated moving average of the compression ratio, includes: The previous compression ratio sliding average is updated using the target compression ratio, the size of the data to be written, the previous compression ratio sliding average, and the preset sliding statistical window size to obtain the updated compression ratio sliding average; wherein, the preset sliding statistical window size is a preset number of statistical bytes.

7. The method for dynamically storing data in a storage array according to claim 1, characterized in that, Before determining the target recyclable unit handle and target recycling group corresponding to the target mapping identifier based on the correspondence between the preset mapping identifier and the storage location identifier, the method further includes: Construct a matrix with recyclable unit handles as rows and recyclable groups as columns, and use the matrix as a storage location identifier; Each unique mapping identifier is mapped to the storage location identifier in a preset order to obtain the correspondence between the preset mapping identifier and the storage location identifier.

8. The method for dynamically storing data in a storage array according to claim 1, characterized in that, Determining the reclaimable unit handle and reclaimable group corresponding to the new verification data includes: The target recyclable unit handle and the target recyclable group are determined as the recyclable unit handle and recyclable group corresponding to the new verification data.

9. The method for dynamically storing data in a storage array according to claim 1, characterized in that, Determining the reclaimable unit handle and reclaimable group corresponding to the new verification data includes: Check whether the write operation type corresponding to the data to be written is random write or sequential write; When the write operation type is random write, the first reclaimable unit handle and the first reclaimable group are determined to be the reclaimable unit handle and reclaimable group corresponding to the new verification data; When the write operation type is sequential write, the second reclaimable unit handle and the second reclaimable group are determined as the reclaimable unit handle and reclaimable group corresponding to the new verification data.

10. The method for dynamically storing data in a storage array according to claim 1, characterized in that, Determining the reclaimable unit handle and reclaimable group corresponding to the new verification data includes: Retrieve the handle of the recyclable unit and the recyclable group corresponding to the new verification data from the preset storage location identifier library.

11. The method for dynamically storing data in a storage array according to claim 1, characterized in that, The storage array controller is a standalone disk redundant array controller.

12. The method for dynamically storing data in a storage array according to any one of claims 1 to 11, characterized in that, Also includes: A first processing thread and a second processing thread are constructed and started in parallel. The first processing thread is used to perform compression processing on the data to be written to obtain the target compression ratio of the data to be written. The second processing thread is used to read the original data and original verification data corresponding to the data to be written.

13. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the dynamic storage method for storage array data as described in any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the dynamic storage method for storage array data as described in any one of claims 1 to 12.

15. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the dynamic storage method for storage array data as described in any one of claims 1 to 12.

Images