Buffer generation method and device, equipment and storage medium

Abstract

The invention discloses a buffer generation method and device, equipment and a storage medium, and relates to the technical field of solid state disks. Comprising the steps of obtaining a disk array level of a target server and a hard disk type of the target server; generating a first buffer area of the target server based on the disk array level and the hard disk type; the first buffer area is used for temporarily storing verification data of a disk stripe which is being processed by the target server; generating a second buffer area of the target server from a host memory of the target server; the second buffer area is used for temporarily storing the verification data of the disk stripe which is not processed by the target server. The write-in speed and the hardware cost of the disk stripe can be considered, and effective balance between the write-in speed and the hardware cost is achieved.

Description

Technical Field

[0001] This application belongs to the field of solid-state drive technology, and particularly relates to a buffer generation method, apparatus, device and storage medium. Background Technology

[0002] With the rapid development of information technology, servers are increasingly demanding higher reliability and performance in data storage. Solid-state drives (SSDs) have become widely used in the server field due to their significant advantages such as fast read / write speeds and strong shock resistance. However, NAND flash memory, as the core storage medium of SSDs, is inherently susceptible to data errors due to its characteristics. Therefore, employing effective data protection mechanisms is crucial to ensure data security.

[0003] Currently, servers commonly employ data protection mechanisms based on disk arrays, such as RAID 5 and RAID 6. To cache the parity data generated by disk stripes using these data protection mechanisms, a buffer needs to be created in the server to temporarily store the parity data.

[0004] However, as the complexity of NAND flash memory technology continues to rise, and servers become increasingly cost-conscious, existing buffer generation methods struggle to balance disk stripe write speed and hardware cost, failing to achieve an effective balance between the two. Summary of the Invention

[0005] This application provides a buffer generation method, apparatus, device, and storage medium that can effectively balance disk stripe write speed and hardware cost.

[0006] A first aspect of this application provides a buffer generation method, comprising: Obtain the target server's disk array level and hard drive type; Based on the disk array level and hard drive type, a first buffer is generated for the target server; the first buffer is used to temporarily store the verification data of the disk stripes being processed by the target server. A second buffer for the target server is generated from the host memory of the target server; the second buffer is used to temporarily store the verification data of the disk stripes that have not been processed by the target server.

[0007] Furthermore, this application also proposes generating a first buffer for the target server based on the disk array level and hard disk type, including: In response to the disk array level being RAID5 and the hard drive type being TLC that supports write-through, a first buffer of the target server with the first capacity is generated. In response to the disk array being RAID5 and the hard drive type being TLC that does not support write-through, a first buffer for the target server with a second capacity is generated; the second capacity is less than the first capacity.

[0008] Furthermore, this application also proposes that, in response to RAID5 including N disk array parity modes, the first capacity is set to N page sizes and the second capacity is set to one page size; N is a positive integer.

[0009] Furthermore, this application also proposes generating a first buffer for the target server based on the disk array level and hard disk type, including: In response to the disk array level being RAID6 and the hard drive type being TLC that supports write-through, a first buffer of the target server with a third capacity is generated; the third capacity is twice the first capacity, and the first capacity is the size of the first buffer of the target server when the disk array level is RAID5 and the hard drive type is TLC that supports write-through; In response to the disk array level being RAID6 and the hard drive type being TLC that does not support write-through, a first buffer of the target server with a fourth capacity is generated; the fourth capacity is twice the second capacity, and the second capacity is the size of the first buffer of the target server when the disk array level is RAID5 and the hard drive type is TLC that does not support write-through.

[0010] Furthermore, this application also proposes that, after generating the first buffer of the target server based on the disk array level and hard disk type, the method further includes: In response to the disk array being RAID5 and disk striping being completed, the parity data of the disk stripes is written to the permanent flash memory; In response to the disk array being RAID6 and the disk striping process being completed, the parity data of the disk stripes is discarded.

[0011] Furthermore, this application also proposes that, after generating the second buffer of the target server from the host memory of the target server, the method further includes: In response to the target server switching the data write mode, the first check data in the first buffer is swapped into the second buffer, and the second check data in the second buffer is swapped into the first buffer. The first verification data is the verification data of the disk stripe that was first written to the first buffer and is not currently processed, and the second verification data is the verification data of the next disk stripe to be processed.

[0012] Furthermore, this application also proposes that, after generating the first buffer of the target server based on the disk array level and hard disk type, the method further includes: Obtain the parity data load of the disk stripes being processed by the target server; In response to the verification data load meeting the dynamic adjustment conditions, the buffer capacity of the first buffer of the target server is dynamically adjusted.

[0013] A second aspect of this application provides a buffer generation apparatus, comprising: The information acquisition module is used to obtain the disk array level and hard drive type of the target server. The buffer generation module is used to generate the first buffer of the target server based on the disk array level and hard disk type; the first buffer is used to temporarily store the verification data of the disk stripe being processed by the target server; The buffer generation module is also used to generate a second buffer for the target server from the host memory of the target server; the second buffer is used to temporarily store the verification data of the disk stripes that have not been processed by the target server.

[0014] A third aspect of the embodiments of this application provides an electronic device, the device comprising: a memory and a program or instructions stored in the memory and executable on a processor, wherein when the program or instructions are executed by the processor, they implement the buffer generation method provided in any of the embodiments of this application described above.

[0015] A fourth aspect of the embodiments of this application provides a readable storage medium on which a program or instructions are stored, and when the program or instructions are executed by a processor, they implement the buffer generation method provided by any aspect of the embodiments of this application described above.

[0016] A fifth aspect of the embodiments of this application provides a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform a buffer generation method as provided in any aspect of the embodiments of this application described above.

[0017] In the buffer generation method provided in this application embodiment, a first buffer is generated by obtaining the disk array level and hard disk type of the target server. Based on the server's own characteristics, a suitable buffer can be generated to temporarily store the verification data of the disk stripe being processed. This helps ensure the smoothness of the disk stripe writing process and improves the write speed. Simultaneously, a second buffer is generated from the host memory of the target server to temporarily store the verification data of the unprocessed disk stripe, utilizing host memory resources without incurring excessive additional hardware costs. Thus, this application effectively balances disk stripe write speed and hardware cost. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating a buffer generation method provided in one embodiment of this application; Figure 2 This is a schematic diagram of the structure of a buffer generation device provided in one embodiment of this application; Figure 3 This is a schematic diagram of the structure of a buffer generation device provided in one embodiment of this application. Detailed Implementation

[0020] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0021] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0022] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of this application all comply with the relevant provisions of national laws and regulations.

[0023] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0024] In traditional server data storage systems, the increasing complexity of NAND flash memory technology and increasingly stringent hardware cost control requirements make it difficult for existing buffer generation mechanisms to achieve a balance between disk stripe write speed and hardware resource consumption. Specifically, when servers use disk arrays for data protection, the temporary storage requirements for verification data lead to an inflexible buffer configuration, creating an irreconcilable contradiction between system performance optimization and hardware cost constraints, which in turn affects the continuity of data processing and resource utilization efficiency.

[0025] If the above problems are not resolved, the imbalance in resource allocation during the data verification process will further exacerbate the system performance bottleneck, the data integrity guarantee mechanism may be weakened, the server's reliable operation capability under continuous high load will be significantly restricted, and ultimately the scalability and stability of the data storage system will be limited.

[0026] In this regard, such as Figure 1 As shown, this application proposes a flowchart of a buffer generation method, which can be applied to electronic devices. The buffer generation method may include the following steps S100 to S300: S100, obtain the disk array level of the target server and the hard disk type of the target server; S200, based on the disk array level and hard drive type, generates the first buffer of the target server; the first buffer is used to temporarily store the verification data of the disk stripe being processed by the target server; S300 generates a second buffer for the target server from the host memory of the target server; the second buffer is used to temporarily store the verification data of the disk stripes that have not been processed by the target server.

[0027] In this embodiment, a Redundant Array of Independent Disks (RAID) level is a technology that combines multiple physical hard drives to form one or more logical hard drives. Its main purpose is to improve data storage performance, reliability, and fault tolerance. Common RAID levels include RAID 5 and RAID 6. For example, RAID 5 achieves data redundancy through distributed parity checking, allowing data to be retained even if a single hard drive fails; RAID 6 provides higher fault tolerance through dual parity checking, allowing for the simultaneous failure of two hard drives. Different RAID levels have different requirements for the generation and storage of parity data.

[0028] Hard drive type specifically refers to the type of flash memory media used in solid-state drives (SSDs), such as Triple-Level Cell (TLC). Each TLC flash memory cell can store 3 bits of data, offering high storage density and low cost. "Supports write-through TLC" typically means that the TLC flash memory has the ability to be written directly, i.e., no additional erase operation is required when writing data, thus improving write efficiency; while "does not support write-through TLC" requires erasing before writing, resulting in decreased write performance.

[0029] The first buffer refers to a memory area generated on the target server. Its main function is to temporarily store the parity data of the disk stripe currently being processed by the target server. This buffer is usually configured as a cache to ensure that the parity data can be accessed quickly, thereby supporting efficient disk stripe write operations.

[0030] The second buffer is a memory area allocated from the target server's host memory. Its main function is to temporarily store the parity data of disk stripes that the target server has not yet processed. This buffer is used to store parity data that has been generated but has not yet entered the first buffer for processing, serving as a supplement to the first buffer to cope with peak data writes or processing delays.

[0031] Disk striping refers to the practice in a disk array where data is divided into fixed-size blocks, which are then distributed across different hard drives in the array. Each such data block and its corresponding parity data together constitute a disk stripe. Striping technology enables parallel data reading and writing, thereby improving overall I / O performance.

[0032] Checksum data refers to the data generated in a disk array to achieve data redundancy and fault tolerance. When raw data is written to the disk array, corresponding checksum data is generated according to a specific algorithm (such as XOR operation). If a hard drive in the array fails, the lost data can be reconstructed using the remaining data and checksum data, thereby ensuring data integrity and reliability.

[0033] This method includes obtaining the target server's disk array level and hard drive type. In practice, this information can be obtained in several ways. For example, the system can automatically identify the current disk array level and hard drive type by reading the server's hardware configuration information or through APIs provided by the operating system. Another method is for the administrator to manually input or select the corresponding configuration parameters during system initialization or deployment. Alternatively, this information can be loaded through a pre-defined configuration file. The purpose of these methods is to provide the necessary basis for subsequent buffer generation.

[0034] Secondly, based on the disk array level and hard drive type, a first buffer is generated for the target server. This first buffer is used to temporarily store the parity data of the disk stripes being processed by the target server. When generating the first buffer, capacity allocation can be based on different strategies. For example, a default capacity can be allocated based on the detected disk array level, such as RAID 5. Alternatively, an additional capacity can be allocated based on the hard drive type, such as TLC flash memory. These methods can all achieve the generation of the first buffer, but the appropriateness of its capacity varies depending on the actual application scenario.

[0035] Next, a second buffer is generated from the target server's host memory. This second buffer is used to temporarily store the checksum data of the unprocessed disk stripes on the target server. The method for generating the second buffer is also varied. For example, a fixed-size region in the host memory can be reserved as the second buffer; the size of this region is determined at system startup. For instance, a 64MB region could be selected as the second buffer. Alternatively, the system can dynamically allocate a portion of the host memory as the second buffer based on its availability.

[0036] As an example, suppose a server system needs to handle a large number of data write requests. The server is configured with a RAID 5 disk array, and its hard drives are solid-state drives that support write-through TLC. During the data write process, a large amount of parity data is generated that needs to be temporarily stored.

[0037] First, the server system performs an acquisition operation to identify that the current disk array level is RAID5 and the hard drive type is TLC that supports write-through. This information is captured by the system and used as the basis for subsequent decisions.

[0038] Next, the system generates the server's first buffer based on the acquired RAID 5 array level and the type of hard drive supporting write-through TLC. For example, the system can configure a base capacity for RAID 5 according to a preset strategy, and adjust this base capacity appropriately based on the write-through TLC feature to ensure that the first buffer can efficiently cache the parity data of the disk stripes being processed. This first buffer is configured as a high-speed cache to meet the need for fast access to parity data.

[0039] Simultaneously, the system allocates a portion of the server's host memory to create a second buffer. This second buffer is used to temporarily store parity data for disk stripes that have been generated but not yet processed by the first buffer. For example, if the data write speed is very fast, causing the amount of parity data generated to exceed the immediate processing capacity of the first buffer, the excess parity data can be temporarily stored in the second buffer, waiting to be processed when the first buffer becomes idle.

[0040] Therefore, when a new data write request arrives, the parity data of the corresponding disk stripe is first sent to the first buffer for processing. If the first buffer is full or is processing other data, the newly generated parity data is temporarily stored in the second buffer. In this way, the parity data being processed and the parity data to be processed are effectively distinguished and managed, ensuring a smooth parity data flow and avoiding write delays or performance degradation caused by insufficient buffers.

[0041] Based on the above examples, the technical concept of this embodiment demonstrates its technical contribution by generating a buffer by comprehensively considering the server's disk array level and hard drive type. In the prior art, buffers are usually generated using a fixed size or a single-dimensional consideration, which may lead to unreasonable resource allocation. For example, if a fixed-size buffer is used, in a RAID5 scenario with hard drives supporting write-through TLC, the write performance of the hard drives may not be fully utilized, or buffer overflow may easily occur when the amount of parity data is large, thus affecting data write efficiency.

[0042] This embodiment, by acquiring the disk array level and hard drive type, can customize the generation of the first buffer based on the actual hardware configuration and data protection requirements. This customization strategy allows the buffer capacity to more accurately match the processing needs of the verification data, avoiding resource waste and improving the processing efficiency of the verification data. For example, in the above example, if the hard drive type is write-through TLC, the system can allocate a more optimized first buffer capacity to fully utilize the hard drive's write performance.

[0043] Furthermore, this embodiment introduces a second buffer, generated from host memory, for temporarily storing unprocessed verification data. This design effectively separates the management of verification data being processed from that of data awaiting processing. Compared to schemes relying solely on a single buffer, the existence of the second buffer provides an additional buffer area for verification data, especially during peak data write periods. This effectively alleviates the pressure on the first buffer, ensuring that verification data can be cached and processed in an orderly manner, thereby improving the stability and reliability of the entire data write process. This dual-buffer collaborative approach allows the server to process verification data more flexibly and efficiently, thus optimizing hardware resource utilization while maintaining disk stripe write speed.

[0044] In this embodiment, the disk array level and hard drive type of the target server are obtained to generate a first buffer. Based on the server's own characteristics, a suitable buffer can be generated to temporarily store the verification data of the disk stripes being processed. This helps ensure the smoothness of the disk stripe writing process and improves the write speed. Simultaneously, a second buffer is generated from the target server's host memory to temporarily store the verification data of unprocessed disk stripes, utilizing host memory resources without incurring excessive additional hardware costs. Thus, this application effectively balances disk stripe write speed and hardware cost.

[0045] In some of the embodiments described above in this application, a first buffer for the target server is generated based on the disk array level and hard drive type. However, in actual implementation, how to reasonably determine the capacity of the first buffer according to specific hardware characteristics, especially for RAID5 configurations, in order to optimize the temporary storage efficiency of verification data and system performance, is a problem that needs further refinement and resolution.

[0046] In this regard, this application further proposes that S200 includes: In response to the disk array level being RAID5 and the hard drive type being TLC that supports write-through, a first buffer of the target server with the first capacity is generated. In response to the disk array being RAID5 and the hard drive type being TLC that does not support write-through, a first buffer for the target server with a second capacity is generated; the second capacity is less than the first capacity.

[0047] In this embodiment, when the target server's disk array level is RAID5 and its hard drive type supports write-through TLC, the system allocates a preset first capacity for the first buffer. The characteristics of RAID5 dictate that it requires the calculation and storage of parity data during data writing, while hard drives supporting write-through TLC typically have better write performance, enabling more efficient parity data processing. The first capacity is set to fully utilize the advantages of these high-performance hard drives, ensuring sufficient space for temporary storage of the parity data of the disk stripes being processed, thus avoiding performance bottlenecks caused by insufficient buffer space. For example, the first capacity can be estimated based on the parity algorithm characteristics of RAID5 and the IOPS (input / output operations per second) capability of the write-through TLC hard drive, or configured according to the system's preset performance targets.

[0048] When the target server's disk array is also RAID5, but its hard drives do not support write-through TLC, the system will allocate a second capacity to the first buffer, and this second capacity will be smaller than the first capacity. Hard drives that do not support write-through TLC generally have lower write performance than those that do, and their parity data processing speed may be relatively slower. Therefore, allocating a smaller second capacity can avoid wasting memory resources by allocating an excessively large buffer, while also accommodating its relatively lower write processing capabilities. The determination of the second capacity can be based on the actual write performance of hard drives that do not support write-through TLC, the parity data generation rate of RAID5, and the system's resource utilization considerations. For example, the second capacity can be set to a small value that can meet basic parity data storage needs without excessively consuming system resources, or it can be optimized based on actual test data.

[0049] This application's solution first obtains the target server's disk array level and hard drive type, and then further refines the generation strategy of the first buffer for the specific RAID5 disk array level. When the hard drive type is detected to support write-through TLC, the system allocates a larger first capacity to the first buffer to fully utilize the high write performance of this type of hard drive, ensuring that the parity data can be quickly and efficiently stored and processed temporarily. Conversely, when the hard drive type is detected to not support write-through TLC, the system allocates a relatively smaller second capacity to accommodate its relatively lower write performance and avoid unnecessary waste of memory resources. This mechanism of dynamically adjusting the first buffer capacity according to the RAID5 configuration and the specific write characteristics of the hard drive allows the temporary storage of parity data to more accurately match the capabilities of the underlying hardware, thereby optimizing resource utilization and improving the overall efficiency of parity data processing.

[0050] The following example illustrates this. Assume a target server is configured with a RAID 5 disk array. If the server uses enterprise-grade SSDs that support write-through TLC, the system can allocate, for example, a first capacity of 16MB when creating the first buffer. This larger capacity effectively handles parity data generated by high-concurrency write operations, ensuring smooth data flow. However, if the server uses consumer-grade SSDs that do not support write-through TLC, the system will allocate, for example, a second capacity of 4MB. This smaller capacity satisfies the temporary storage needs of parity data while avoiding idle memory resources due to disk write performance limitations, thus achieving a more economical and efficient resource allocation.

[0051] Through the above technical solution, this application can adaptively adjust the capacity of the first buffer according to the specific write characteristics of the hard drive in the RAID5 disk array (whether it supports write-through TLC). When the hard drive supports write-through TLC, allocating a larger first capacity can fully utilize the high write performance of the hard drive, improve the processing efficiency of parity data, and reduce write latency caused by insufficient buffer. When the hard drive does not support write-through TLC, allocating a smaller second capacity can avoid unnecessary waste of memory resources, while ensuring that parity data can be effectively stored and processed, thereby optimizing system resource utilization. This adaptive buffer generation strategy enables the system to maintain efficient and stable parity data processing capabilities under different hardware environments, improving the overall performance and data reliability of the target server.

[0052] In some embodiments described above in this application, a method for generating a first buffer and a second buffer based on the target server's disk array level and hard drive type is proposed. Specifically, when the disk array level is RAID5, the capacity of the first buffer is determined as either a first capacity or a second capacity based on whether the hard drive type supports write-through TLC. However, in practical applications, if the capacity of the first buffer is improperly set, failing to fully consider the internal parity mechanism of the RAID5 disk array, it may lead to inaccurate allocation of buffer resources, thereby affecting the temporary storage efficiency of parity data and overall system performance.

[0053] In response, this application further proposes that, in response to RAID5 including N disk array parity modes, the first capacity is set to N page sizes and the second capacity is set to one page size; N is a positive integer.

[0054] In this embodiment, "responding to RAID5 including N disk array parity modes" means that when a RAID5 disk array processes data, there may be N different ways to generate or store parity data. Each mode may correspond to one or more parity blocks, and these parity modes directly affect the size and structure of the parity data. For example, a RAID5 disk array includes three parity modes: LP parity mode, XP parity mode, and UP parity mode.

[0055] In the phrase "first capacity set to N page sizes," the "page size" is the smallest unit of data transfer and management in a storage system, typically 18KB or larger. Setting the first capacity to N page sizes means that the total capacity of the first buffer is a collection of N basic storage units. This setting is determined based on the N parity modes included in the RAID 5 disk array, aiming to reserve one page size of storage space for each parity mode. This ensures efficient storage and processing of all possible parity data, especially when the hard drive type supports write-through TLC. For example, the page size can be preset by the operating system or file system, or it can be dynamically negotiated based on hardware characteristics. For instance, in a RAID 5 disk array that includes three parity modes—LP, XP, and UP—the first capacity would be set to three page sizes.

[0056] "Setting the second capacity to one page size" means that when the hard drive type does not support write-through TLC, the capacity of the first buffer is limited to the size of a basic storage unit. This setting takes into account the writing mechanism or performance characteristics of hard drives that do not support write-through TLC when processing parity data. Unlike hard drives that support write-through TLC, they do not need to reserve multiple page-sized buffers. Their parity data processing is more centralized, and a one-page-sized buffer is sufficient to meet their temporary storage needs.

[0057] "N is a positive integer" indicates the number of parity modes included in the RAID5 disk array. The requirement that it is a positive integer ensures that the number of parity modes is a positive integer, thus making the calculation of the first capacity always based on the actual number of parity modes, which has a clear physical meaning.

[0058] This application's solution further considers the N parity modes included in the RAID5 disk array when generating the first buffer, and precisely sets the capacity of the first buffer accordingly. Specifically, when the hard drive type supports write-through TLC, the capacity of the first buffer is set to N page sizes. This allows the buffer to provide independent, appropriately sized storage space for each parity mode of RAID5, thus avoiding data overflow or frequent read / write operations due to an insufficiently small buffer, and also avoiding memory resource waste due to an excessively large buffer. When the hard drive type does not support write-through TLC, the capacity of the first buffer is set to one page size. This reflects the optimized consideration of the parity data processing characteristics of this type of hard drive, ensuring efficient temporary storage of parity data under different hardware conditions. This mechanism of finely adjusting the buffer capacity based on the number of RAID5 parity modes and the hard drive type allows the first buffer to more accurately match the actual needs of the RAID5 disk array, thereby optimizing the storage and processing flow of parity data and improving the overall I / O performance and resource utilization of the system.

[0059] The following is a concrete example. Assume the target server's disk array is RAID 5. In one scenario, if the RAID 5 array supports three (N=3) different parity modes, and the target server's hard drives support write-through TLC, then according to the scheme of this application, the capacity of the generated first buffer will be set to 3 pages. For example, if a page size is 18KB, the capacity of the first buffer will be 54KB. In another scenario, if the RAID 5 array also supports three parity modes, but the target server's hard drives do not support write-through TLC, then the capacity of the generated first buffer will be set to 1 page, i.e., 18KB.

[0060] Through the above technical solution, the capacity of the first buffer can be precisely matched according to the actual number of parity modes and hard drive types of the RAID5 disk array. This fine-grained capacity setting avoids blindly setting the buffer capacity and effectively solves the problem of resource waste or performance bottlenecks caused by the mismatch between the buffer capacity and actual needs. It ensures that parity data can be efficiently and stably temporarily stored under different hardware configurations, thereby improving the overall read / write performance and data processing efficiency of the disk array, while optimizing the utilization of system memory resources.

[0061] In some embodiments described above in this application, a method for generating a first buffer and a second buffer for a target server based on the disk array level and hard drive type is proposed. However, in its implementation, there is still room for optimization in determining the capacity of the first buffer reasonably for different combinations of disk array levels (e.g., RAID6) and hard drive types to ensure data processing efficiency and verify the accuracy of data storage.

[0062] In this regard, this application further proposes that S200 includes: In response to the disk array level being RAID6 and the hard drive type being TLC that supports write-through, a first buffer of the target server with a third capacity is generated; the third capacity is twice the first capacity, and the first capacity is the size of the first buffer of the target server when the disk array level is RAID5 and the hard drive type is TLC that supports write-through; In response to the disk array level being RAID6 and the hard drive type being TLC that does not support write-through, a first buffer of the target server with a fourth capacity is generated; the fourth capacity is twice the second capacity, and the second capacity is the size of the first buffer of the target server when the disk array level is RAID5 and the hard drive type is TLC that does not support write-through.

[0063] In this embodiment, "in response to the disk array level being RAID6 and the hard drive type being write-through TLC, generating a first buffer of the target server with a third capacity" means that when the system recognizes that the target server's disk array is configured as RAID6 and the hard drive type used is a flash hard drive supporting write-through TLC, the system will allocate a memory area of ​​a specific size as a first buffer for the target server. RAID6 is a high-level disk array technology characterized by a dual parity check mechanism, which can tolerate the simultaneous failure of any two hard drives in the array without data loss. While this mechanism improves data redundancy and reliability, it also increases the computational complexity and storage requirements of the parity data. Write-through TLC hard drives refer to those using TLC flash memory chips, and whose controllers support direct writing of data to TLC storage cells without conversion through a Single-Level Cell (SLC) cache layer. These hard drives typically have high sustained write performance, but their write-through characteristics may impose specific requirements on the parity data processing flow and buffer management. Generating a first buffer of the third capacity aims to efficiently store and process the parity data of the disk stripes being processed under the RAID6 level. This memory allocation can be implemented through memory management functions provided by the operating system, or managed through dedicated storage drivers or firmware modules.

[0064] The statement "The third capacity is twice the first capacity, and the first capacity is the size of the first buffer of the target server when the disk array level is RAID5 and the hard drive type supports write-through TLC" means that the third capacity is set to be twice the first buffer capacity (i.e., the first capacity) determined based on RAID5 and the hard drive type supporting write-through TLC. RAID5 uses single parity checking, and its parity data volume is relatively small. The third capacity is set to twice the first capacity because RAID6 needs to store two independent parity data (usually called P-parity and Q-parity) to achieve dual fault tolerance. Therefore, compared to the single parity of RAID5, RAID6 logically requires twice the parity data storage space. This multiple relationship directly reflects the characteristics of RAID6 in data redundancy and parity mechanism, ensuring sufficient space to accommodate additional parity information.

[0065] "In response to a RAID 6 disk array and a hard drive type that does not support write-through TLC, the system generates a first buffer of a fourth capacity for the target server." This means that when the system recognizes that the target server's disk array is configured as RAID 6, but the hard drives used are flash drives that do not support write-through TLC, the system allocates another memory area of ​​a specific size as a first buffer for the target server. Hard drives that do not support write-through TLC refer to those using TLC flash memory chips, but whose controllers do not support direct data writing to TLC storage units. Data is typically written first to a cache area simulating SLC mode, and then moved from the SLC cache layer to the TLC storage units when the cache area is full or free. When continuously writing large amounts of data, the write performance of such hard drives may be limited by the SLC cache size and the background data transfer mechanism. Generating a first buffer of a fourth capacity aims to balance memory resource usage and parity data processing efficiency, while simultaneously meeting the dual parity data storage requirements of RAID 6.

[0066] The statement "The fourth capacity is twice the second capacity, and the second capacity is the size of the first buffer of the target server when the disk array level is RAID5 and the hard drive type does not support write-through TLC" means that the fourth capacity is set to be twice the first buffer capacity (i.e., the second capacity) determined based on RAID5 and the hard drive type not supporting write-through TLC. Similarly, even with hard drive types that do not support write-through TLC, RAID6 still needs to store two copies of parity data. This multiple relationship ensures that RAID6 can obtain sufficient parity data storage space under different hard drive types, but the specific base capacity (second capacity) can be adjusted according to the write characteristics of the hard drive to adapt to its performance limitations.

[0067] This application's solution obtains the target server's disk array level and hard drive type, and dynamically adjusts the capacity of the first buffer based on this information. Specifically, when the system detects that the disk array level is RAID6, since RAID6 requires the calculation and storage of dual parity data (P-check and Q-check), its required first buffer capacity will be twice that of RAID5. This capacity doubling is directly based on the RAID6 parity mechanism to ensure sufficient space for temporary storage of two sets of parity data. Furthermore, this method further distinguishes whether the hard drive type supports write-through TLC. For hard drives that support write-through TLC, since they typically have higher write performance, the system can allocate a relatively large first buffer (third capacity) to fully utilize the hard drive's performance advantage, reduce write latency caused by insufficient buffer, and thus improve data write efficiency. For hard drives that do not support write-through TLC, since their write performance may be limited by the SLC caching mechanism, the system will allocate a relatively small first buffer (fourth capacity) to avoid excessive memory resource consumption while still meeting the RAID6 parity data storage requirements. By intelligently adjusting the capacity of the first buffer based on specific hardware configurations (RAID level and hard drive type), this method ensures the effective storage and processing of verification data, thereby optimizing disk I / O performance and improving data write efficiency and data reliability.

[0068] The following is a concrete example. Assume that in one implementation, the system pre-defines a first buffer capacity of 54KB for RAID 5 and 18KB for RAID 5 but not supporting TLC hard drives. When the target server's RAID level is identified as RAID 6 and its hard drive type is identified as supporting TLC, the system generates a first buffer of a specific size for that target server. This specific size, or third size, will be set to twice the capacity of the RAID 5 and TLC-supporting hard drive according to preset rules, i.e., 2 * 54KB = 108KB. In another example, when the target server's RAID level is also RAID 6, but its hard drive type is identified as not supporting TLC, the system generates a first buffer of another specific size. This specific size, or fourth size, will be set to twice the capacity of the RAID 5 and TLC-supporting hard drive according to preset rules, i.e., 2 * 18KB = 36KB. These capacities are designed to accommodate the dual-checking requirements of RAID 6 and are optimized based on the write characteristics of hard drives to ensure that check data can be efficiently and temporarily stored, thereby supporting the normal operation of the disk array.

[0069] Through the above technical solution, this application can intelligently adjust the capacity of the first buffer according to the target server's disk array level (especially RAID6) and hard drive type (whether it supports or does not support write-through TLC). This dynamic adjustment mechanism ensures that the buffer configured for RAID6 can fully meet its dual parity data storage requirements, avoiding data processing bottlenecks due to insufficient buffer capacity or resource waste due to excessive capacity. Specifically, for RAID6, by setting the first buffer capacity to twice that of RAID5, it can effectively cope with the more complex parity calculations and larger parity data volume of RAID6. At the same time, distinguishing between hard drive types that support and do not support write-through TLC makes the allocation of buffer capacity more refined, thereby optimizing disk I / O performance and data write efficiency under different hardware configurations, and improving the overall stability and reliability of the system.

[0070] In some embodiments described above in this application, a first buffer and a second buffer for temporarily storing verification data are proposed, based on the disk array level and hard drive type. However, after disk striping processing is complete, how to efficiently and reliably manage this temporarily stored verification data, especially regarding data persistence and resource release strategies for different RAID levels, is a problem that requires further consideration. Failure to handle this properly may lead to the risk of data loss or waste of storage resources.

[0071] In this regard, this application further proposes that, following S200, it also includes: In response to the disk array being RAID5 and disk striping being completed, the parity data of the disk stripes is written to the permanent flash memory; In response to the disk array being RAID6 and the disk striping process being completed, the parity data of the disk stripes is discarded.

[0072] In this embodiment, "the disk array level is RAID5" means that the target server's storage system is configured at the RAID5 level. This mode distributes data and parity information across multiple hard drives, allowing data recovery even if a single hard drive fails, thanks to the parity information. Its advantages include data redundancy and performance improvement, but the integrity of the parity data is crucial for data recovery. "Disk stripe processing complete" means that all data write operations, parity data calculations, and necessary synchronization operations related to the disk stripe have been successfully executed. The stripe's data is stably stored in the disk array, and its corresponding parity data is no longer in a "processing" state. This can be indicated by specific flags, state machine transitions, or completion callback functions within the system. "Parity data written to permanent flash memory" means storing the parity data in a non-volatile storage medium, such as NAND flash or NOR flash, which ensures data retention even in the event of a power outage. Writing the parity data to permanent flash memory ensures that the critical parity information of RAID5 is persistently preserved to cope with unexpected system restarts or power outages, thereby guaranteeing the reliability of data recovery. Write operations can be performed using a dedicated flash controller, or data can be stored on the flash device via a file system interface.

[0073] Furthermore, "the disk array is RAID 6" means that the target server's storage system is configured at RAID 6. This mode, through the use of dual distributed parity checking, can tolerate the simultaneous failure of two hard drives without data loss. Due to its higher redundancy, the handling strategy for parity data may differ from RAID 5. "Discarding parity data" means that after disk striping is complete, the system no longer needs the parity data temporarily stored in the buffer, so it is removed from the buffer, releasing the occupied memory resources. This can be achieved by marking the corresponding memory area as reusable, releasing the memory allocation, or directly overwriting the area. For RAID 6, due to its dual parity mechanism, after the data and parity blocks are written to the disk array, the parity data in the temporary buffer may have already completed its calculation and writing tasks, and no further persistent storage is needed to save capacity.

[0074] This application's solution, based on obtaining the target server's disk array level and hard drive type, and generating a first and second buffer for temporary storage of parity data, further optimizes the subsequent management mechanism of the parity data. When the system detects that the processing of a disk stripe has been completed, it adopts different strategies according to the pre-obtained disk array level. Specifically, if the disk array level is identified as RAID5, since RAID5 relies on only a single parity information for data recovery, the persistence of its parity data is crucial. Therefore, the system will retrieve the parity data of the processed disk stripe from the temporary buffer and reliably write it to permanent flash memory to ensure that even after an unexpected power outage or restart, this critical parity information can still be retained, thereby guaranteeing the data integrity and recoverability of the RAID5 array. Conversely, if the disk array level is identified as RAID6, since RAID6 uses a dual parity mechanism, its data redundancy is stronger, and usually, when data is written to the disk array, the parity data is also synchronously written to the corresponding disk blocks. In this case, the necessity for the parity data stored in the temporary buffer to continue to be retained after its calculation and writing tasks are completed is reduced. Therefore, the system will choose to discard these processed verification data, thereby releasing buffer resources in a timely manner, avoiding unnecessary memory occupation, and improving the utilization efficiency of system resources. Through this differentiated processing method, this solution can intelligently manage the lifecycle of verification data according to the characteristics of different RAID levels, optimizing the allocation and use of storage resources while ensuring data reliability.

[0075] The following is a concrete example. Assume a target server is processing a large number of data write requests. First, the system obtains the server's disk array level, such as RAID 5 or RAID 6, and its hard drive type. Based on this information, the system generates a first buffer and a second buffer for temporary storage of parity data. When the data of a disk stripe and its corresponding parity data are successfully written to the disk array, and all related write operations and parity calculations are confirmed to be complete, the system triggers subsequent parity data processing logic. As one specific implementation, if the current disk array level is RAID 5 and the processing of the disk stripe is complete, the system will start a background task or call a specific storage interface to read the parity data of the disk stripe from the first buffer and write it to a permanent flash memory module configured on the server. This permanent flash memory module can be a separate NVRAM card, ensuring that the parity data is not lost after power failure. As another specific implementation, if the current disk array level is RAID 6 and the processing of the disk stripe is complete, the system will perform a memory management operation, marking the memory space occupied by the parity data of the disk stripe stored in the first buffer as reclaimable or directly released. For example, the area can be marked as free by updating a memory allocation table, or the data in the area can be cleared to zero so that new parity data can be written to that space later. In this case, due to the redundancy of RAID6, it is usually not necessary to persist temporary parity data to permanent flash memory.

[0076] Through the above technical solutions, this application can adopt differentiated management strategies for the parity data of processed disk stripes according to different disk array levels. For RAID5, by writing the parity data to permanent flash memory, the durability and reliability of critical parity information are effectively improved, thereby enhancing the data recovery capability of the RAID5 array in the event of unexpected power outages or system failures. For RAID6, by promptly discarding processed parity data, unnecessary storage overhead is avoided, the utilization efficiency of buffer resources is optimized, and the resource consumption of system operation is reduced. This intelligent parity data management mechanism not only ensures data security but also improves the overall performance and resource management efficiency of the storage system.

[0077] In some embodiments described above, this application proposes a buffer generation method. This method first obtains the disk array level and hard disk type of the target server, and based on this, generates a first buffer for temporarily storing the checksum data of the disk stripes being processed, and a second buffer from the host memory for temporarily storing the checksum data of the unprocessed disk stripes. However, in actual operation, the data writing mode of the target server may change, for example, switching from one writing strategy to another. This mode switch may cause the original buffer data allocation and processing logic to become unsuitable for the new writing mode, thereby affecting the correct processing order and efficiency of the checksum data, and may even cause data inconsistency or processing delays.

[0078] In this regard, this application further proposes that, following S300, it also includes: In response to the target server switching the data write mode, the first check data in the first buffer is swapped into the second buffer, and the second check data in the second buffer is swapped into the first buffer. The first verification data is the verification data of the disk stripe that was first written to the first buffer and is not currently processed, and the second verification data is the verification data of the next disk stripe to be processed.

[0079] In this embodiment, the data write mode switching of the target server refers to a change in the way or strategy by which its underlying storage system or file system writes data to the hard drive. This may include switching from structured data write (tab write) to host single-level cell write (host SLC write), or from structured data write (tab write) to host extended-level cell write (host XLC write). This switching is typically manually configured by the system administrator, automatically adjusted by system load, or triggered by specific application requirements, and its purpose is to enable the system to adapt to different performance, reliability, or power consumption requirements.

[0080] When this mode switch is detected, the system swaps the first parity data in the first buffer into the second buffer, and vice versa. Swapping the first parity data from the first buffer into the second buffer means moving specific parity data stored in the first buffer to the second buffer. The first buffer temporarily stores parity data for disk stripes currently being processed, while the second buffer temporarily stores parity data for unprocessed disk stripes. When the data write mode switches, parity data that was previously waiting to be processed in the first buffer may require an adjustment in processing priority or method; therefore, it is moved to the second buffer for subsequent scheduling and processing according to the new mode. This "swap-in" operation can be implemented through resetting memory address pointers, physical copying of data blocks, or logical mapping. Correspondingly, swapping the second parity data from the second buffer into the first buffer means moving specific parity data stored in the second buffer to the first buffer. When the data write mode switches, some parity data that was originally waiting in the second buffer may need to be prioritized due to the new write mode, or its processing logic may be more compatible with the current function of the first buffer. Therefore, it is moved to the first buffer so that it can be recognized by the system as data that is "being processed" or "to be processed immediately". This "swap-in" operation can also be achieved by resetting the memory address pointer, physically copying the data block, or logically mapping it.

[0081] In this process, the first checksum data specifically refers to the checksum data of the disk stripe that was written earliest in chronological order but is not currently processed in the first buffer. It represents the "old" data in the current processing flow, and its processing priority or method may need to be re-evaluated after the mode switch. The second checksum data specifically refers to the checksum data of the disk stripe that, according to the system scheduling logic, will be preferentially selected for processing under normal circumstances in the second buffer. It represents data that may need to have its processing priority increased or its processing method more compatible with the new write mode after the mode switch.

[0082] This application's solution triggers a buffer data exchange mechanism when the target server's data write mode switches. Specifically, the first parity data of the earliest written and currently unprocessed disk stripe in the first buffer is exchanged with the second parity data of the next unprocessed disk stripe in the second buffer. The logic behind this exchange mechanism is that the first buffer typically carries currently ongoing or soon-to-be-completed parity data processing tasks, while the second buffer serves as a reserve pool of unprocessed parity data. When the data write mode changes, the original processing flow and priorities may no longer be applicable. For example, if switching from a mode with lower real-time requirements to a mode with higher real-time requirements, the data previously "unprocessed" in the second buffer may need to be immediately prioritized and put into a "processing" state. Conversely, the "old" data that has not yet been processed in the first buffer may need to be temporarily downgraded in processing priority, waiting for the new mode to stabilize or being re-queued in the second buffer. Through this parity data exchange, it is ensured that after a data write mode switch, the first buffer can immediately carry parity data with higher priority or greater urgency matching the new mode, while parity data no longer suitable for the current processing mode is moved to the second buffer for rescheduling. This allows the system to smoothly adapt to changes in write mode, avoiding data processing chaos, delays, or errors caused by mode switching, thereby ensuring the continuity, correctness, and efficiency of verification data processing.

[0083] The following is a concrete example. Suppose the target server is initially configured in a data write mode A, in which the system tends to aggregate parity data in a first buffer for a longer period before writing. At this time, the first buffer stores multiple written but unprocessed first parity data, while the second buffer stores a large amount of second parity data awaiting processing. When the target server switches to data write mode B for some reason (e.g., reduced system load or user-manual configuration), mode B requires parity data to be processed and written more quickly to improve response speed. In this case, the system responds to the data write mode switch. First, it identifies the earliest written and currently unprocessed first parity data of the disk stripe in the first buffer. Simultaneously, it determines the next unprocessed second parity data of the disk stripe in the second buffer. Subsequently, the system performs a swap operation: moving the first parity data from the first buffer to the second buffer, while moving the second parity data from the second buffer to the first buffer. For example, this can be achieved by updating memory pointers pointing to these parity data blocks without performing a large number of data copies, thus reducing operational overhead. In this way, the first buffer immediately obtains the second verification data that is more compatible with the processing logic of mode B and needs to be processed quickly, while the first verification data that originally required a longer aggregation time in the first buffer is moved to the second buffer and waits to be processed under the new scheduling strategy.

[0084] Through the above technical solution, when the data write mode of the target server switches, the system can respond promptly and dynamically adjust the allocation of verification data in the first and second buffers. This verification data exchange mechanism ensures that the first buffer always stores the verification data that best matches the current data write mode and requires priority processing, while verification data that is no longer suitable for the current processing mode is moved to the second buffer for rescheduling. This effectively solves the problem of potential confusion, reduced efficiency, or data inconsistency in the verification data processing flow when the data write mode switches. By ensuring that critical verification data is processed promptly in the correct buffer, this solution significantly improves the flexibility and adaptability of data processing, guarantees the continuity, correctness, and efficiency of verification data processing under different write modes, thereby improving the stability and performance of the entire storage system.

[0085] In some of the embodiments described above in this application, a method for generating a first buffer based on the disk array level and hard disk type is proposed. However, in actual operation, the parity data load of disk stripes may change dynamically over time. If the capacity of the first buffer remains constant, it may lead to low buffer resource utilization or buffer overflow during peak periods, thereby affecting data processing efficiency and system stability.

[0086] In this regard, this application further proposes that, following S200, it also includes: Obtain the parity data load of the disk stripes being processed by the target server; In response to the verification data load meeting the dynamic adjustment conditions, the buffer capacity of the first buffer of the target server is dynamically adjusted.

[0087] In this embodiment, the parity data load of the disk stripe being processed by the target server is acquired to monitor the amount of parity data generated by the disk stripe currently being processed by the target server in real time or periodically. The parity data load can be understood as the total amount or rate of parity data that needs to be stored or processed per unit time. Its purpose is to provide a basis for decision-making regarding subsequent dynamic adjustment of the buffer capacity. One implementation method is to set a counter on the data write path to count the number of parity data blocks or the total number of bytes entering the first buffer within a specific time window. Another implementation method is to monitor the current utilization rate of the first buffer, for example, by periodically checking the ratio of occupied space to total capacity, thereby indirectly reflecting the size of the parity data load.

[0088] In response to the verification data load meeting the dynamic adjustment conditions, the buffer capacity of the first buffer of the target server is dynamically adjusted. This describes determining whether the capacity of the first buffer needs adjustment based on the acquired verification data load and performing the corresponding adjustment operation. The dynamic adjustment conditions are rules or thresholds that trigger capacity adjustment, while capacity adjustment refers to increasing or decreasing the storage space of the first buffer. Its purpose is to match the buffer capacity with the actual load and optimize resource utilization. The dynamic adjustment conditions can be set as follows: when the verification data load exceeds a preset high threshold for N consecutive time periods, an expansion operation is triggered; when the verification data load is below a preset low threshold for M consecutive time periods, a reduction operation is triggered. Dynamic adjustment conditions can also be based on a predictive model, predicting future load trends based on historical load data and adjusting capacity in advance. One implementation method for dynamic capacity adjustment is to request or release memory pages from the system through the memory management interface provided by the operating system, thereby changing the actual size of the first buffer. Another implementation method is to pre-allocate multiple buffer blocks of different sizes and switch or combine these blocks according to adjustment needs to achieve dynamic capacity changes.

[0089] This application's solution, after generating a first buffer on the target server and using it to temporarily store the parity data of the disk stripes being processed, further introduces a real-time monitoring and dynamic response mechanism for the parity data load. Specifically, the system continuously acquires the parity data load of the disk stripes being processed by the target server, which reflects the current data writing activity and the rate of parity data generation. When the parity data load is detected to meet preset dynamic adjustment conditions—for example, a persistently high load indicating insufficient current buffer capacity, or a persistently low load indicating buffer redundancy—the system dynamically adjusts the capacity of the first buffer. In this way, the capacity of the first buffer is no longer fixed but can elastically scale according to actual parity data processing needs. This allows for more efficient use of buffer resources, avoiding data processing bottlenecks caused by insufficient capacity and resource waste caused by excess capacity. This dynamic adjustment mechanism, combined with a scheme that generates the initial buffer capacity based on disk array level and hard drive type, forms a more intelligent and adaptive buffer management strategy, significantly improving the system's performance and stability under different workloads.

[0090] As a specific implementation, during the operation of the target server, a background service process can be set up. This process periodically (e.g., every 5 seconds) retrieves the amount of checksum data generated by the disk stripes currently being written from the storage controller or file system interface. This checksum data amount can specifically be represented as the total size of checksum data written to the first buffer in the past 5 seconds. The system can preset two thresholds: a high-load threshold (e.g., buffer utilization exceeding 80%) and a low-load threshold (e.g., buffer utilization below 20%). When the buffer utilization is detected to exceed 80% three times consecutively, the system determines that the checksum data load meets the dynamic adjustment conditions for expansion. At this time, it can request additional memory pages from the operating system to increase the capacity of the first buffer by a preset increment. Conversely, when the buffer utilization is detected to be below 20% five times consecutively, the system determines that the checksum data load meets the dynamic adjustment conditions for shrinking. At this time, it can release some memory pages to reduce the capacity of the first buffer by a preset reduction, but ensures that it does not fall below the initially set minimum capacity. This dynamic adjustment process can be executed by a dedicated buffer management module, which can be a standalone software component or integrated into the storage driver.

[0091] Through the above technical solution, this application can dynamically adjust the capacity of the first buffer according to the actual verification data load of the target server. This effectively solves the problem of low resource utilization or performance bottlenecks that may occur in traditional fixed-capacity buffers when facing dynamically changing verification data loads. When the verification data load is high, the buffer capacity can be expanded in a timely manner to ensure that verification data can be written smoothly and avoid data loss or processing delays; when the verification data load is low, the buffer capacity can be reduced in a timely manner to release redundant memory resources for use by other system processes, thereby improving the overall resource utilization efficiency of the system. This adaptive buffer management mechanism significantly improves the flexibility and efficiency of data processing and enhances the stability and reliability of the system.

[0092] Based on the buffer generation method provided in this application, correspondingly, this application also provides specific embodiments of a buffer generation apparatus.

[0093] like Figure 2 The diagram shows a schematic of a buffer generation device. The buffer generation device 200 includes: The information acquisition module 210 is used to acquire the disk array level of the target server and the hard disk type of the target server; The buffer generation module 220 is used to generate a first buffer for the target server based on the disk array level and hard disk type; the first buffer is used to temporarily store the verification data of the disk stripe being processed by the target server. The buffer generation module 220 is also used to generate a second buffer for the target server from the host memory of the target server; the second buffer is used to temporarily store the verification data of the disk stripes that have not been processed by the target server.

[0094] In the buffer generation apparatus provided in this application embodiment, a first buffer is generated by obtaining the disk array level and hard disk type of the target server. Based on the server's own characteristics, a suitable buffer can be generated to temporarily store the verification data of the disk stripe being processed. This helps ensure the smoothness of the disk stripe writing process and improves the write speed. Simultaneously, a second buffer is generated from the host memory of the target server to temporarily store the verification data of the unprocessed disk stripe, utilizing host memory resources without incurring excessive additional hardware costs. Thus, this application effectively balances disk stripe write speed and hardware cost.

[0095] Based on the buffer generation method provided in this application, correspondingly, this application also provides specific embodiments of a buffer generation device.

[0096] Figure 3 A schematic diagram of the hardware structure of the buffer generation device provided in an embodiment of this application is shown.

[0097] The buffer generation device may include a processor 301 and a memory 302 storing computer program instructions.

[0098] Specifically, the processor 301 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0099] Memory 302 may include mass storage for data or instructions. For example, and not limitingly, memory 302 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be internal to an integrated gateway disaster recovery device. In a particular embodiment, memory 302 is non-volatile solid-state memory.

[0100] The processor 301 implements any of the buffer generation methods in the above embodiments by reading and executing computer program instructions stored in the memory 302.

[0101] In one example, the buffer generation device may further include a communication interface 303 and a bus 310. For example, Figure 3 As shown, the processor 301, memory 302, and communication interface 303 are connected through bus 310 and complete communication with each other.

[0102] The communication interface 303 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0103] Bus 310 includes hardware, software, or both, that couples components of a buffer generation device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 310 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0104] Furthermore, in conjunction with the buffer generation methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the buffer generation methods in the above embodiments.

[0105] In addition, in conjunction with the buffer generation method in the above embodiments, this application embodiment can provide a computer program product for implementation. When the instructions in the computer program product are executed by the processor of an electronic device, the electronic device executes the buffer generation method provided by any aspect of the above embodiments of this application.

[0106] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0107] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0108] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0109] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0110] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for generating a buffer, characterized in that, The method includes: Obtain the disk array level of the target server, as well as the hard disk type of the target server; Based on the disk array level and the hard disk type, a first buffer for the target server is generated; the first buffer is used to temporarily store the verification data of the disk stripe being processed by the target server; A second buffer for the target server is generated from the host memory of the target server; the second buffer is used to temporarily store the verification data of the disk stripes that have not been processed by the target server.

2. The method according to claim 1, characterized in that, The step of generating the first buffer for the target server based on the disk array level and the hard disk type includes: In response to the disk array level being RAID5 and the hard drive type being TLC that supports write-through, a first buffer of the target server with a first capacity is generated; In response to the disk array level being RAID5 and the hard drive type being TLC that does not support direct write, a first buffer of the target server with a second capacity is generated; the second capacity is less than the first capacity.

3. The method according to claim 2, characterized in that, In response to the RAID5 configuration including N disk array parity modes, the first capacity is set to N page sizes, and the second capacity is set to one page size; N is a positive integer.

4. The method according to claim 1, characterized in that, The step of generating the first buffer for the target server based on the disk array level and the hard disk type includes: In response to the disk array level being RAID6 and the hard drive type being TLC that supports write-through, a first buffer of the target server with a third capacity is generated; the third capacity is twice the first capacity, and the first capacity is the size of the first buffer of the target server when the disk array level is RAID5 and the hard drive type is TLC that supports write-through; In response to the disk array level being RAID6 and the hard drive type being TLC that does not support direct write, a first buffer of the target server with a fourth capacity is generated; the fourth capacity is twice the second capacity, and the second capacity is the size of the first buffer of the target server when the disk array level is RAID5 and the hard drive type is TLC that does not support direct write.

5. The method according to claim 1, characterized in that, After generating the first buffer for the target server based on the disk array level and the hard disk type, the method further includes: In response to the disk array level being RAID5 and the disk striping process being completed, the parity data of the disk stripe is written to the permanent flash memory; In response to the disk array level being RAID6 and the disk striping process being completed, the parity data of the disk stripe is discarded.

6. The method according to claim 1, characterized in that, After generating the second buffer of the target server from the host memory of the target server, the method further includes: In response to the target server switching the data writing mode, the first verification data in the first buffer is swapped into the second buffer, and the second verification data in the second buffer is swapped into the first buffer; The first verification data is the verification data of the disk stripe that was first written to the first buffer and is currently unprocessed, and the second verification data is the verification data of the next disk stripe to be processed.

7. The method according to claim 1, characterized in that, After generating the first buffer for the target server based on the disk array level and the hard disk type, the method further includes: Obtain the checksum data load of the disk stripe being processed by the target server; In response to the verification data load meeting the dynamic adjustment conditions, the buffer capacity of the first buffer of the target server is dynamically adjusted.

8. A buffer generation apparatus, characterized in that, The device includes: The information acquisition module is used to acquire the disk array level of the target server and the hard disk type of the target server; A buffer generation module is used to generate a first buffer for the target server based on the disk array level and the hard disk type; the first buffer is used to temporarily store the verification data of the disk stripe being processed by the target server; The buffer generation module is further configured to generate a second buffer for the target server from the host memory of the target server; the second buffer is used to temporarily store the verification data of the disk stripes that have not been processed by the target server.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the buffer generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the buffer generation method according to any one of claims 1 to 7.

Images