Disk array data writing method and device, electronic equipment and storage medium

Abstract

The application provides a disk array data writing method and device, electronic equipment and a storage medium, relates to the technical field of data storage, and through determining the full writing, large writing or small writing mode after receiving a host writing request, the new data and the old data of the target data block are acquired first for incremental calculation to generate the check incremental data in the small writing scene, the check incremental data is combined after the incremental calculation of the target data block is completed, and the updated triple check block is generated in combination with the old data, and finally, the target data block data and the triple check block are respectively persisted, so that the problems that the RAID 5 and the RAID 6 in the prior art need to read the old data and the old parity check when small writing, the writing amplification is caused, the check calculation is complex and low in efficiency, and it is difficult to tolerate multiple disk failures can be solved, and the technical effects of reducing the read-write overhead in the small writing scene, reducing the writing amplification effect, and improving the check calculation and data writing efficiency are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data storage, and particularly relates to a disk array data writing method and device, electronic equipment and a storage medium. BACKGROUND

[0002] As a core fault-tolerant mechanism of a storage system, RAID technology is widely used in enterprise-level data centers and key business systems. With the increase of single-disk capacity and the expansion of storage array size, the traditional RAID 5 and RAID 6 architecture faces significant limitations, such as insufficient data security protection in scenarios with extremely high reliability requirements. The traditional RAID architecture is difficult to support the dual requirements of performance and reliability of the storage system, especially in PB-level data application scenarios such as distributed storage and hyper-converged architecture, the performance bottleneck and the defect of insufficient fault tolerance are more prominent, and a new RAID architecture supporting triple verification is needed to be built to solve the performance problems caused by write amplification and the multi-disk fault tolerance problem. SUMMARY

[0003] The present application provides a disk array data writing method and device, electronic equipment and a storage medium to at least solve the problem of write amplification caused by the triple verification RAID architecture in the related art.

[0004] The present application provides a disk array data writing method, comprising:

[0005] receiving a write request of a host, determining a write mode of corresponding data based on the write request, and the write mode comprising full write, large write and small write;

[0006] if the write mode corresponding to the write request is small write, obtaining new data and old data of at least one target data block corresponding to the write request;

[0007] performing incremental calculation based on the new data and the old data, generating verification incremental data of the at least one target data block, and writing the data corresponding to the write request into the at least one target data block for persistent processing;

[0008] After the incremental calculation of the at least one target data block is completed, the verification incremental data of the at least one target data block is merged, and the updated triple verification block is generated based on the merging result and the old data;

[0009] persisting the triple verification blocks to physical disks respectively, so as to write the data corresponding to the write request into the physical disks in the disk array.

[0010] The present application also provides a disk array data writing device, comprising:

[0011] The determining unit is configured to receive a write request of a host, determine a write mode of corresponding data based on the write request, and the write mode comprises full write, large write and small write.

[0012] The obtaining unit is configured to, if the write mode corresponding to the write request is small write, obtain new data and old data of at least one target data block corresponding to the write request.

[0013] The first generating unit is configured to perform incremental calculation based on the new data and the old data, generate check incremental data of the at least one target data block, and write data corresponding to the write request into the at least one target data block for persistent processing.

[0014] The second generating unit is configured to merge the check incremental data of the block, and generate updated triple check sub-blocks based on the merging result and the old data.

[0015] The writing unit is configured to persist the triple check sub-blocks to physical disks respectively, so as to write data corresponding to the write request into the physical disks in the disk array.

[0016] The application further provides an electronic device, comprising a memory configured to store a computer program, and a processor configured to execute the computer program to implement the steps of any of the above methods.

[0017] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of any of the above methods.

[0018] The application further provides a computer program product comprising a computer program, and the computer program is executed by a processor to implement the steps of any of the above methods.

[0019] The application provides a disk array data write method and device, an electronic device and a storage medium. The write mode of full write, large write or small write is determined after receiving the write request of the host. The new data and the old data of the target data block are obtained for incremental calculation to generate check incremental data in the small write scenario. The check incremental data is merged after the incremental calculation of the target data block is completed, and the updated triple check sub-blocks are generated in combination with the old data. Finally, the target data block data and the triple check sub-blocks are persisted respectively. Therefore, the problems of write amplification, complex and low-efficiency check calculation, and difficulty in tolerating multiple disk failures of the existing RAID 5 and RAID 6 in small write can be solved. The technical effects of reducing read-write overhead in small write scenario, reducing write amplification effect, improving check calculation and data write efficiency, guaranteeing data integrity in multiple disk failure through triple check, and improving storage system performance and reliability are achieved.

[0020] It is to be understood that the details set forth herein do not limit the key or critical features of the present disclosure. Other features of the present disclosure will be apparent from the following specification, and the claims. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings are used to better understand the present application, and do not limit the present disclosure. Among them:

[0022] Figure 1 A flowchart of a disk array data writing method provided by an embodiment of the present application is shown in the figure.

[0023] Figure 2 A structural diagram of a disk array data writing device provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0024] The exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help the understanding, and should be considered as merely exemplary. Therefore, those skilled in the art should realize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, in order to be clear and concise, the description below omits the description of well-known functions and structures.

[0025] In order to make those skilled in the art better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] In combination with the specific application environment architecture or specific hardware architecture on which the disk array data writing method depends, the specific application environment architecture or specific hardware architecture is described herein.

[0027] The embodiments of the present application provide a disk array data writing method, Figure 1 A flowchart of a disk array data writing method provided by an embodiment of the present application is shown in the figure.

[0028] As Figure 1 shown, the method comprises the following steps:

[0029] Step 101, receiving a write request of a host, determining a write mode of corresponding data based on the write request, the write mode including full write, large write and small write.

[0030] In the embodiment of the present application, during the operation of the RAID-TP storage system, a write request from the host is first received, which contains information related to the data to be written by the host, such as data volume, target storage location identifier, etc. After receiving the write request, the data corresponding to the write request needs to be analyzed to determine the adaptive data write mode, wherein the write mode specifically includes three types of full write, large write and small write.

[0031] Here, "full write" refers to the data involved in the host write request being able to completely cover all data blocks in a stripe in the storage system, without the need to additionally read the existing old data in the stripe to complete the data write related processing; "large write" is that after analyzing the data corresponding to the host write request, the data range does not completely cover all data blocks in a stripe, but by reading the old data in the stripe that is not covered, the complete data that can cover the entire stripe can be combined with the data to be written by the host, and then the processing is completed according to the logic similar to full write; "small write" is that after analyzing the data corresponding to the host write request, the data range only involves part of the data blocks in a stripe, and the write mode does not need to be completed by reading the old data in the non-covered part to combine into complete stripe data.

[0032] By first receiving the host write request and determining the corresponding write mode, the storage system can subsequently use the adaptive process to process data write, avoiding the problem of low processing efficiency caused by using a unified write process, laying a foundation for subsequent efficient completion of data write and verification processing, and helping to improve the overall write performance and adaptability of the storage system.

[0033] Step 102, if the write mode corresponding to the write request is small write, the new data and old data of at least one target data block corresponding to the write request are obtained.

[0034] In the embodiment of the present application, after determining that the write mode corresponding to the host write request is small write, the storage system will further locate at least one target data block associated with the write request, which is a specific data storage unit in the storage system that needs to be updated by the data of this write request. Subsequently, the storage system needs to obtain the new data and old data of the target data block respectively: wherein the new data refers to the latest data of the target data block to be written by the host through this write request, which will be used to overwrite the original data content in the target data block; the old data refers to the original data stored in the target data block before receiving this write request, which is an important basis for subsequent data update and verification calculation.

[0035] When obtaining new data, the storage system receives write data issued by the host from the upper module, and performs preliminary processing on the write data according to a preset data storage format, to ensure that the new data conforms to the storage specification of the target data block; when obtaining old data, the storage system initiates a data reading request to the corresponding storage module according to the physical storage location information of the target data block, and retrieves the historical storage data of the target data block from the storage medium, to complete the obtaining of the old data.

[0036] By accurately obtaining the new data and the old data of the target data block, necessary data support is provided for subsequent data increment calculation and verification update in the small write scenario, calculation errors caused by incomplete data acquisition are avoided, and a foundation is laid for subsequent reduction of write overhead and optimization of write performance, which helps to improve the data processing accuracy and efficiency of the storage system in the small write scenario.

[0037] In step 103, an increment calculation is performed based on the new data and the old data, to generate verification increment data of at least one target data block, and the data corresponding to the write request is written into at least one target data block for persistent processing.

[0038] In the embodiment of the present application, after obtaining the new data and the old data of at least one target data block in the small write scenario, the storage system will perform an increment calculation operation based on the two types of data to generate the verification increment data corresponding to the target data block. The core of the increment calculation is to perform logical operation on the new data and the old data to obtain the difference data between the two, which is the verification increment data. The verification increment data is a key basis for subsequent update of the verification chunk, and compared with directly recalculating the verification value of the complete data, the calculation amount can be greatly reduced.

[0039] After the generation of the verification increment data is completed, the storage system will simultaneously write the new data corresponding to the write request into at least one target data block, and start a persistent processing procedure. Persistent processing refers to an operation of stably storing the new data in the target data block to the storage medium, to ensure that the written new data will not be lost even in the case of subsequent temporary failure of the storage system, and to guarantee the stability of data storage.

[0040] By generating the verification increment data through increment calculation, the redundant operation of repeated operation on complete data in traditional verification calculation is effectively avoided, and the consumption of calculation resources is reduced; at the same time, the new data is written into the target data block in time and persistent processing is completed, to ensure the immediacy and safety of the data. This process lays a foundation for efficient update of the verification chunk in the future, helps to improve the overall processing efficiency of the storage system in the small write scenario, and reduces unnecessary resource overhead.

[0041] Step 104, after the incremental calculation of the at least one target data block is completed, the check incremental data of the at least one target data block is merged, and an updated triple check block is generated based on the merging result and the old data.

[0042] In the embodiments of the present application, after the incremental calculation of the at least one target data block is completed and the corresponding check incremental data is generated, the storage system will start the merging operation of the check incremental data. The merging operation here refers to integrating the check incremental data generated by the plurality of target data blocks according to the preset logical rule, eliminating the redundant information between the plurality of check incremental data, and forming a unified merging result. The merging result can collectively reflect the overall impact of the data update of all target data blocks on the check block, avoiding repeated calculation and resource waste caused by processing each group of check incremental data separately.

[0043] After obtaining the merging result of the check incremental data, the storage system will call the preset check calculation logic to cooperatively operate the merging result and the old check data stored in the storage medium, and generate an updated triple check block by updating the old check data. The triple check block contains three independent check information, which can provide higher level data fault tolerance protection for the storage system, and can further improve the security of the data in the multi-disk failure scenario compared with the traditional double check architecture.

[0044] By merging the check incremental data of the plurality of target data blocks, the data processing amount in the check calculation process is greatly reduced, and the computing resource overhead is reduced. At the same time, based on the merging result and the old data, the triple check block is generated, which realizes efficient update of the check block under the premise of ensuring the accuracy of the check, and lays a foundation for subsequent data persistence and improvement of the fault tolerance of the storage system.

[0045] Step 105, persisting the triple check block to the physical disk respectively, so as to write the data corresponding to the write request into the physical disk in the disk array.

[0046] In the embodiment of the present application, after generating the updated triple-checking chunk, the storage system will start the persistence process of the triple-checking chunk, and write the three independent checking chunks into the corresponding physical disks in the disk array. Here, "persistence" means that the checking information of the triple-checking chunk is stably stored in the non-volatile storage area of the physical disk, ensuring that the checking information will not be lost due to temporary power failure, device fluctuation, etc. of the storage system, and guaranteeing the long-term reliability of the checking data. In the specific execution process, the storage system will determine the physical disk location corresponding to each group of checking chunks according to the hardware configuration and chunk storage rules of the disk array, initiate a write instruction to the target physical disk through the underlying storage interface, write the checking chunk data into the disk storage unit in the preset format, and monitor the data transmission state and disk response during the writing process to ensure that each group of checking chunks can be completely and accurately stored in the corresponding physical disk.

[0047] When the triple-checking chunk is completely persisted, the data corresponding to the host write request is completely written into the physical disk of the disk array in combination with the new data of the target data block that has completed the persistence. This process completes the persistence of the triple-checking chunk separately and sequentially, which not only guarantees the storage consistency of the checking data and the target data block data, but also relies on the independent storage characteristics of the triple-checking chunk to strengthen the fault tolerance capability of the disk array in response to multiple disk failures. The beneficial effects are that it avoids the problem of invalid data checking caused by untimely or incomplete storage of the checking chunk, and at the same time, through the solidification of the checking data at the physical disk level, it provides bottom protection for the data security of the storage system, and further improves the reliability and stability of the overall data storage.

[0048] The present application provides a disk array data write method, which determines the full write, large write or small write mode after receiving the host write request, generates checking incremental data by first obtaining the new data and old data of the target data block for incremental calculation in the small write scenario, merges the checking incremental data after the target data block completes the incremental calculation, and generates the updated triple-checking chunk in combination with the old data. Finally, the target data block data and the triple-checking chunk are respectively persisted, so that the problems of write amplification, complex and low-efficiency checking calculation caused by reading old data and old parity check in RAID 5 and RAID 6 in small write, and difficulty in tolerating multiple disk failures in the prior art can be solved, achieving the technical effects of reducing read-write overhead in small write scenario, reducing write amplification effect, improving checking calculation and data write efficiency, and at the same time, guaranteeing data integrity in multiple disk failures through triple-checking, improving the performance and reliability of the storage system.

[0049] Under the technical solution framework disclosed in the foregoing embodiments, in order to achieve better resource allocation for data writing, the embodiments of the present application further provide an implementation of resource allocation, comprising: dynamically allocating computing and caching resources for a stripe corresponding to a write request before obtaining new data and old data of at least one target data block corresponding to the write request; creating a stripe lock and synchronizing state information of nodes in a failure domain through a lock communication mechanism.

[0050] Further, in the embodiments of the present application, the dynamic allocation of computing and caching resources for the stripe corresponding to the write request is embodied as: estimating and reserving volatile memory pages for data temporary storage according to the total number of data blocks and check blocks in a stripe write unit; allocating a plurality of input / output control blocks for each data block and each check block; reserving memory resources for creating data block control structures and check block control structures.

[0051] Specifically, under the technical solution framework disclosed in the foregoing embodiments, in order to achieve efficient resource allocation for data writing, the embodiments of the present application further optimize the resource management link for the data writing process in a small writing scenario. Specifically, before obtaining new data and old data of at least one target data block corresponding to a write request, dynamic allocation of computing and caching resources for the stripe associated with the write request needs to be completed, and a stripe lock is created at the same time, and the state information of nodes in a failure domain is synchronized through a lock communication mechanism. Wherein, the stripe is the basic unit of data division in a storage system, which is composed of multiple data blocks and check blocks, and the blocks in the same stripe are distributed on different physical disks; the stripe lock is used to avoid data conflicts caused by multiple write requests operating on the same stripe at the same time, and the lock communication mechanism can ensure that all nodes in the failure domain are aware of the current occupancy and processing state of the stripe in real time, thereby ensuring the consistency of resource allocation.

[0052] When dynamically allocating computing and caching resources for the stripe, specific operations need to be performed in combination with the actual data size and processing demand of the stripe: first, estimate and reserve volatile memory pages according to the total number of data blocks and check blocks in the stripe write unit, the memory pages are used for temporarily storing new data, old data of the data blocks and intermediate data generated in the subsequent calculation process, to ensure that there is no storage resource shortage in the data processing process; second, allocate a plurality of input / output control blocks (IOB) for each data block and each check block, the input / output control block is a key component that connects data and hardware interface, and is used to manage the transmission logic of data between memory and physical disk; finally, reserve special memory resources for creating data block control structures (such as IPK, responsible for managing IO operations of a single data block, containing offset, length and other information of data read / write) and check block control structures (such as PSIO, used for managing the calculation and disk writing process of the check block), to provide structural support for efficient processing of subsequent data blocks and check blocks.

[0053] Through the above resource allocation manner, the accurate reservation and synchronization of resources can be completed before data writing, processing delay caused by insufficient resources or unsynchronized node states is avoided, and the fluency and stability of data writing in the small write scenario are effectively improved, and a reliable resource foundation is laid for subsequent new data and old data acquisition, verification and incremental calculation.

[0054] Under the technical solution framework disclosed in step 102, the technical solution in step 102 is embodied as: saving the new data corresponding to the write request to the input / output buffer indicated by the block input / output management structure; for the old data that needs to be read, asynchronously reading the disk data through the virtualization layer module and saving it to the input / output buffer indicated by the stripe cache management structure.

[0055] Specifically, under the technical solution framework of step 102 determining that the write mode is small write, and the new data and old data of at least one target data block need to be acquired, the specific implementation process needs to rely on specific data management structures and modules in the storage system to complete data storage and reading. For the new data corresponding to the write request, it needs to be saved to the input / output buffer (IOB) indicated by the block input / output management structure (IPK, which is used to manage the IO operation of a single data block and contains key information such as data read / write offset and length), through the accurate positioning of the data storage location by IPK, it is ensured that the new data can be temporarily stored according to the preset path, and preparation is made for subsequent data processing and persistence.

[0056] As for the old data that needs to be read, since it is stored in the physical disk, it needs to initiate an asynchronous read operation with the help of the virtualization layer (VL) module - asynchronous reading can avoid blocking other processes due to waiting for disk data to return, and improve the overall processing efficiency. The virtualization layer module is responsible for coordinating the interaction logic between the storage system and the physical disk, ensuring that the data read request can be accurately issued to the corresponding disk. The read old data will finally be saved to the input / output buffer indicated by the stripe cache management structure (SDE, which is used to manage the read cache and other information of the stripe, and the stripe is the basic unit of data division, containing multiple blocks distributed in different disks). Through the management of the stripe-level cache by SDE, the centralized temporary storage and rapid calling of the old data are realized, and data support is provided for the subsequent comparison and incremental calculation with the new data.

[0057] Through the above specific implementation, the ordered storage and efficient reading of new data and old data can be realized, avoiding the influence of data storage disorder or reading delay on subsequent processes, and relying on the synergistic effect of IPK, SDE and the virtualization layer module to ensure the accuracy and timeliness of data acquisition, laying a reliable data foundation for the incremental calculation link in step 102, and further improving the efficiency and stability of data processing in the small write scenario.

[0058] Under the technical solution framework disclosed in step 103, the technical solution of performing incremental calculation in step 103 is further specified as follows: performing exclusive-OR operation on the new data and the old data of the at least one target data block to obtain the check incremental data; and saving the check incremental data in the management structure of the first check chunk, which has a mapping relationship with the management structures of the second check chunk and the third check chunk.

[0059] Specifically, under the technical solution framework of performing incremental calculation based on the new data and the old data to generate the check incremental data in step 103, the specific implementation of the incremental calculation needs to be realized through the cooperation of specific operation logic and data management structure. Specifically, for the new data (i.e., the latest data to be written by the host) and the old data (i.e., the original stored data to be updated in the target data block) of the at least one target data block that has been obtained, exclusive-OR operation is used to complete the incremental calculation operation. Exclusive-OR operation can efficiently capture the difference information between the new data and the old data, and the result obtained through this operation is the check incremental data. This data can accurately reflect the impact of the updated target data block on the check chunk, and compared with traditional full-check calculation, it greatly reduces the calculation complexity and resource consumption.

[0060] After obtaining the check incremental data, it needs to be stored in the management structure of the first check chunk, wherein the management structures of the first check chunk, the second check chunk, and the third check chunk have a mapping relationship established in advance. The check chunk management structure (such as the IPK structure corresponding to the check chunk, which is used to manage the IO operation and data storage information of the check chunk) is the core of ensuring the ordered management of the check data. Through the mapping relationship among the three, the management structures of the second check chunk and the third check chunk do not need to store the check incremental data separately, but can obtain the data from the management structure of the first check chunk through mapping association, avoiding the repeated storage of the check incremental data, saving the memory resources, and at the same time ensuring that the subsequent triple check chunk update can be calculated based on the unified check incremental data, and guaranteeing the consistency of the check data.

[0061] Through the above specific implementation of the incremental calculation, not only the efficient generation of the check incremental data is realized by means of exclusive-OR operation, but also the data storage logic is optimized by relying on the mapping relationship among the check chunk management structures, effectively reducing the calculation and storage resource overhead, providing accurate and efficient data support for subsequent check incremental data merging and triple check chunk update, and further improving the processing efficiency of the storage system in the small write scenario.

[0062] Under the technical solution framework disclosed in step 104, the merging of the check delta data of the at least one target data block in step 104 is embodied as follows: an exclusive or calculation structure is created, and the check delta data of the at least one target data block is sequentially merged; the check chunk management structure generated by the first target data block is taken as a reference structure, and the check chunk management structures generated by the subsequent target data blocks are taken as subsequent structures to participate in the merging calculation.

[0063] Specifically, under the technical solution framework of merging the check delta data of the at least one target data block in step 104, the specific implementation of the merging operation needs to rely on the cooperation of a specific calculation structure and a chunk management structure. First, an exclusive or calculation structure (i.e., XOR structure, used to uniformly manage the merging calculation logic of the check delta data and record the data source information and intermediate results in the calculation process) needs to be created. Through this structure, the check delta data generated by each target data block is sequentially included in the merging process in a predetermined order, ensuring that each group of check delta data can be accurately identified and processed, avoiding the problem of data omission or repeated calculation in the merging process.

[0064] In the selection of the structure of the merging calculation, the check chunk management structure generated by the first target data block is taken as a reference structure (i.e., baseIpk, which stores the check delta data corresponding to the first target data block and related IO management information, providing an initial data reference for subsequent merging calculations), and the check chunk management structures generated by the subsequent target data blocks are taken as subsequent structures (i.e., NextIpk) to gradually participate in the merging calculation. In the specific merging process, the exclusive or calculation structure will first take the check delta data in the reference structure as the basis, and then integrate the check delta data in the subsequent structure into the calculation result of the reference structure through exclusive or operation. Through this progressive merging method, a unified merging result reflecting the data update influence of all target data blocks is gradually formed, ensuring that the merging result can comprehensively cover the check delta information of all target data blocks, providing complete and accurate data basis for generating the updated triple check chunk based on the merging result and the old data.

[0065] Through the above merging method, the ordered integration of the check delta data is realized by means of the exclusive or calculation structure, and the integrity and accuracy of the merging result are guaranteed by relying on the progressive merging logic of the reference structure and the subsequent structure, avoiding the calculation redundancy caused by the dispersed processing of multiple groups of check delta data, effectively reducing the resource consumption in the merging process, laying a reliable foundation for the efficient generation of the subsequent triple check chunk, and further improving the efficiency of the check data processing in the small write scenario.

[0066] Under the technical solution framework disclosed in step 104, generating the updated triple-checking chunk based on the merging result and the old data in step 104 is embodied as: calculating the old check data and the merged check delta data by exclusive-OR operation to obtain new check data; saving the new check data in the stripe cache management structure, and updating the context state of the acceleration processing unit to a check data ready state.

[0067] Specifically, under the technical solution framework of generating the updated triple-checking chunk based on the merging result and the old data in step 104, the specific implementation process needs to be completed through the cooperation of specific operation logic and data management structure. First, the old check data (i.e., the historical check data before the update of the triple-checking chunk, including the old data corresponding to the P, Q, and R triple-checking chunks) stored in the storage system is subjected to exclusive-OR operation with the check delta data obtained by the previous merging. The exclusive-OR operation can efficiently utilize the logical relationship between the two and quickly derive new check data reflecting the check state after the data update. The new check data is the core content of the updated triple-checking chunk, and compared with the traditional full-recomputing check data mode, the calculation amount and time consumption are greatly reduced.

[0068] After obtaining the new check data, it needs to be saved in the stripe cache management structure (i.e., SDE, which is used to manage the read cache and other information of the stripe, and can realize the temporary storage and quick calling of the new check data, and prepare for the subsequent persistent operation). At the same time, the context state (i.e., ApuContext, which is used to record the running state, check block size, and other key information of the APU) of the acceleration processing unit (i.e., APU, which is a hardware unit for improving data calculation and processing efficiency) needs to be updated to a check data ready state (such as setting ApuContext->events = Pr|Qr|Rr). This state updating operation can real-time feedback the preparation of the new check data, ensure that the subsequent triple-checking chunk persistent process can be started in time, and avoid process blocking caused by different state information.

[0069] Through the above specific implementation, the efficient generation of new check data is realized by means of exclusive-OR operation, the reliable storage of new check data is guaranteed by relying on the stripe cache management structure, and the smoothness of process connection is ensured by updating the context state of the acceleration processing unit, which lays an accurate and timely data and state foundation for the subsequent persistence of the triple-checking chunk, and further improves the efficiency and stability of the triple-checking chunk update in the lowercase scenario.

[0070] Under the technical solution framework disclosed in the foregoing embodiments, the embodiments of the present application are further embodied as: in response to the completion of the incremental calculation of the target data block, performing data landing processing on the target data block.

[0071] Specifically, under the technical solution framework disclosed in the foregoing embodiments, the subsequent processing operation of the target data block is further refined for the data processing flow in the small write scenario. Specifically, after the target data block completes the incremental calculation (that is, the process of generating the check incremental data based on the exclusive OR operation of the new data and the old data), the storage system will immediately perform data landing processing on the target data block. The "data landing processing" here refers to the operation of writing the new data (that is, the data to be updated transmitted by the host through the write request) obtained in the target data block from the temporary storage area (such as the input / output buffer) to the physical disk, ensuring that the new data can be stably stored in the physical storage medium of the disk array, avoiding data loss due to temporary system failure or resource fluctuation.

[0072] When performing the data landing processing, the storage system will rely on the preset block input / output management structure (such as IPK, used to manage the IO operation of a single data block, including the offset, length and other key information of data read / write) to explicitly determine the physical disk location and storage format corresponding to the target data block, and initiate a write request to the corresponding physical disk through the underlying storage interface. At the same time, the system will monitor the data transmission state in the write process in real time, including data integrity check, disk response feedback, etc., to ensure that the new data can be completely and accurately written to the physical disk. It is worth noting that this data landing processing does not need to wait for the completion of the subsequent check incremental data merging and triple check block update flow, and can be independently and parallelly executed, thereby reducing the waiting delay of the overall data write.

[0073] By immediately performing the data landing processing after the target data block completes the incremental calculation, the physical storage solidification of the new data can be quickly realized, the safety and timeliness of data storage are improved, the parallel processing logic is used to reduce process blocking, the overall write efficiency of the storage system in the small write scenario is further optimized, sufficient time is reserved for the promotion of subsequent check-related processes, and the smoothness of the data write whole process is ensured.

[0074] Under the technical solution framework disclosed in the foregoing embodiments, if the write mode determined in step 101 is full write, the full write operation can be performed using but not limited to the following mode: based on the write request, data of all data blocks in the stripe is obtained; after the data of all data blocks is obtained, the first check block, the second check block and the third check block are calculated and generated based on the data cached by the data blocks; after the calculation of the check blocks is initiated, the landing operation of all data blocks is performed; and after the calculation of the check blocks is completed, each check block is respectively persisted to the corresponding physical disk.

[0075] Further, in the embodiments of the present application, the first, second and third check sub-blocks are calculated based on the data cached in the data blocks, which specifically includes: memory resources and acceleration processing unit resources are respectively applied for the first, second and third check sub-blocks; and the first, second and third check sub-blocks are sequentially calculated and the calculation results are temporarily stored in the respective associated memory pages.

[0076] Further, in the embodiments of the present application, each check sub-block is respectively persisted to the corresponding physical disk, which includes: after all the check sub-blocks are calculated, the write disk operation of each check sub-block is sequentially initiated; after the write disk of each check sub-block is successful, the state of the check sub-block in the stripe acceleration processing unit context is updated, and the memory and acceleration processing unit resources occupied by the check sub-block are released.

[0077] Specifically, under the technical solution framework disclosed in the foregoing embodiments, when it is determined in step 101 that the write mode is full write, the data write operation needs to be performed according to the process adapted to the full write scenario. Specifically, first, based on the write request of the host, the stripe corresponding to the request is located, and the to-be-written data of all data blocks in the stripe is obtained - here, the stripe is a basic storage unit in the storage system that contains multiple data blocks and check blocks, and in the full write scenario, all data blocks in the stripe need to be overwritten, so it is necessary to ensure that the to-be-written data of all data blocks is completely obtained. After the data of all data blocks is obtained, instead of relying on the old data and old check blocks, the first, second and third check sub-blocks are calculated based on the to-be-written data temporarily stored in the data block cache, so as to build a triple check mechanism and provide higher level fault tolerance protection for data. After initiating the check sub-block calculation process, without waiting for the completion of the calculation of the check sub-blocks, the landing operation of all data blocks can be synchronously performed, that is, the data in the data block cache is written to the corresponding physical disk through the underlying storage interface, so as to realize the rapid storage of the data blocks; after the calculation of the check sub-blocks is completed, the first, second and third check sub-blocks are respectively persisted to the corresponding physical disks in the disk array, and the complete data write in the full write scenario is completed.

[0078] When the triple check sub-blocks are calculated based on the data cached in the data blocks, independent memory resources and acceleration processing unit (APU) resources need to be respectively applied for the first, second and third check sub-blocks - the memory resources are used to temporarily store the calculation results and related context information of the check sub-blocks, and the acceleration processing unit resources are used to improve the efficiency of the check calculation, so as to avoid the influence of the calculation delay on the overall write performance. After the resource application is completed, the first, second and third check sub-blocks are sequentially calculated according to a preset order, and after the calculation of each check sub-block is completed, the calculation result of the check sub-block is temporarily stored in the memory page associated with the check sub-block, so as to ensure that the calculation results of the check sub-blocks are independently stored and do not interfere with each other, and to provide accurate data basis for subsequent persistence operation.

[0079] In the process of persisting each check chunk to the corresponding physical disk, after all check chunks are calculated, write disk operation is initiated to the physical disk corresponding to each check chunk in a preset order to ensure that the write disk process proceeds in order. When a single check chunk is successfully written, the state of the check chunk in the corresponding ApuContext (a structure used to record the running state of APUs in a stripe and check block information) needs to be updated in time, for example, updating the state to "write disk complete (Pw / Qw / Rw)", and releasing the memory resources and acceleration processing unit resources previously occupied by the check chunk to avoid resource waste caused by long-term occupation and to ensure efficient recycling of storage system resources.

[0080] Through the design of the above full-write process, on the one hand, the parallel execution of data block landing and check chunk calculation reduces the overall write delay; on the other hand, through independent resource allocation and orderly state management, the accuracy of triple check chunk calculation and storage is ensured, which not only improves the write efficiency in full-write scenarios, but also relies on triple check to strengthen data reliability, effectively adapting to storage scenarios with high requirements for write performance and data security.

[0081] Under the technical solution framework disclosed in the foregoing embodiments, if the write mode determined in step 101 is large write, the embodiments of the present application can also use but are not limited to the following methods for full-write operation: combining the old data on the member disk and the data of the host write request to form a full stripe; and multiplexing the full-write process to land the data chunk and the check chunk in parallel.

[0082] Specifically, under the technical solution framework disclosed in the foregoing embodiments, when the write mode determined in step 101 is large write, the data write operation needs to be performed according to a specific process adapted to the large write scenario. Specifically, the core feature of the large write scenario is that the data range corresponding to the host write request does not completely cover all data blocks in the stripe, so first the old data corresponding to the data blocks not covered by the host write request needs to be read through the underlying interaction logic of the storage system - these old data are stored in the member disk (i.e. the physical disk constituting the RAID array) of the disk array, and the data transmission link needs to be established between the virtualization layer module and the member disk to ensure that the old data can be completely and accurately transferred from the physical disk to the system cache area.

[0083] After the old data reading of the member disk is completed, the old data is combined with the new data delivered by the host write request according to the data block distribution rule of the strip to form full-strip data that can completely cover the entire strip. Here, the "full strip" refers to the data amount that matches the storage capacity of all data blocks in the strip, which meets the basic condition for subsequent full-write process. The combination process needs to map the old data and the new data to different data blocks in the strip according to the strip block mapping relationship, so as to ensure that the data distribution logic in the strip is consistent with the full-write scenario.

[0084] After the full-strip data is constructed, the data processing flow in the previous full-write scenario is directly reused, that is, based on the combined full-strip data, the first, second and third check blocks are first calculated and generated, and then the parallel disk writing operation of the data blocks and the check blocks is started. Parallel disk writing refers to the synchronous execution of the writing of the data blocks (including the combined old data and new data) to the corresponding member disk and the writing of the check blocks to the specified physical disk. It is not necessary to wait for the writing of one type of block to be completed before starting the writing of another type of block. Through parallel processing, the overall writing time is reduced. At the same time, relying on the mature resource scheduling and state management logic in the full-write process, the accuracy and stability of the disk writing of the data blocks and the check blocks are ensured.

[0085] Through the design of the above full-write process, it is not necessary to develop a completely new processing logic for the full-write scenario. Only by reading the old data and combining the full-strip, the full-write process is reused, which not only reduces the system design complexity, but also continues the high performance advantage of the full-write scenario by means of parallel disk writing, and at the same time, relying on the three check blocks, the data reliability is guaranteed, which effectively adapts to the storage scenario where the host write data does not cover the full strip but has a higher requirement for the write efficiency.

[0086] It should be noted that the embodiments of the present disclosure can include a plurality of steps, which are numbered for the convenience of description, but these numbers do not limit the execution time slots and execution order between the steps; these steps can be implemented in any order, and the embodiments of the present disclosure do not limit this.

[0087] Corresponding to the above-mentioned disk array data writing method, the present disclosure also proposes a disk array data writing device. Since the device embodiment of the present disclosure corresponds to the above-mentioned method embodiment, for the details not disclosed in the device embodiment, please refer to the above-mentioned method embodiment, which will not be described in detail in the present disclosure.

[0088] Figure 2 A structural schematic diagram of a disk array data writing device provided by an embodiment of the present disclosure is shown in Figure 2 as shown, comprising:

[0089] A determination unit 21 is configured to receive a write request of a host, determine a write mode of corresponding data based on the write request, and the write mode includes full-write, full-write and small-write. A determination unit 21 is configured to receive a write request of a host, determine a write mode of corresponding data based on the write request, and the write mode includes full-write, full-write and small-write.

[0090] The acquisition unit 22 is configured to acquire new data and old data of at least one target data block corresponding to the write request if the write mode corresponding to the write request is lowercase;

[0091] The first generation unit 23 is configured to perform incremental calculation based on the new data and the old data, generate check incremental data of the at least one target data block, and write data corresponding to the write request into the at least one target data block for persistent processing;

[0092] The second generation unit 24 is configured to merge the check incremental data of the block, and generate updated triple check sub-blocks based on the merging result and the old data;

[0093] The write unit 25 is configured to persist the triple check sub-blocks to physical disks respectively, so as to write data corresponding to the write request into the physical disks in the disk array.

[0094] It should be noted that the foregoing explanation and description of the method embodiments are also applicable to the device of the present embodiment, and the principles are the same, which will not be limited herein.

[0095] The features of the embodiments corresponding to the disk array data writing device can be referred to the related description of the embodiments corresponding to the disk array data writing method, which will not be repeated here.

[0096] The embodiments of the present application also provide an electronic device including a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above-mentioned disk array data writing method embodiments.

[0097] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, wherein the computer program is configured to perform the steps in any of the above-mentioned disk array data writing method embodiments when running.

[0098] In an exemplary embodiment, the above-mentioned computer readable storage medium can include but is not limited to: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0099] The embodiments of the present application also provide a computer program product, which includes a computer program, and the computer program is executed by a processor to perform the steps in any of the above-mentioned disk array data writing method embodiments.

[0100] The embodiment of the present application further provides another computer program product, comprising a nonvolatile computer readable storage medium, the nonvolatile computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the steps in any of the above-mentioned disk array data writing method embodiments.

[0101] Those skilled in the art will further appreciate that the functions implemented by the elements and algorithms described herein can be implemented in electronic hardware, computer software, or any combination thereof. To clearly illustrate this interchangeability of hardware and software, various examples can be described throughout this specification in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0102] The above describes in detail a disk array data writing method and device, an electronic device and a storage medium provided by the present application. The principles and implementation modes of the present application are described by applying specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that, for those skilled in the art, some improvements and modifications can be made to the present application without departing from the principles of the present application. These improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A method for writing data to a disk array, characterized in that, include: Receive a write request from the host, and determine the write method of the corresponding data based on the write request. The write method includes full write, uppercase, and lowercase. If the write mode corresponding to the write request is lowercase, obtain the new data and old data of at least one target data block corresponding to the write request; Incremental calculations are performed based on the new data and the old data to generate verification incremental data for at least one target data block, and the data corresponding to the write request is written into the at least one target data block for persistence processing; wherein, the step of generating verification incremental data for at least one target data block based on the new data and the old data includes: performing an XOR operation on the new data and the old data of the at least one target data block to obtain the verification incremental data; storing the verification incremental data in the management structure of the first verification block, wherein the management structure of the first verification block is mapped to the management structures of the second verification block and the third verification block; After the incremental calculation of the at least one target data block is completed, the verification incremental data of the at least one target data block is merged, and an updated triple verification block is generated based on the merging result and the old data; wherein, merging the verification incremental data of the at least one target data block includes: Create an XOR calculation structure to sequentially merge the verification increment data of the at least one target data block; The verification block management structure generated by the first target data block is used as the baseline structure, and the verification block management structures generated by subsequent target data blocks are used as subsequent structures to participate in the merging calculation. The method for generating updated triple-checked blocks based on the merged results and old data includes: The new check data is obtained by calculating the old check data and the merged check increment data through an XOR operation; The new verification data is stored in the stripe cache management structure, and the context state of the accelerated processing unit is updated to the verification data ready state. The triple check blocks are persisted to physical disks respectively, so that the data corresponding to the write request is written to the physical disk in the disk array.

2. The disk array data writing method according to claim 1, characterized in that, Before obtaining the new and old data for at least one target data block corresponding to the write request, the process also includes: Dynamically allocate computation and cache resources for the stripe corresponding to the write request; Create a striped lock and synchronize the status information of each node in the fault domain through the lock communication mechanism.

3. The disk array data writing method according to claim 2, characterized in that, The step of dynamically allocating computation and caching resources for the stripe corresponding to the write request includes: Based on the total number of data blocks and check blocks in the stripe writing unit, estimate and reserve volatile memory pages for temporary data storage; Assign multiple input / output control blocks to each data block and each check block; Memory resources are reserved for creating data block control structures and check block control structures.

4. The disk array data writing method according to claim 1, characterized in that, The step of obtaining the new data and old data corresponding to at least one target data block of the write request includes: Save the new data corresponding to the write request to the input / output buffer indicated by the block input / output management structure; For older data that needs to be read, the disk data is read asynchronously through the virtualization layer module and saved to the input / output buffer indicated by the stripe cache management structure.

5. The disk array data writing method according to claim 1, characterized in that, Also includes: In response to the completion of incremental calculation of the target data block, data write-to-disk processing is performed on the target data block.

6. The disk array data writing method according to claim 1, characterized in that, When the writing mode is full write, it also includes: Based on the write request, data is obtained for all data blocks within the stripe; Once all data blocks have been retrieved, the first verification block, the second verification block, and the third verification block are calculated and generated based on the data cached in the data blocks. After initiating the verification block calculation, execute the write operation for all data blocks to disk; After the checksum calculation is completed, each checksum block is persisted to its corresponding physical disk.

7. The disk array data writing method according to claim 6, characterized in that, The calculation of the first check block, the second check block, and the third check block based on the data block cache includes: Memory resources and acceleration processing unit resources are allocated for the first verification block, the second verification block and the third verification block, respectively; The first check block, the second check block, and the third check block are calculated sequentially, and the calculation results are temporarily stored in their respective associated memory pages.

8. The disk array data writing method according to claim 7, characterized in that, The step of persisting each check block to its corresponding physical disk includes: After all check blocks have been calculated, write operations for each check block are initiated sequentially. After each parity block is successfully written to disk, its state in the context of the stripe acceleration processing unit is updated, and the memory and acceleration processing unit resources it occupies are released.

9. The disk array data writing method according to claim 8, characterized in that, When the writing mode is uppercase, it also includes: A full stripe is formed by combining old data read from the member disk with data from the host write request. The full write process is reused to write data blocks and verification blocks to disk in parallel.

10. A disk array data writing device, characterized in that, include: A determining unit is configured to receive a write request from the host and determine the write method of the corresponding data based on the write request, wherein the write method includes full write, uppercase, and lowercase. The acquisition unit is used to acquire new data and old data of at least one target data block corresponding to the write request if the write mode corresponding to the write request is lowercase. The first generation unit is used to perform incremental calculations based on the new data and the old data to generate verification incremental data for the at least one target data block, and to write the data corresponding to the write request into the at least one target data block for persistent processing. The second generation unit is used to merge the check increment data of the block and generate an updated triple check block based on the merge result and the old data. The write unit is used to persist the triple check blocks to the physical disks respectively, so as to write the data corresponding to the write request to the physical disks in the disk array.

11. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the disk array data writing method according to any one of claims 1-9.

12. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the disk array data writing method according to any one of claims 1-9.

Images