A data read-write method, a data read-write device and a computer storage medium

Abstract

This application proposes a data read / write method, a data read / write device, and a computer storage medium. The method includes: the data read / write device responding to a data read / write command to determine the region type of a group of regions to be written; when the region type is an SMR region, determining the buffer bound to the group of regions to be written; writing target data to the group of regions to be written in a striped manner, and determining whether the current write scenario is a full striped write; if not, writing the checksum of the target data to the buffer; continuing to write the target data until the write pointers of all storage regions within the region group are aligned, and writing the checksum of the target data to the checksum area. By identifying the region type and dynamically binding the buffer, an optimal write path is constructed. Write requests to SMR regions are automatically directed to a buffer with random write capability, fundamentally avoiding the write penalty problem caused by SMR directly facing random writes.

Description

Technical Field

[0001] This application relates to the field of storage technology, and in particular to a data reading and writing method, a data reading and writing device, and a computer storage medium. Background Technology

[0002] SMR (Shingled Magnetic Recording) hard drives utilize a technology that overlaps the tracks within the drive to increase track density, thereby increasing capacity without increasing hardware costs. However, this track overlap results in sequential writes only, while random reads are possible. To mitigate this, the hard drive is typically divided into different zones where tracks overlap, but tracks do not overlap between zones.

[0003] SMR hard drives are divided into Host-Manage SMR and Device-Manage SMR. Host-Manage SMR only allows writing to the LBA address specified by the write pointer; any write that does not match the write pointer will fail, greatly limiting the compatibility and flexibility of the storage system. Device-Manage SMR triggers time-consuming loops and garbage collection processes, resulting in a sharp drop in random write performance and significant latency fluctuations. Summary of the Invention

[0004] To address the aforementioned technical problems, this application proposes a data reading and writing method, a data reading and writing device, and a computer storage medium.

[0005] To address the aforementioned technical problems, this application proposes a data read / write method, comprising: responding to a data read / write instruction, determining the region type of the region group to be written; when the region type is an SMR region, determining the buffer bound to the region group to be written; writing target data into the region group to be written in a striped manner, and determining whether the current write scenario is a full striped write; if not, writing the verification data of the target data into the buffer; continuing to write the target data until the write pointers of all storage regions within the region group are aligned, and writing the verification data of the target data into the verification area.

[0006] The data read / write method further includes: when the current write scenario is full stripe write, the verification data of the target data is not written to the cache area, but the verification data of the target data is written to the verification area.

[0007] The data read / write method further includes: responding to a device malfunction command by reading verification data from the cache bound to all region groups; and checking and restoring the data of all region groups based on the verification data.

[0008] The step of determining the cache bound to the area group to be written includes: reading the binding relationship of the area group to be written in the RAID metadata area of ​​the CMR area; and determining the cache bound to the area group to be written based on the binding relationship.

[0009] The step of determining the buffer bound to the area group to be written includes: reading the binding relationship of the area group to be written in the RAID metadata area of ​​the CMR area; if the reading fails, obtaining all buffers in the CMR area; determining an idle buffer from all buffers and binding it to the area group to be written, and writing the binding relationship into the RAID metadata area.

[0010] The data read / write method further includes, after writing the verification data of the target data into the verification area, determining whether the group of areas to be written is full; if so, removing the binding relationship between the group of areas to be written and the cache area, and marking the cache area as an idle cache area.

[0011] The region group consists of storage regions or sub-region groups on different hard drives.

[0012] The region group is mapped in the RAID logical address space to the RAID physical address space of the CMR region.

[0013] To address the aforementioned technical problems, this application proposes a data read / write device, which includes a memory and a processor coupled to the memory; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the aforementioned data read / write method.

[0014] To address the aforementioned technical problems, this application proposes a computer storage medium for storing program data, which, when executed by a computer, is used to implement the aforementioned data read / write method.

[0015] Compared with existing technologies, the beneficial effects of this application are as follows: The data read / write device responds to data read / write commands by determining the region type of the region group to be written; when the region type is an SMR region, it determines the buffer bound to the region group to be written; it writes the target data to the region group in a striped manner, and determines whether the current write scenario is a full-strip write; if not, it writes the verification data of the target data to the buffer; it continues writing the target data until the write pointers of all storage regions within the region group are aligned, and then writes the verification data of the target data to the verification area. By identifying the region type and dynamically binding the buffer, an optimal write path is constructed, and write requests to SMR regions are automatically directed to a buffer with random write capabilities, fundamentally avoiding the write penalty problem caused by SMR media directly facing random writes. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0017] Figure 1 This is a flowchart illustrating the first embodiment of the data read / write method provided in this application; Figure 2 This is a schematic diagram of the overall process of the data reading and writing method provided in this application; Figure 3 This is a schematic diagram of an embodiment of the SMR hard disk structure provided in this application; Figure 4 This is a schematic diagram of the regional group mapping provided in this application; Figure 5 This is a schematic diagram of an embodiment of the data read / write process provided in this application; Figure 6 This is a schematic diagram of an embodiment of the disk structure located in the SMR region provided in this application; Figure 7 This is a schematic diagram of another embodiment of the data read / write process provided in this application; Figure 8 This is a schematic diagram of an embodiment of the data read / write device provided in this application; Figure 9 This is a schematic diagram of the structure of an embodiment of the computer storage medium provided in this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0019] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] Please refer to the details. Figures 1-2 , Figure 1 This is a flowchart illustrating the first embodiment of the data read / write method provided in this application. Figure 2 This is a schematic diagram of the overall process of the data reading and writing method provided in this application.

[0021] The data read / write method of this application is applied to a data read / write device, which can be a server, a terminal device, or a system in which the server and the terminal device cooperate with each other. Accordingly, the various parts of the data read / write device, such as each unit, subunit, module, and submodule, can all be set in the server, all in the terminal device, or separately in the server and the terminal device.

[0022] Furthermore, the aforementioned server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules, such as software or software modules used to provide distributed server functionality, or as a single software program or software module; no specific limitations are made here.

[0023] Step S11: In response to the data read / write command, determine the region type of the region group to be written.

[0024] SMR (Shingled Magnetic Recording) hard drives are a type of high-capacity mechanical hard drive. Their core technology lies in arranging the tracks on the disk platter in a partially overlapping manner, like roof tiles, which significantly increases the data track density, thereby achieving a larger storage capacity than traditional CMR hard drives in the same area.

[0025] Each SMR hard drive platter is not entirely constructed using shingled magnetic recording technology. Instead, it is divided into two functionally distinct areas: the CMR area and the SMR area. The CMR area uses traditional magnetic recording technology, allowing for direct, random data overwriting of tracks. The SMR area, on the other hand, occupies the majority of the hard drive's capacity, achieving extremely high storage density by tightly overlapping tracks.

[0026] This application provides a method for setting up region groups, where each region group consists of storage regions or sub-region groups on different hard drives. Please refer to [link / reference] for details. Figure 3 , Figure 3 This is a schematic diagram of an embodiment of the SMR hard disk structure provided in this application.

[0027] The CMR (Concurrent Range) data area and SMR (Self-Range Range) area on each hard drive are divided into different regions, named Zone0 to ZoneN, based on the SMR zone. Each zone on each hard drive forms a logical zone group ZG (ZoneGroup), named ZG0 to ZGN. ZG is the zone group.

[0028] A zone group ZG is composed of a zone on each hard drive. In this embodiment, these zones are not required to be in the same location. For example, ZG0 can also consist of Zone0 on SMR DISK0, Zone1 on SMR DISK1, and Zone2 on SMR DISK2. The zones within a zone group ZG generally originate from the same region, such as a CMR or SMR region. In some specific embodiments, the zones within a zone group ZG may also originate from different regions.

[0029] Alternatively, each zone group (ZG) can also consist of multiple contiguous zones on each hard drive, for example, […]. Figure 2 ZG0 and ZG1 are merged into a new ZG0, and ZG2 and ZG3 are merged into a new ZG1.

[0030] The grouping of each ZG, including which Zones it comprises, is recorded as part of the RAID metadata in the RAID metadata area of ​​the CMR region.

[0031] like Figure 2As shown, the data read / write device can determine the region of region group ZG by judging whether region group ZG is randomly writable.

[0032] Furthermore, in this embodiment of the application, the RAID logical address space of the region group is mapped to the RAID physical address space of the CMR region.

[0033] Please see details. Figure 4 , Figure 4 This is a schematic diagram of the region group mapping provided in this application. The mapping mechanism of the region group ZG from the RAID logical address space to the CMR region physical address space essentially constructs an efficient and stable write channel.

[0034] This method divides the continuous logical block addresses provided by the RAID controller into management units based on fixed-capacity area groups, with each area group statically bound to a pre-reserved continuous physical space on the CMR disk.

[0035] Specifically, the ZG (zone group) with random write capability can be arbitrarily distributed in the logical address space of RAID as needed, but they will eventually be mapped to the ZG in the CMR (Cross-Regional Mapping) area in the physical address space.

[0036] In both the logical and physical spaces of RAID, the index of the region group ZG is counted starting from 0.

[0037] The metadata area of ​​a file system needs to have the property of random write, but the metadata area of ​​a file system may be distributed in any area of ​​the RAID.

[0038] Once the RAID is created, the file system can configure which addresses in the RAID logical space need to have random attributes, according to its own needs. The RAID program will map the region group (ZG) corresponding to this logical address to a region group (ZG) within the CMR region on the physical address, thus enabling this logical address to have random write characteristics.

[0039] Once the file system specifies random write characteristics, a formatting operation can be performed to write the file system metadata into these areas and update them randomly as needed.

[0040] Other address spaces that are not specifically designated are generally considered to be the spaces where data files are stored. The region group ZG in this space may come from the SMR region or the CMR region.

[0041] This mapping relationship between logical space and physical space is also recorded in the metadata area of ​​each hard drive.

[0042] In this way, the file system can directly mark the metadata areas requiring random write capabilities within the RAID logical space. The RAID system then uses an underlying mapping mechanism to precisely direct these logical addresses to physical area groups composed of CMR hard drives, thereby providing true random write capability for the file system metadata. Ordinary data areas are automatically mapped to SMR areas to fully utilize their large capacity. This intelligent mapping relationship is persistently recorded in the hard drive metadata area, forming a dynamic resource allocation scheme that satisfies both file system performance requirements and storage economy.

[0043] Therefore, this application can integrate SMR and CMR two heterogeneous media into a unified, efficient and cost-controllable storage pool, which is a very promising technical path to solve the application problems of SMR hard drives in enterprise-level environments.

[0044] Step S12: When the region type is SMR region, determine the cache area bound to the region group to be written.

[0045] In this embodiment, the data read / write device reads the binding relationship of the region group to be written in the RAID metadata area of ​​the CMR region. Based on the binding relationship, the cache area bound to the region group to be written is determined.

[0046] Specifically, when the data read / write device performs a write operation on the SMR area, it first accesses the RAID metadata area located in the CMR storage area, reads the pre-established area mapping table from it, and determines the dedicated CMR cache area corresponding to the SMR area group to be written by querying the table.

[0047] By using predefined binding relationships, the overhead of dynamically searching the cache on each write is avoided, providing a deterministic data path and ensuring high performance. Storing this critical binding information in a non-volatile and stable CMR area ensures that the mapping relationship is not lost even after system failure and recovery, guaranteeing data accessibility.

[0048] In another embodiment of this application, the data read / write device reads the binding relationship of the group of regions to be written in the RAID metadata area of ​​the CMR area. If the read fails, it obtains all the buffers in the CMR area; determines the free buffer from all the buffers and binds it to the group of regions to be written, and writes the binding relationship into the RAID metadata area.

[0049] Specifically, when the data read / write device determines the corresponding buffer for the SMR area group, it first attempts to read the existing binding relationship from the RAID metadata area of ​​the CMR area. If the read is successful, the existing mapping is used directly; if the read fails, the automatic recovery procedure is started.

[0050] The data read / write device scans all available buffer resources in the CMR area, selects a suitable free buffer through a space management algorithm, establishes a new binding relationship between it and the current SMR area group, and persists this newly established mapping relationship to the RAID metadata area. This solves the current write requirement and repairs the metadata record to ensure the normal operation of subsequent operations.

[0051] Step S13: Write the target data into the group of regions to be written in a striped manner, and determine whether the current writing scenario is a full stripe write.

[0052] like Figure 5 As shown, Figure 5 This is a schematic diagram of an embodiment of the data read / write process provided in this application. Reading and writing are performed within region group ZG in a striped manner. If a read / write process only writes the data D2, this write is not a full-strip write.

[0053] If it is not a full stripe write, then proceed to step S14.

[0054] Step S14: Write the verification data of the target data into the cache area.

[0055] like Figure 6 As shown, Figure 6 This is a schematic diagram of an embodiment of the disk structure located in the SMR region provided in this application. Figure 6 In this context, CacheZone is the cache area, and ParityZone is the parity area. CacheZone and ParityZone can both originate from the same physical disk.

[0056] When a stripe is not fully written, the parity data P1' needs to be written to the cache zone first. At this time, the write pointer positions in Zone 0 and Zone 1 will be different. Each write requires updating the parity data. If the zone containing the parity data can only be written sequentially, it is obviously not possible to update it multiple times. In this case, a CMR area cache, i.e., Cache Zone, needs to be prepared for this zone.

[0057] The cache zone is only allocated to those region groups (ZGs) that are performing write operations. The number of cache zones varies depending on the size of each hard disk cache zone, and there can be multiple cache zones on each disk to facilitate scenarios where multiple region groups (ZGs) are writing concurrently.

[0058] In this embodiment, when encountering fragmented data writes that cannot fill the entire stripe, to avoid performance loss caused by immediately updating the parity area, the new parity information corresponding to these fragmented data is temporarily stored in the cache. This ensures the fast completion of the write operation and transforms the original random small writes to the parity area into subsequent batch sequential writes, which is particularly suitable for storage media such as SMR that are friendly to sequential writes, thereby significantly improving system write performance while ensuring data consistency.

[0059] Furthermore, after writing the verification data of the target data into the verification area, the data read / write device determines whether the group of areas to be written is full; if so, it removes the binding relationship between the group of areas to be written and the cache area, and marks the cache area as an idle cache area.

[0060] After the data read / write device completes writing the target data verification information to the verification area, it will simultaneously check the storage status of the currently processed SMR area group. If it is confirmed that the area group is completely full, the resource release process will be triggered immediately.

[0061] The resource release process is as follows: First, the binding relationship between the region group and the currently occupied cache area is released, and then the status of the cache area is marked as free and available.

[0062] This mechanism enables dynamic recycling of cache resources, ensuring that SMR regions that are already full of data no longer occupy valuable CMR cache resources, while also releasing available cache space in a timely manner for subsequent new SMR region write requests, thereby maintaining the efficient operation of the entire storage system's cache allocation system.

[0063] Step S15: Continue writing the target data until the write pointers of all storage areas within the region group are aligned, and write the verification data of the target data into the verification area.

[0064] like Figure 7 As shown, Figure 7 This is a schematic diagram of another embodiment of the data read / write process provided in this application.

[0065] If you continue writing data D3, you will get a new parity data P1. After D3 is written, the write pointers of Zone0 and Zone1 will be consistent. At this point, write the parity data P1 to ParityZone, and the write pointers of Zone0, Zone1, and ParityZone will be consistent.

[0066] When writing a non-full stripe, the data read / write device does not immediately update the parity area. Instead, it temporarily stores the parity data in the buffer while continuously receiving subsequent write requests. As more data is written, the write pointers of each storage area gradually advance. When the write pointers of all associated storage areas have moved to the beginning of the next stripe, it indicates that a complete stripe of data has been accumulated. Only then will the data read / write device write the final correct parity data to the parity area all at once.

[0067] By aggregating multiple scattered writes into batch operations in the above manner, the write efficiency of the storage system is significantly improved.

[0068] In this embodiment, when the written data is insufficient to fill a complete RAID stripe (i.e., a non-full stripe write), the data read / write device directly writes the user data to its corresponding data storage area, while temporarily storing the verification information corresponding to this write operation in a cache. For each subsequent sporadic data write, the user data is written to the corresponding position in real time, and only the corresponding verification information is continuously accumulated and updated in the cache. Until the current stripe is full and the write positions of all hard drives are aligned to the starting point of the next new stripe, the complete stripe verification information accumulated in the cache is written to the verification area in a batch at once.

[0069] This method aggregates verification information without accumulating or aggregating user data itself, thus avoiding the performance loss of traditional partial write verification updates.

[0070] Furthermore, in this embodiment of the application, when the current writing scenario is full stripe writing, the verification data of the target data is not written to the cache area, but the verification data of the target data is written to the verification area.

[0071] If the file system writes D2 and D3 simultaneously in a single read / write operation, which is a full stripe write, then the parity data P1 does not need to be written to the cache zone; it can be written directly to the parity zone.

[0072] Furthermore, this application also proposes an embodiment for checking and recovery. Specifically, in response to a device malfunction command, the data read / write device reads the verification data in the cache bound to all region groups; and checks and recovers the data of all region groups based on the verification data.

[0073] The binding relationship between the CacheZone and the Zone Group (ZG) is recorded in the RAID metadata area. In the event of an abnormal situation such as a power outage, if the parity data is not written to the ParityZone, the parity data can be read through the bound CacheZone, ensuring that the data is not lost.

[0074] Upon receiving a device anomaly command, such as hard drive failure or unexpected power outage, the emergency recovery process is immediately initiated. First, it reads the verification data temporarily stored in the CMR cache for all area groups. This verification data contains critical verification information from recent write operations. Then, based on this verification data and data blocks stored on normal hard drives, the RAID verification algorithm checks and reconstructs the data integrity of all area groups. This ensures rapid restoration of data consistency in the event of anomalies and effectively prevents data loss or corruption caused by non-persistent cached data.

[0075] Please refer to the details. Figure 8 , Figure 8 This is a schematic diagram of an embodiment of the data read / write device provided in this application.

[0076] The data read / write device 700 of this embodiment includes a processor 71, a memory 72, an input / output device 73, and a bus 74.

[0077] The processor 71, memory 72, and input / output device 73 are respectively connected to the bus 74. The memory 72 stores program data, and the processor 71 is used to execute the program data to implement the data read / write method described in the above embodiments.

[0078] In this embodiment, processor 71 can also be referred to as a CPU (Central Processing Unit). Processor 71 may be an integrated circuit chip with signal processing capabilities. Processor 71 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor, or processor 71 can be any conventional processor.

[0079] This application also provides a computer storage medium; please refer to the following: Figure 9 , Figure 9 This is a schematic diagram of a computer storage medium according to an embodiment of the present application. The computer storage medium 600 stores a computer program 61, which, when executed by a processor, is used to implement the data read / write method of the above embodiment.

[0080] When the embodiments of this application are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0081] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A data read / write method, characterized in that, The data read / write method includes: In response to data read / write commands, determine the region type of the region group to be written; When the region type is SMR region, determine the cache area bound to the region group to be written; The target data is written to the group of regions to be written in a striped manner, and it is determined whether the current writing scenario is a full stripe write. If not, write the verification data of the target data into the cache area; Continue writing the target data until the write pointers of all storage areas within the region group are aligned, and then write the verification data of the target data into the verification area.

2. The data read / write method according to claim 1, characterized in that, The data read / write method further includes: When the current write scenario is a full stripe write, the verification data of the target data is not written to the cache area, but the verification data of the target data is written to the verification area.

3. The data read / write method according to claim 1, characterized in that, The data read / write method further includes: In response to a device malfunction command, read the verification data from the cache bound to all region groups; The data of all the regional groups are checked and restored based on the verification data.

4. The data read / write method according to claim 1, characterized in that, Determining the cache area bound to the group of regions to be written includes: Read the binding relationship of the group of regions to be written in the RAID metadata area of ​​the CMR area; The cache area bound to the region group to be written is determined based on the binding relationship.

5. The data read / write method according to claim 4, characterized in that, Determining the cache area bound to the group of regions to be written includes: Read the binding relationship of the area group to be written in the RAID metadata area of ​​the CMR area. If the read fails, obtain all the buffers in the CMR area. A free buffer is determined from all the buffers and bound to the group of areas to be written, and the binding relationship is written to the RAID metadata area.

6. The data read / write method according to claim 4 or 5, characterized in that, After writing the verification data of the target data into the verification area, the data read / write method further includes: Determine whether the group of regions to be written is full; If so, unbind the group of regions to be written from the cache area and mark the cache area as an idle cache area.

7. The data read / write method according to claim 1, characterized in that, The region group consists of storage regions or sub-region groups on different hard drives.

8. The data read / write method according to claim 1 or 7, characterized in that, The region group is mapped in the RAID logical address space to the RAID physical address space of the CMR region.

9. A data read / write device, characterized in that, The data read / write device includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the data read / write method as described in any one of claims 1 to 8.

10. A computer storage medium, characterized in that, The computer storage medium is used to store program data, which, when executed by the computer, is used to implement the data read / write method as described in any one of claims 1 to 8.

Images