A bare disk-oriented data storage method, system, device, medium and product

Abstract

The application discloses a kind of bare disk-oriented data storage method, system, equipment, medium and product, by responding to write request, obtain request data, the request data is cached in target logical sector group in logical continuous mode;The target logical sector group includes at least one logical sector;According to the covering relationship of the request data and the logical sector, the request data is divided, and the sub-request list of each logical sector is obtained;According to the mapping relationship between the logical sector and the physical sector, the request data is stored to bare disk in append write mode.Using the embodiment of the application, the request data of logical sector is divided based on the covering relationship using high-performance append write mode, to provide the access semantics of file internal data continuity for upper layer application.

Description

Technical Field

[0001] This invention relates to the field of bare disk storage technology, and in particular to a data storage method, system, device, medium and product for bare disks. Background Technology

[0002] The core of raw disk data storage lies in directly taking over block storage devices through the storage engine, without relying on a file system. This architecture can be deeply optimized and customized for specific application scenarios and workloads. It also allows for customization of the physical layout and storage format of data on the raw disk, and precise control over the allocation logic of storage resources, thereby improving the efficiency of raw disk space utilization.

[0003] However, existing raw disk-based storage engines have strict constraints on data writing methods and storage formats in order to adapt to the block storage characteristics of raw disks and ensure read and write performance. For example, the write address must meet a certain alignment method, which makes it impossible for them to provide upper-layer applications with the ability to access the continuous semantics within the file. Summary of the Invention

[0004] The embodiments of the present invention aim to provide a data storage method, system, device, medium and product for bare disks, which can utilize high-performance append write mode and combine request data of logical sectors based on overlay relationship to provide upper layer applications with continuous access semantics for data within the file.

[0005] In a first aspect, embodiments of the present invention provide a data storage method for bare disks, comprising:

[0006] In response to a write request, request data is obtained and cached in a target logical sector group in a logically contiguous manner; the target logical sector group includes at least one logical sector.

[0007] Based on the coverage relationship between the request data and the logical sector, the request data is segmented to obtain a sub-request list for each logical sector;

[0008] Based on the mapping relationship between the logical sectors and the physical sectors, the requested data is stored to the raw disk in an append-only manner.

[0009] As an improvement to the above scheme, the logical sector includes a client data area and a verification data area; the client data area is used to cache the request data; the verification data area is used to cache the verification data generated when writing data to disk.

[0010] As an improvement to the above solution, the step of responding to a write request, obtaining the requested data, and caching the requested data in a logically contiguous manner in the target logical sector group includes:

[0011] In response to at least one write request, establish an ordered queue;

[0012] According to the order of the write requests, the request data of the write requests are inserted into the ordered queue;

[0013] Based on the ordered queue, all the requested data are cached in the target logical sector group in a logically contiguous manner.

[0014] As an improvement to the above scheme, the step of establishing an ordered queue in response to at least one write request includes:

[0015] In response to at least one write request, an initial ordered queue is established based on the number of write requests;

[0016] If a new write request is received within the preset waiting time, the initial ordered queue is updated and the preset waiting time is reset.

[0017] Otherwise, an ordered queue is obtained based on the initial ordered queue and its update status.

[0018] As an improvement to the above solution, the step of segmenting the request data according to the coverage relationship between the request data and the logical sector to obtain a sub-request list for each logical sector includes:

[0019] Obtain the logical sector where the current request data is located, and obtain the first logical sector and the second logical sector based on the request data in the logical sector; the first logical sector includes request data other than the current request data, and the second logical sector only includes the current request data, but it is not completely covered;

[0020] Based on the boundary between the first logical sector and the second logical sector, the current request data is divided into at least one current sub-request to obtain the sub-requests corresponding to all request data;

[0021] Based on the sub-requests in the logical sector, a list of sub-requests for each logical sector is obtained.

[0022] As an improvement to the above scheme, the step of dividing the current request data into at least one current sub-request based on the boundary between the first logical sector and the second logical sector to obtain the sub-requests corresponding to all request data includes:

[0023] If the current request data falls within the boundary of the first logical sector or the boundary of the second logical sector, the current request data is segmented at the boundary of the first logical sector and the second logical sector to obtain at least one current sub-request.

[0024] Otherwise, the current request data will be used as the current sub-request;

[0025] After splitting all request data, we get the sub-requests corresponding to all request data.

[0026] As an improvement to the above scheme, the step of obtaining a list of sub-requests for each logical sector based on the sub-requests in the logical sector includes:

[0027] Based on the alignment of the boundary of the sub-request with the boundary of any of the logical sectors, the attributes of the sub-request are set as header, middle, and tail.

[0028] Based on the attributes, establish a mapping relationship between sub-requests and logical sectors; wherein, middle sub-requests are mapped to at least one logical sector, and head sub-requests and tail sub-requests are mapped to one logical sector;

[0029] Based on the mapping relationship, a sub-request list for each logical sector is obtained.

[0030] As an improvement to the above scheme, the step of setting the attributes of the sub-request as header, middle, and tail based on the alignment of the sub-request's boundary with the boundary of any of the logical sectors includes:

[0031] If the front and rear boundaries of the sub-request are aligned with the boundaries of the logical sector, then the attribute of the sub-request is set to the middle.

[0032] If the sub-request's front boundary is aligned with the boundary of any logical sector, then the sub-request's attribute is set to tail.

[0033] Otherwise, set the attribute of the sub-request as a header.

[0034] As an improvement to the above solution, establishing the mapping relationship between sub-requests and logical sectors based on the attributes includes:

[0035] If the sub-request is a middle sub-request, then obtain at least one third logical sector covered by the middle sub-request and map the sub-request to at least one third logical sector;

[0036] If the sub-request is not a middle sub-request, then all sub-requests in the fourth logical sector where the sub-request is located are connected to form a connected sub-request, and the connected sub-request is mapped to the fourth logical sector.

[0037] As an improvement to the above solution, the step of storing the requested data to the raw disk in an append-only manner according to the mapping relationship between the logical sectors and physical sectors includes:

[0038] Based on the sub-request list of the logical sector, generate verification data and add the verification data to the verification data area of ​​the logical sector;

[0039] Based on the mapping relationship between the logical sectors and physical sectors, the data in the logical sectors is copied to the corresponding physical sectors until all requested data is stored on the raw disk in append mode.

[0040] As an improvement to the above solution, the step of storing the requested data to the raw disk in an append-only manner according to the mapping relationship between the logical sectors and physical sectors includes:

[0041] If the list of sub-requests in a logical sector completely covers the logical sector, then after the list of sub-requests is written to the raw disk in an append-only manner according to the mapping relationship between the logical sector and the physical sector, the logical sector is released.

[0042] As an improvement to the above scheme, if the sub-request list in the logical sector completely covers the logical sector, then after writing the sub-request list to the raw disk in append mode according to the mapping relationship between the logical sector and the physical sector, the logical sector is released, including:

[0043] If the sub-request list of a logical sector contains only one sub-request and the logical sector completely covers it, filter at least one third logical sector corresponding to the sub-request.

[0044] Based on the mapping relationship between the at least one third logical sector and the physical sector, the sub-request is written to the raw disk in an append-only manner, and the logical sector is released after the write.

[0045] Secondly, embodiments of the present invention provide a data storage system for bare disks, comprising:

[0046] The request-response module is used to respond to a write request, obtain the request data, and cache the request data in a logically contiguous manner in a target logical sector group; the target logical sector group includes at least one logical sector;

[0047] The request segmentation module is used to segment the request data according to the coverage relationship between the request data and the logical sector to obtain a sub-request list for each logical sector;

[0048] The request storage module is used to store the request data to the raw disk in an append-write manner according to the mapping relationship between the logical sectors and the physical sectors.

[0049] Thirdly, embodiments of the present invention provide a data storage device for bare disks, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the data storage method for bare disks as described above.

[0050] Fourthly, embodiments of the present invention provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the data storage method for bare disk as described above.

[0051] Fifthly, embodiments of the present invention provide a computer program product, the computer program product including a computer program or computer instructions, wherein when the computer program or computer instructions are executed by a processor, the data storage method for bare disks as described above is performed.

[0052] Compared with existing technologies, this invention discloses a data storage method, system, device, medium, and product for raw disks. In response to a write request, it acquires request data and caches the request data in a logically contiguous manner within a target logical sector group. The target logical sector group includes at least one logical sector. Based on the coverage relationship between the request data and the logical sectors, the request data is segmented to obtain a sub-request list for each logical sector. Based on the mapping relationship between the logical sectors and physical sectors, the request data is stored on the raw disk using an append-only write method. By employing this embodiment of the invention, a high-performance append-only write method is utilized, combined with the segmentation of request data within logical sectors based on coverage relationships, providing upper-layer applications with continuous access semantics for data within a file. Attached Figure Description

[0053] Figure 1 This is a flowchart illustrating the steps of a data storage method for bare disks provided in an embodiment of the present invention.

[0054] Figure 2 This is a schematic diagram illustrating the result of caching and segmenting concurrent request data according to an embodiment of the present invention;

[0055] Figure 3 This is a schematic diagram of the structure of a data storage system for bare disks provided in an embodiment of the present invention;

[0056] Figure 4 This is a schematic diagram of the structure of a data storage device for bare disks provided in an embodiment of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] In the description and claims, it should be understood that the terms "first," "second," etc., used in the description and claims are only for the purpose of distinguishing the description of the same technical features, and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated, nor necessarily the order of description or chronological order. The terms are interchangeable where appropriate. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature.

[0059] A raw disk refers to a block storage device that does not contain any file system structure and is directly bound to the physical storage medium. A block storage device mainly consists of three areas: the superblock area, the metadata area, and the data area.

[0060] The superblock region contains important global metadata information for some storage engines, such as storage engine version, status, block device size and type, logical data block size and number, etc.

[0061] The metadata area is mainly used to store file index information inside the storage engine, such as file name, version, status, size, and mapping relationship between file data and logical data blocks.

[0062] The data area is divided into several logical data blocks according to a fixed block size. Each logical data block corresponds to a contiguous region on the raw disk, and each logical data block contains several logical sectors. Each logical sector corresponds to one or more physical sectors on the raw disk. A physical sector is the basic unit for reading and writing to the raw disk, mapped from a logical sector address to a specific physical sector address on the disk. The size of the logical sector is configured when the block device is formatted.

[0063] Overwrite and append are different methods of writing data to a "file." Here, "file" is used in a broad sense; it can be a traditional file or a dedicated data storage and management medium in a storage system, such as a segment or extent. Overwrite involves writing to any address in the file, overwriting existing data. For the hard drive containing the data, this is a random write operation. Append, on the other hand, appends new content to the end of the file; addresses that have already been written to or are currently being written to cannot be rewritten. For the hard drive containing the data, this is a sequential write operation. For block storage devices, due to limitations in their internal structure or caching strategies, their sequential read / write performance is significantly higher than their random read / write performance.

[0064] Existing data storage solutions that directly utilize raw disks manage storage devices through storage engines. These engines can be deeply optimized for specific application scenarios and workloads, such as adjusting data block size and I / O scheduling strategies. Simultaneously, storage engines can customize data layout and storage formats, precisely controlling the allocation and management of storage resources, thereby improving raw disk space utilization efficiency. Furthermore, to fully leverage the sequential read / write performance of raw disks, some storage engines employ append-only writes when writing data to the raw disk; this approach is more suitable for building high-performance storage engines.

[0065] However, the applicant found that the existing schemes for writing raw disk data via overwrite cannot fully leverage the sequential write performance advantages of raw disks; and the existing schemes for writing raw disk data via append have specific requirements for the writing method or format of the data, such as the writing address must meet a certain alignment, which has problems such as the storage engine being unable to provide access methods with continuous semantics within the file to the upper layer, i.e. the client.

[0066] Based on the above considerations, embodiments of the present invention provide a data storage method for raw disks, which can be deployed in a storage engine. Please refer to [link to relevant documentation]. Figure 1 In this embodiment, the data storage method for raw disks is specifically executed through steps S1 to S3:

[0067] S1. In response to a write request, obtain the requested data and cache the requested data in a logically contiguous manner in a target logical sector group; the target logical sector group includes at least one logical sector.

[0068] S2. Based on the coverage relationship between the request data and the logical sector, the request data is segmented to obtain a sub-request list for each logical sector.

[0069] S3. Based on the mapping relationship between the logical sectors and the physical sectors, the requested data is stored to the raw disk in an append-only manner.

[0070] It should be noted that the target logical sector group includes at least one free logical sector. Cached request data in the target logical sector group in a logically contiguous manner means that the request data is received in logically contiguous logical sectors. Specifically, starting from a certain free logical sector, the data is sequentially filled into subsequent consecutive logical sectors until the request data is completely cached.

[0071] It should also be noted that the embodiments of the present invention support the client to send request data at the byte level. For the same write request, through the segmentation logic of the embodiments of the present invention, the request data can be cached across logical sectors or even across logical blocks. In the embodiments of the present invention, the request data will be marked to determine which write request it comes from. Furthermore, the segmented request data can also show which part of the write request it belongs to.

[0072] Due to the limitations of raw disks, when reading data, even if only a few bytes of data within a sector are needed, the entire sector must be read into memory before the required data is filtered. Similarly, when writing data, even if only a few bytes need to be written or updated, the data in that sector needs to be merged (if the data in the sector is not in memory, it needs to be read into memory first) before the sector is written to disk. This results in continuous logical data potentially being split into multiple discrete physical sectors under physical sector alignment requirements or append-only write strategies, requiring upper-layer clients to traverse all physical sectors to access the data.

[0073] In some preferred embodiments, the data storage method for bare disks is deployed on a storage engine base and can be applied to block storage, file storage, and object storage products for mobile clouds.

[0074] In this embodiment of the invention, the request data is cached in logical sectors in a logically continuous manner. Subsequently, the request data is divided into sub-requests according to the coverage relationship, which can quickly locate which sub-request each byte of the written request belongs to. Through the binding relationship between sub-requests, logical sectors and physical sectors, the upper-layer client can perceive the continuous storage logic of the data through the logical sector address, thus building an adaptation bridge for raw disk block storage and upper-layer continuous semantics.

[0075] In a preferred embodiment, the logical sector includes a client data area and a verification data area; the client data area is used to cache the request data; and the verification data area is used to cache the verification data generated when writing data to disk.

[0076] The raw disk itself only provides sector-level read / write interfaces and lacks error detection capabilities. To ensure the correctness of data written to the raw disk, each logical sector is divided into two parts: a client data area and a verification data area. During the logical sector data caching phase, the client's request data is cached in the client data area, while during the disk writing phase, dynamically generated verification data is added to the verification data area to ensure that upper-layer business logic can correctly perceive the data.

[0077] In some preferred embodiments, the verification object of the verification data is not a single request data, but the complete data within the client data area of ​​the current logical sector; the verification data includes CRC verification information.

[0078] Preferably, the size of the logical sector is M+N bytes, where the first M bytes are the client data area and the last N bytes are the verification data area.

[0079] It is understood that the requested data is only stored in the client data area. Therefore, when the requested data is subsequently segmented according to the coverage relationship between the requested data and the logical sector, the requested data is segmented based on the size of the client data area of ​​the logical sector.

[0080] As a preferred implementation, step S1, in response to a write request, obtaining request data and caching the request data in a logically contiguous manner in the target logical sector group, includes:

[0081] In response to at least one write request, establish an ordered queue;

[0082] According to the order of the write requests, the request data of the write requests are inserted into the ordered queue;

[0083] Based on the ordered queue, all the requested data are cached in the target logical sector group in a logically contiguous manner.

[0084] It should be noted that, in this embodiment of the invention, multiple write requests issued concurrently by the client can be responded to. However, due to network or other reasons, these write requests cannot be guaranteed to arrive at the storage engine layer in order, nor can it be guaranteed that all write requests will arrive. The order of multiple write requests is determined by the client, and the storage engine cannot change the order of these requested data on the disk after they are written to disk.

[0085] It should also be noted that each request data has its own address and size, which can be used to determine the continuity of request data arriving on that logical sector. Based on the append-only write feature, valid data within a file must be continuous, with no gaps allowed. Therefore, request data located later in a logical sector must wait for request data located earlier; otherwise, it does not conform to the semantics of continuous data, and even if the write is successful, it will not be considered valid by the storage engine. In this embodiment of the invention, the establishment of an ordered queue ensures that multiple concurrent request data can be cached in an orderly manner.

[0086] Further, preferably, the step of establishing an ordered queue in response to at least one write request includes:

[0087] In response to at least one write request, an initial ordered queue is established based on the number of write requests;

[0088] If a new write request is received within the preset waiting time, the initial ordered queue is updated and the preset waiting time is reset.

[0089] Otherwise, an ordered queue is obtained based on the initial ordered queue and its update status.

[0090] In addition to the scenario of concurrent write requests mentioned above, there is also the situation where write requests arrive continuously. In this embodiment of the invention, a preset waiting time is set. If the wait expires, the first consecutive write requests are cached and subsequently written to disk. This allows the request data at the beginning of the logical sector to wait for the request data at the end, reducing the number of disk writes. At the same time, it ensures that the last request data of the client is triggered by a timer timeout rather than waiting indefinitely for subsequent requests, thus ensuring successful disk writing.

[0091] In a preferred implementation, step S2 involves segmenting the request data according to the coverage relationship between the request data and the logical sector to obtain a sub-request list for each logical sector, which is then executed via steps S21-S23.

[0092] S21. Obtain the logical sector where the current request data is located, and obtain the first logical sector and the second logical sector based on the request data in the logical sector; the first logical sector includes request data other than the current request data, and the second logical sector only includes the current request data, but is not completely covered;

[0093] S22. Based on the boundary between the first logical sector and the second logical sector, the current request data is divided into at least one current sub-request to obtain the sub-requests corresponding to all request data.

[0094] S23. Obtain a list of sub-requests for each logical sector based on the sub-requests in the logical sector.

[0095] It should be noted that in some embodiments, the amount of data currently requested is large, which may cover several logical sectors. In this case, the current requested data may partially fall into the first logical sector and partially fall into the second logical sector. In other embodiments, the amount of data currently requested is small, which may only cover one logical sector. In this case, the logical sector is further determined to be either the first or the second logical sector based on the data storage situation of the requested data within that logical sector.

[0096] For example, the current requested data actually only has the first logical sector, and the second logical sector is marked as empty.

[0097] The client data area of ​​the first logical sector may be completely covered or not completely covered, characterized in that it includes request data corresponding to different write requests; the data in the second logical sector may be part of the request data or complete request data of a single write request, characterized in that there is only one request data corresponding to a write request, and the client data area is not completely covered.

[0098] It is understandable that the order of write requests will lead to different cache locations within the target logical sector group, and due to differences in the size of the requested data, a single request may be cached in a single logical sector or across multiple logical sectors. In this case, a single request may only have the first logical sector or the second logical sector, or it may have both.

[0099] In a preferred embodiment, please refer to Figure 2 ,exist Figure 2 There are 7 concurrent / sequential write requests, labeled 1 through 7. The requested data is sorted according to the order of the write requests and then cached in the target logical sector group, which consists of 8 logical sectors labeled a through h. Each logical sector is 4KB in size, with the client data area being 4032B and the checksum data area being 64B.

[0100] For request data 1, it is cached in logical sector a, which is the first logical sector and has no second logical sector. Request data 1 is split into a sub-request through logical sector a, and this sub-request shares the cache space with the sub-requests of other request data. For request data 2, it spans logical sectors a to b, and both logical sectors a and b are first logical sectors. Request data 2 is split into two sub-requests through the boundary between logical sectors a and b. For request data 3, its cache spans logical sectors b to e, where logical sector b is the first logical sector and logical sector e is the second logical sector. Request data 3 is split into three sub-requests through the boundary between logical sectors b and e. Each request data is split into at least one sub-request and at most three sub-requests. The splitting of other request data follows the same logic and will not be elaborated further here.

[0101] After segmenting the requested data, a sub-request list corresponding to each logical sector can be obtained. For example, the sub-request list for logical sector a includes the first sub-request for requested data 1 and the first sub-request for requested data 2. The sub-request list for logical sector b includes the second sub-request for requested data 2 and the first sub-request for requested data 3. More specifically, the sub-request lists for logical sectors c and d both consist of the second sub-request for requested data 3. It should be noted that the second sub-request for requested data 3 is a complete sub-request data block, which can be written to disk in one go based on the sub-request, effectively improving the write throughput of the storage engine.

[0102] Further, as a preferred implementation, step S22, at the boundary between the first logical sector and the second logical sector, divides the current request data into at least one current sub-request to obtain the sub-requests corresponding to all request data, including:

[0103] If the current request data falls within the boundary of the first logical sector or the boundary of the second logical sector, the current request data is segmented at the boundary of the first logical sector and the second logical sector to obtain at least one current sub-request.

[0104] Otherwise, the current request data will be used as the current sub-request;

[0105] After splitting all request data, we get the sub-requests corresponding to all request data.

[0106] In this embodiment of the invention, the segmentation logic is further refined to cover how sub-requests should be confirmed in all cases.

[0107] Please see Figure 2 For requested data 1-3 and requested data 5-7, the boundaries of their first or second logical sectors fall within their requested data, and sub-requests can be determined by directly dividing them by logical sectors. For requested data 4, logical sector e is its first logical sector, but the boundary of logical sector e does not fall within requested data 4. In this embodiment, considering the scenario where the requested data is small and does not cover the logical sector boundary, this requested data is directly treated as a sub-request. Similarly, if the boundary of a requested data exactly aligns with the boundary of a logical sector, according to the principle of raw disk storage, there is no need for data merging or waiting for subsequent data; data can be directly written to disk, and therefore this requested data is also directly treated as a sub-request.

[0108] In a preferred embodiment, step S23, obtaining a sub-request list for each logical sector based on the sub-requests in the logical sector, is executed via steps S231-S233:

[0109] Step S231: Based on the alignment of the boundary of the sub-request with the boundary of any of the logical sectors, set the attributes of the sub-request as header, middle and tail.

[0110] Step S232: Based on the attributes, establish a mapping relationship between sub-requests and logical sectors; wherein, middle sub-requests are mapped to at least one logical sector, and head sub-requests and tail sub-requests are mapped to one logical sector;

[0111] Step 233: Based on the mapping relationship, obtain the sub-request list for each logical sector.

[0112] In this embodiment of the invention, attributes are set for sub-requests based on their alignment with the boundaries of logical sectors. It should be noted that a middle sub-request can be mapped to multiple logical sectors; that is, a middle sub-request is a relatively large sub-request, while the storage of the header and tail sub-requests within a logical sector may not occupy an entire logical sector.

[0113] Further, preferably, step S231, setting the attributes of the sub-request as header, middle, and tail based on the alignment of the sub-request's boundary with the boundary of any of the logical sectors, includes:

[0114] If the front and rear boundaries of the sub-request are aligned with the boundaries of the logical sector, then the attribute of the sub-request is set to the middle.

[0115] If the sub-request's front boundary is aligned with the boundary of any logical sector, then the sub-request's attribute is set to tail.

[0116] Otherwise, set the attribute of the sub-request as a header.

[0117] With the above attribute setting scheme, under normal circumstances, the back boundary of the header sub-request is aligned with the boundary of the logical sector, the front and back boundaries of the middle sub-request are aligned with the boundary of the logical sector, and the front boundary of the tail sub-request is aligned with the boundary of the logical sector.

[0118] In some preferred embodiments, if the requested data is divided into only one or two sub-requests, including a middle sub-request, the attributes of the middle sub-request are updated according to the position of the sub-request.

[0119] In this embodiment, considering that the middle sub-request should be between the header and tail sub-requests, the definition of the middle sub-request may not exist if the header and tail sub-requests do not exist simultaneously. Therefore, after the initial attribute settings, further updates are performed to ensure the priority setting of header and tail attributes.

[0120] exist Figure 2In the process, request data 1 is split into a tail sub-request. Request data 2 is split into a header sub-request located in logical sector a and a tail sub-request located in logical sector b. Request data 3 is split into three sub-requests: a header sub-request located in logical sector b, a middle sub-request spanning logical sectors c and d, and a tail sub-request located in logical sector e. Request data 4 is split into a header sub-request. Request data 5 is split into a header sub-request. The tail sub-request of request data 3, the header sub-request of request data 4, and the header sub-request of request data 5 are sequentially concatenated and cached in logical sector e. Request data 6 is split into a header sub-request located in logical sector f and a tail sub-request located in logical sector g. Request data 7 is split into a header sub-request located in logical sector g and a tail sub-request located in logical sector h.

[0121] The boundaries of request data 4 are not aligned with the logical sector boundaries, so its sub-request can be considered as a header sub-request or a tail sub-request. In a preferred embodiment of the present invention, it is considered as a header sub-request. When processing this sub-request, it is processed together with the tail sub-request of the preceding request data 3 and the header sub-request of the following request data 5.

[0122] In some other preferred embodiments, the sub-request attributes are set directly based on the overlap of boundaries. In this case, the attribute setting differs from the previous embodiment in that request data 6 is divided into a middle sub-request located in logical sector f and a tail sub-request located in logical sector g; request data 7 is divided into a head sub-request located in logical sector g and a middle sub-request located in logical sector h.

[0123] In this embodiment, if the size of a request data is large, and it spans at least 3 logical sectors starting from the front boundary of a certain logical sector, it is possible to capture the larger sub-request based on the logical sector boundary.

[0124] In some preferred embodiments, step S232, establishing a mapping relationship between sub-requests and logical sectors based on the attributes, includes:

[0125] If the sub-request is a middle sub-request, then obtain at least one third logical sector covered by the middle sub-request and map the sub-request to at least one third logical sector;

[0126] If the sub-request is not a middle sub-request, then all sub-requests in the fourth logical sector where the sub-request is located are connected to form a connected sub-request, and the connected sub-request is mapped to the fourth logical sector.

[0127] It should be noted that the request data volume of the middle sub-request is exactly equal to the total capacity of N logical sectors, and is cached by N consecutive third logical sectors. During mapping, the middle sub-request is mapped to N third logical sectors. That is, multiple logical sectors may be bound to the same sub-request.

[0128] For example, Figure 2 The third logical sector of the middle sub-request of request data 3 includes logical sectors c and d, and the middle sub-request of request data 3 is mapped to logical sectors c and d simultaneously.

[0129] In the request data segmentation and attribute setting scheme of this invention, since caching is performed in a logically consecutive manner, if a logical sector is not completely occupied by a sub-request, then that logical sector will definitely not contain any middle sub-requests of any write request. In this case, all sub-requests in the logical sector are connected according to the sub-requests to form connected sub-requests, and then mapped. This ensures that the logical sector is bound to all its sub-requests.

[0130] For example, Figure 2 The tail sub-request of request data 3, the header sub-request of request data 4, and the header sub-request of request data 5 are all cached in logical sector e. During mapping, since logical sector e does not contain the middle sub-request, all sub-requests in logical sector e are connected in sequence to form a connection sub-request and mapped to logical sector e.

[0131] In some preferred embodiments, step S3, based on the mapping relationship between the logical sectors and physical sectors, stores the requested data to the raw disk in an append-only manner, and is executed through steps S31-S32:

[0132] S31. Generate verification data based on the sub-request list of the logical sector, and add the verification data to the verification data area of ​​the logical sector.

[0133] S32. Based on the mapping relationship between the logical sector and the physical sector, copy the data in the logical sector to the corresponding physical sector until all requested data is stored to the raw disk in append mode.

[0134] It should be noted that a logical sector may include multiple sub-requests, or may be bound to the same sub-request with other logical sectors; and the client data area in a logical sector may be completely covered or not completely covered. In this embodiment of the invention, these various situations can be handled uniformly. After the write request response is completed, all request data is cached in the target logical sector group. At this time, regardless of the attributes of the sub-requests in each logical sector or the data coverage of the logical sector, the cached request data in each logical sector is written to disk on a logical sector-by-logical-sector basis. During the data writing process, verification data is generated based on the sub-requests stored in the logical sector, the request data content, and the logical sector coverage integrity, ensuring that the request data in each logical sector can be written to disk and read and written by the upper-layer client.

[0135] As a preferred embodiment, this invention also provides a data persistence method for specific scenarios. Step S3, based on the mapping relationship between logical sectors and physical sectors, stores the requested data to the raw disk in an append-only manner, and may further include:

[0136] If the list of sub-requests in a logical sector completely covers the logical sector, then after the list of sub-requests is written to the raw disk in an append-only manner according to the mapping relationship between the logical sector and the physical sector, the logical sector is released.

[0137] In this embodiment of the invention, when the client data area of ​​a logical sector is full, verification data can be added to it before writing to disk. Since no further write requests will be made to this logical sector, the data in this logical sector can be released directly after the disk writing is completed.

[0138] Understandably, for logical sectors that are not fully written to the client's data area, they may still respond to subsequent write requests. For example, if a new write request is received within a preset waiting time, the request data corresponding to the new write request will be written to the currently not-fully-written logical sector in a logically continuous manner. Therefore, for these logical sectors, the data on the logical sector will not be released after writing to disk, so that it can be merged and used for the next write operation.

[0139] Furthermore, there are two possible scenarios when the client data area of ​​a logical sector is full: one is that the logical sector includes multiple sub-requests, in which case the logical sector will necessarily not include the middle sub-requests; the other is that the logical sector only includes the middle sub-requests.

[0140] In the first case, the correspondence between logical sectors and sub-requests is one-to-many, and the data in the logical sectors can be directly written to disk and then released. In the second case, the correspondence between logical sectors and sub-requests may be one-to-one or many-to-one. In the preferred embodiment, the data is written to disk based on at least one third logical sector cached by the middle sub-request.

[0141] Preferably, if the sub-request list in the logical sector completely covers the logical sector, then after writing the sub-request list to the raw disk in append mode according to the mapping relationship between the logical sector and the physical sector, the logical sector is released, including:

[0142] If the sub-request list of a logical sector contains only one sub-request and the logical sector completely covers it, filter at least one third logical sector corresponding to the sub-request.

[0143] Based on the mapping relationship between the at least one third logical sector and the physical sector, the sub-request is written to the raw disk in an append-only manner, and the logical sector is released after the write.

[0144] For the middle sub-request, it is aligned with the logical sector and does not need to be merged with the data of other sub-requests. After adding verification data, this part of the data can be written to the block storage device. Especially for large request data, the middle sub-requests are also large. The one-time disk write method of this invention can effectively improve write throughput performance.

[0145] For example, after filtering out at least one third logical sector corresponding to the sub-request, memory is allocated according to the number of third logical sectors and the correspondence between logical sector and physical sector size.

[0146] The data storage method for raw disks provided in this embodiment of the invention caches requested data in logical sectors in a logically continuous manner. Subsequently, the requested data is divided into sub-requests according to the overlay relationship, which can quickly locate which sub-request each byte of the write request belongs to. Through the binding relationship between sub-requests, logical sectors and physical sectors, the upper-layer client can perceive the continuous storage logic of data through the logical sector address, thus building an adaptation bridge between raw disk block storage and upper-layer continuous semantics.

[0147] This invention provides a data storage system for raw disks. Please refer to [link to relevant documentation]. Figure 3 The data storage system for raw disks includes a request response module 11, a request partitioning module 12, and a request storage module 13, wherein:

[0148] Request response module 11 is used to respond to a write request, obtain request data, and cache the request data in a logically contiguous manner in a target logical sector group; the target logical sector group includes at least one logical sector;

[0149] The request segmentation module 12 is used to segment the request data according to the coverage relationship between the request data and the logical sector to obtain a sub-request list for each logical sector.

[0150] The request storage module 13 is used to store the request data to the raw disk in an append write mode according to the mapping relationship between the logical sector and the physical sector.

[0151] In a preferred embodiment, the logical sector includes a client data area and a verification data area; the client data area is used to cache the request data; and the verification data area is used to cache the verification data generated when writing data to disk.

[0152] In a preferred embodiment, the request-response module 11 includes:

[0153] An ordered queue creation unit is used to create an ordered queue in response to at least one write request;

[0154] A request data sorting unit is used to insert the request data of the write requests into the ordered queue according to the order of the write requests;

[0155] The request data caching unit is used to cache all the request data in a logically contiguous manner in the target logical sector group according to the ordered queue.

[0156] Further, preferably, the ordered queue establishment unit is specifically used for:

[0157] In response to at least one write request, an initial ordered queue is established based on the number of write requests;

[0158] If a new write request is received within the preset waiting time, the initial ordered queue is updated and the preset waiting time is reset.

[0159] Otherwise, an ordered queue is obtained based on the initial ordered queue and its update status.

[0160] In a preferred embodiment, the request segmentation module 12 includes:

[0161] The logical sector acquisition unit is used to acquire the logical sector where the current request data is located, and to obtain a first logical sector and a second logical sector based on the request data in the logical sector; the first logical sector includes request data other than the current request data, and the second logical sector only includes the current request data, but is not completely covered;

[0162] The sub-request generation unit is used to divide the current request data into at least one current sub-request based on the boundary between the first logical sector and the second logical sector, so as to obtain the sub-requests corresponding to all request data.

[0163] The sub-request list generation unit is used to obtain a sub-request list for each logical sector based on the sub-requests in the logical sector.

[0164] Further, preferably, the sub-request generation unit is specifically used for:

[0165] If the current request data falls within the boundary of the first logical sector or the boundary of the second logical sector, the current request data is segmented at the boundary of the first logical sector and the second logical sector to obtain at least one current sub-request.

[0166] Otherwise, the current request data will be used as the current sub-request;

[0167] After splitting all request data, we get the sub-requests corresponding to all request data.

[0168] Preferably, the sub-request list generation unit is specifically used for:

[0169] Based on the alignment of the boundary of the sub-request with the boundary of any of the logical sectors, the attributes of the sub-request are set as header, middle, and tail.

[0170] Based on the attributes, establish a mapping relationship between sub-requests and logical sectors; wherein, middle sub-requests are mapped to at least one logical sector, and head sub-requests and tail sub-requests are mapped to one logical sector;

[0171] Based on the mapping relationship, a sub-request list for each logical sector is obtained.

[0172] Furthermore, preferably, setting the attributes of the sub-request as header, middle, and tail based on the alignment of the sub-request's boundary with the boundary of any of the logical sectors includes:

[0173] If the front and rear boundaries of the sub-request are aligned with the boundaries of the logical sector, then the attribute of the sub-request is set to the middle.

[0174] If the sub-request's front boundary is aligned with the boundary of any logical sector, then the sub-request's attribute is set to tail.

[0175] Otherwise, set the attribute of the sub-request as a header.

[0176] Preferably, establishing the mapping relationship between sub-requests and logical sectors based on the attributes includes:

[0177] If the sub-request is a middle sub-request, then obtain at least one third logical sector covered by the middle sub-request and map the sub-request to at least one third logical sector;

[0178] If the sub-request is not a middle sub-request, then all sub-requests in the fourth logical sector where the sub-request is located are connected to form a connected sub-request, and the connected sub-request is mapped to the fourth logical sector.

[0179] In a preferred embodiment, the request storage module 13 includes:

[0180] The verification data addition unit is used to generate verification data based on the sub-request list of the logical sector and add the verification data to the verification data area of ​​the logical sector.

[0181] The data writing unit is used to copy data in the logical sector to the corresponding physical sector according to the mapping relationship between the logical sector and the physical sector, until all requested data is stored to the raw disk in append mode.

[0182] Further, preferably, the request storage module 13 further includes:

[0183] The logical sector release module is used to release the logical sector after writing the sub-request list to the raw disk in an append-only manner according to the mapping relationship between the logical sector and the physical sector if the sub-request list in the logical sector completely covers the logical sector.

[0184] Furthermore, as a preferred embodiment, the logical sector release module is specifically used for:

[0185] If the sub-request list of a logical sector contains only one sub-request and the logical sector completely covers it, filter at least one third logical sector corresponding to the sub-request.

[0186] Based on the mapping relationship between the at least one third logical sector and the physical sector, the sub-request is written to the raw disk in an append-only manner, and the logical sector is released after the write.

[0187] The data storage system for raw disks provided in this embodiment of the invention caches requested data in logical sectors in a logically continuous manner. Subsequently, the requested data is divided into sub-requests according to the overlay relationship, which can quickly locate which sub-request each byte of the write request belongs to. Through the binding relationship between sub-requests, logical sectors and physical sectors, the upper-layer client can perceive the continuous storage logic of data through the logical sector address, thus building an adaptation bridge between raw disk block storage and upper-layer continuous semantics.

[0188] Please see Figure 4 , Figure 4 This is a structural block diagram of a data storage device for bare disks provided in an embodiment of the present invention. The data storage device for bare disks includes a processor 31, a memory 32, and a computer program stored in the memory 32 and executable on the processor 31. When the processor 31 executes the computer program, it implements the steps in the various embodiments of the data storage method for bare disks described above, such as steps S1 to S3.

[0189] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 32 and executed by the processor 31 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the bare disk-oriented data storage device.

[0190] The data storage device for bare disks may include, but is not limited to, a processor 31 and a memory 32. Those skilled in the art will understand that the schematic diagram is merely an example of a data storage device for bare disks and does not constitute a limitation on such a device. It may include more or fewer components than illustrated, or combine certain components, or use different components. For example, the data storage device for bare disks may also include input / output devices, network access devices, buses, etc.

[0191] The processor 31 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 31 is the control center of the bare disk-oriented data storage device, connecting all parts of the bare disk-oriented data storage device via various interfaces and lines.

[0192] The memory 32 can be used to store the computer programs and / or modules. The processor 31 implements various functions of the bare disk-oriented data storage device by running or executing the computer programs and / or modules stored in the memory 32 and calling the data stored in the memory 32. The memory 32 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 32 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0193] Wherein, if the modules / units integrated into the data storage device for bare disks are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor 31, it can implement the steps of the various method embodiments described above. Wherein, the computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0194] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A data storage method for bare disks, characterized in that, include: In response to a write request, the requested data is obtained and the requested data is cached in the target logical sector group in a logically contiguous manner. The target logical sector group includes at least one logical sector; Based on the coverage relationship between the request data and the logical sector, the request data is segmented to obtain a sub-request list for each logical sector; Based on the mapping relationship between the logical sectors and the physical sectors, the requested data is stored to the raw disk in an append-only manner; The step of segmenting the request data according to the coverage relationship between the request data and the logical sector to obtain a sub-request list for each logical sector includes: Obtain the logical sector where the current request data is located, and obtain the first logical sector and the second logical sector based on the request data in the logical sector; the first logical sector includes request data other than the current request data, and the second logical sector only includes the current request data, but it is not completely covered; Based on the boundary between the first logical sector and the second logical sector, the current request data is divided into at least one current sub-request to obtain the sub-requests corresponding to all request data; Based on the sub-requests in the logical sector, a list of sub-requests for each logical sector is obtained; The step of dividing the current request data into at least one current sub-request based on the boundary between the first logical sector and the second logical sector, to obtain the sub-requests corresponding to all request data, includes: If the current request data falls within the boundary of the first logical sector or the boundary of the second logical sector, the current request data is segmented at the boundary of the first logical sector and the second logical sector to obtain at least one current sub-request. Otherwise, the current request data will be used as the current sub-request; After splitting all request data, we get the sub-requests corresponding to all request data.

2. The data storage method for bare disks as described in claim 1, characterized in that, The logical sector includes a client data area and a verification data area; the client data area is used to cache the request data; the verification data area is used to cache the verification data generated when writing data to disk.

3. A data storage method for bare disks as described in claim 1 or 2, characterized in that, The step of responding to a write request, obtaining the requested data, and caching the requested data in a logically contiguous manner in the target logical sector group includes: In response to at least one write request, establish an ordered queue; According to the order of the write requests, the request data of the write requests are inserted into the ordered queue; Based on the ordered queue, all the requested data are cached in the target logical sector group in a logically contiguous manner.

4. The data storage method for bare disks as described in claim 3, characterized in that, The step of establishing an ordered queue in response to at least one write request includes: In response to at least one write request, an initial ordered queue is established based on the number of write requests; If a new write request is received within the preset waiting time, the initial ordered queue is updated and the preset waiting time is reset. Otherwise, an ordered queue is obtained based on the initial ordered queue and its update status.

5. A data storage method for bare disks as described in claim 1, characterized in that, The step of obtaining a list of sub-requests for each logical sector based on the sub-requests in the logical sector includes: Based on the alignment of the boundary of the sub-request with the boundary of any of the logical sectors, the attributes of the sub-request are set as header, middle, and tail. Based on the attributes, establish a mapping relationship between sub-requests and logical sectors; wherein, middle sub-requests are mapped to at least one logical sector, and head sub-requests and tail sub-requests are mapped to one logical sector; Based on the mapping relationship, a sub-request list for each logical sector is obtained.

6. The data storage method for bare disks as described in claim 5, characterized in that, The step of setting the attributes of the sub-request as header, middle, and tail based on the alignment of the sub-request's boundary with the boundary of any of the logical sectors includes: If the front and rear boundaries of the sub-request are aligned with the boundaries of the logical sector, then the attribute of the sub-request is set to the middle. If the sub-request's front boundary is aligned with the boundary of any logical sector, then the sub-request's attribute is set to tail. Otherwise, set the attribute of the sub-request as a header.

7. A data storage method for bare disks as described in claim 5, characterized in that, The step of establishing the mapping relationship between sub-requests and logical sectors based on the attributes includes: If the sub-request is a middle sub-request, then obtain at least one third logical sector covered by the middle sub-request and map the sub-request to at least one third logical sector; If the sub-request is not a middle sub-request, then all sub-requests in the fourth logical sector where the sub-request is located are connected to form a connected sub-request, and the connected sub-request is mapped to the fourth logical sector.

8. A data storage method for bare disks as described in claim 1, characterized in that, The step of storing the requested data to the raw disk in an append-only manner according to the mapping relationship between the logical sectors and the physical sectors includes: Based on the sub-request list of the logical sector, generate verification data and add the verification data to the verification data area of ​​the logical sector; Based on the mapping relationship between the logical sectors and physical sectors, the data in the logical sectors is copied to the corresponding physical sectors until all requested data is stored on the raw disk in append mode.

9. A data storage method for bare disks as described in claim 1 or 8, characterized in that, The step of storing the requested data to the raw disk in an append-only manner according to the mapping relationship between the logical sectors and the physical sectors includes: If the list of sub-requests in a logical sector completely covers the logical sector, then after the list of sub-requests is written to the raw disk in an append-only manner according to the mapping relationship between the logical sector and the physical sector, the logical sector is released.

10. A data storage method for bare disks as described in claim 9, characterized in that, If the list of sub-requests in a logical sector completely covers the logical sector, then after writing the list of sub-requests to the raw disk in an append-only manner according to the mapping relationship between the logical sector and the physical sector, the logical sector is released, including: If the sub-request list of a logical sector contains only one sub-request and the logical sector completely covers it, filter at least one third logical sector corresponding to the sub-request. Based on the mapping relationship between the at least one third logical sector and the physical sector, the sub-request is written to the raw disk in an append-only manner, and the logical sector is released after writing.

11. A data storage system for bare disks, characterized in that, include: The request-response module is used to respond to a write request, obtain the request data, and cache the request data in a logically contiguous manner in the target logical sector group. The target logical sector group includes at least one logical sector; The request segmentation module is used to segment the request data according to the coverage relationship between the request data and the logical sector to obtain a sub-request list for each logical sector; The request storage module is used to store the request data to the raw disk in an append-write manner according to the mapping relationship between the logical sectors and the physical sectors; The request segmentation module includes: The logical sector acquisition unit is used to acquire the logical sector where the current request data is located, and to obtain a first logical sector and a second logical sector based on the request data in the logical sector; the first logical sector includes request data other than the current request data, and the second logical sector only includes the current request data, but is not completely covered; The sub-request generation unit is used to divide the current request data into at least one current sub-request based on the boundary between the first logical sector and the second logical sector, so as to obtain the sub-requests corresponding to all request data. The sub-request list generation unit is used to obtain a sub-request list for each logical sector based on the sub-requests in the logical sector; The sub-request generation unit is specifically used for: If the current request data falls within the boundary of the first logical sector or the boundary of the second logical sector, the current request data is segmented at the boundary of the first logical sector and the second logical sector to obtain at least one current sub-request. Otherwise, the current request data will be used as the current sub-request; After splitting all request data, we get the sub-requests corresponding to all request data.

12. A data storage device for bare disks, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the data storage method for bare disks as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the data storage method for a bare disk as described in any one of claims 1 to 10.

14. A computer program product, characterized in that, The computer program product includes a computer program or computer instructions, which, when executed by a processor, perform the data storage method for bare disks as described in any one of claims 1 to 10.

Images