Data storage method, device, equipment and storage medium

Abstract

The present application relates to the field of computers, and in particular to the field of cloud storage technology, and discloses a data storage method, apparatus, device, and storage medium for improving data storage efficiency and avoiding file fragmentation, wherein the method comprises: receiving a data write request, and after determining that the data to be written is to be written into a target cache area, determining a first cache interval corresponding to the data to be written, and if it is determined that there is an overlapping area between the first cache interval and at least one second cache interval, merging to obtain a merged cache area, and then saving the data to be written based on the merged cache area. In this way, when storing files, not only can the cache interval be accurately and quickly located according to the offset, but also data to be written of any length can be processed, thereby reducing the time complexity of the operation and improving data storage efficiency.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to the field of cloud storage technology, and discloses a data storage method, apparatus, device, and storage medium. Background Art

[0002] The access speed of the cache in the storage device is faster than the access speed of the hard disk. Therefore, in actual applications, data is often stored in the cache from the hard disk in advance. After the storage device receives the input / output (IO) request, it operates on the data in the cache, thereby greatly improving the processing speed of the IO request.

[0003] Page Cache is a commonly used file caching solution. Figure 1 As shown in the figure, files are stored in the cache as pages, using a radix tree as the index structure. When the storage device receives an IO request, it searches the page using the radix tree. If a hit is found, the data to be written is written to the hit page and the written page is written back to the disk.

[0004] However, because the page size is fixed, large sequential writes require splitting the data to be written according to the page size and performing multiple page lookups in the radix tree, resulting in low processing efficiency. Furthermore, writing data to multiple pages can lead to file fragmentation, file holes, and a lack of flexibility. Summary of the Invention

[0005] Embodiments of the present application provide a data storage method, apparatus, device, and storage medium to improve data storage efficiency and avoid file fragmentation.

[0006] In a first aspect, an embodiment of the present application provides a data storage method, the method comprising:

[0007] Receive a data write request triggered for a target file, wherein the data write request carries at least the to-be-written data, the file identifier, and the offset corresponding to the target file;

[0008] When determining to write the data to be written into the target cache area corresponding to the file identifier, determining a first cache interval corresponding to the data to be written within the target cache area based on the area start position of the target cache area and the offset;

[0009] If it is determined that there is an overlapping area between the first cache interval and at least one second cache interval, merging the first cache interval and the at least one second cache interval to obtain a merged cache area; wherein each second cache interval is used to store other to-be-written data corresponding to the target file before the first cache interval is determined;

[0010] The merged cache area is used as a new second cache interval, and the data to be written is saved in the new second cache interval.

[0011] Optionally, after setting the new cache area as the new target cache area and before saving the to-be-written data into the first cache interval within the new target cache area, the method further includes:

[0012] Generate a sequence number corresponding to the new target cache area, where the sequence number is used to indicate the order in which metadata information of the new target cache area is saved;

[0013] After the first cache interval after the data is written is used as a second cache interval in the new target cache area, the method further includes:

[0014] generating metadata information of the new target cache area according to each second cache interval included in the new target cache area, wherein the metadata information at least includes a physical storage address corresponding to each second cache interval in the new target cache area;

[0015] According to the sequence number, the metadata information of the new target cache area is saved to a designated second storage location.

[0016] In a second aspect, an embodiment of the present application provides a data storage device, the data storage device comprising:

[0017] A receiving unit, configured to receive a data write request triggered for a target file, wherein the data write request carries at least the data to be written, a file identifier, and an offset corresponding to the target file;

[0018] a determining unit, configured to determine, when determining to write the data to be written into the target cache area corresponding to the file identifier, a first cache interval corresponding to the data to be written within the target cache area based on a starting position of the target cache area and the offset;

[0019] a merging unit configured to merge the first cache interval and the at least one second cache interval to obtain a merged cache area if it is determined that there is an overlapping area between the first cache interval and the at least one second cache interval; wherein each second cache interval is used to store other to-be-written data corresponding to the target file before the first cache interval is determined;

[0020] A saving unit is configured to use the merged cache area as a new second cache interval and save the to-be-written data into the new second cache interval.

[0021] Optionally, the data write request further carries the amount of data to be written;

[0022] When determining to write the data to be written into the target cache area corresponding to the file identifier, the determining unit is configured to:

[0023] If the sum of the amount of data stored in the target cache area and the amount of data to be written is not greater than a preset data amount threshold, then determining to write the data to be written into the target cache area; or

[0024] If the number of second cache intervals included in the target cache area is not greater than a preset interval number threshold, it is determined to write the to-be-written data into the target cache area.

[0025] Optionally, when determining the first cache interval corresponding to the to-be-written data within the target cache area based on the area starting position of the target cache area and the offset, the determining unit is configured to:

[0026] Determining, within the target cache area, a region starting position of a first cache interval corresponding to the to-be-written data based on a region starting position of the target cache area and the offset;

[0027] Based on the amount of data to be written, determining, within the target cache area, an end position of a first cache interval corresponding to the data to be written;

[0028] The first cache interval corresponding to the to-be-written data is determined based on a region start position of the first cache interval and a region end position of the first cache interval.

[0029] Optionally, when it is determined that there is an overlapping area between the first cache interval and one of the at least one second cache intervals, the determining unit is configured to perform any one or more combinations of the following operations:

[0030] If the region start position or the region end position of the first cache interval is between the region start position and the region end position of the second cache interval, it is determined that there is an overlapping region between the first cache interval and the second cache interval;

[0031] If the region start position or the region end position of the second cache interval is between the region start position and the region end position of the first cache interval, it is determined that there is an overlapping region between the first cache interval and the second cache interval;

[0032] If the region start position and the region end position of the first cache interval are the same as the region start position and the region end position of the second cache interval, it is determined that there is an overlapping area between the first cache interval and the second cache interval;

[0033] If the starting position of the first cache interval is the same as the ending position of the second cache interval, it is determined that there is an overlapping area between the first cache interval and the second cache interval;

[0034] If the end position of the first cache interval is the same as the start position of the second cache interval, it is determined that there is an overlapping area between the first cache interval and the second cache interval.

[0035] Optionally, when merging the first cache interval and the at least one second cache interval to obtain a merged cache area, the merging unit is configured to:

[0036] Based on the area starting positions of the at least one second cache interval and the first cache interval, the area starting position of the at least one second cache interval and the first cache interval that has the smallest distance from the area starting position of the target cache area is used as the area starting position of the merged cache area;

[0037] Based on the region end positions of the at least one second cache interval and the first cache interval, the region end position of the at least one second cache interval and the first cache interval, whichever has the largest distance from the region start position of the target cache area, is used as the region end position of the merged cache area;

[0038] The merged cache area is obtained based on a region start position of the merged cache area and a region end position of the merged cache area.

[0039] Optionally, the data storage device further includes a thread pool unit, and the thread pool unit is configured to:

[0040] Splitting the stored data in the target cache area according to a preset data block size to obtain at least one data block;

[0041] The at least one data block is written back to the designated first storage location.

[0042] Optionally, the data storage device further includes an allocation unit;

[0043] The allocating unit is configured to generate a new cache area corresponding to the file identifier and use the new cache area as a new target cache area if it is determined that the data to be written is not to be written into the target cache area corresponding to the file identifier;

[0044] The determining unit is further configured to determine, within the new target cache area, a first cache interval corresponding to the to-be-written data based on a starting position of the new target cache area and the offset;

[0045] The saving unit is further configured to save the data to be written into a first cache interval in the new target cache area, and use the first cache interval after the data is written as a second cache interval in the new target cache area.

[0046] Optionally, the data storage device further includes a submitting unit;

[0047] After using the new cache area as a new target cache area, the allocation unit is further configured to:

[0048] Generate a sequence number corresponding to the new target cache area, where the sequence number is used to indicate the order in which metadata information of the new target cache area is saved;

[0049] After the first cache interval after the data is written is used as a second cache interval in the new target cache area, the determining unit is further configured to:

[0050] generating metadata information of the new target cache area according to each second cache interval included in the new target cache area, wherein the metadata information at least includes a physical storage address corresponding to each second cache interval in the new target cache area;

[0051] The submitting unit is configured to save the metadata information of the new target cache area to a designated second storage location according to the sequence number.

[0052] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of any one of the above-mentioned data storage methods.

[0053] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which includes a program code. When the program code is run on an electronic device, the program code is used to enable the electronic device to execute the steps of any one of the above-mentioned data storage methods.

[0054] In a fifth aspect, embodiments of the present application provide a computer program product or computer program, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the aforementioned data storage methods.

[0055] The beneficial effects of this application are as follows:

[0056] In an embodiment of the present application, after receiving a data write request, if it is determined that there is an overlapping area between the first cache interval corresponding to the data to be written and at least one second cache interval, the first cache interval and the at least one second cache interval are merged, and then the data to be written is saved based on the merged cache area; in this way, after each data write request is received, the first cache interval can be accurately and quickly located according to the offset, and since the cache intervals can be merged, it is no longer limited by the size of the page in the page cache scheme. When facing data to be written of any length, only one search operation is required, which reduces the time complexity of the operation and improves data storage efficiency.

[0057] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0059] Figure 1 A schematic diagram of a flow chart of a page caching method in related art;

[0060] Figure 2 This is an optional schematic diagram of an application scenario in an embodiment of the present application;

[0061] Figure 3 A flowchart of a data storage method according to an embodiment of the present application;

[0062] Figure 4a This is a first schematic diagram of the overlapping area in the embodiment of the present application;

[0063] Figure 4b This is a second schematic diagram of the overlapping area in the embodiment of the present application;

[0064] Figure 4c This is a third schematic diagram of the overlapping area in the embodiment of the present application;

[0065] Figure 4d This is a fourth schematic diagram of the overlapping area in the embodiment of the present application;

[0066] Figure 4eThis is a fifth schematic diagram of the overlapping area in the embodiment of the present application;

[0067] Figure 4f This is a sixth schematic diagram of the overlapping area in the embodiment of the present application;

[0068] Figure 4g This is a seventh schematic diagram of the overlapping area in the embodiment of the present application;

[0069] Figure 4h 1 is a schematic diagram of a merged buffer area in an embodiment of the present application;

[0070] Figure 5 This is a schematic diagram of the process of storing data in an embodiment of the present application;

[0071] Figure 6 Schematic diagram of FRT in the process of storing data in an embodiment of the present application;

[0072] Figure 7 A schematic diagram of a cache and a first storage location in an embodiment of the present application;

[0073] Figure 8 A schematic diagram of a process for synchronizing data in an embodiment of the present application;

[0074] Figure 9a This is a first schematic diagram of a minimum heap in an embodiment of the present application;

[0075] Figure 9b This is a second schematic diagram of the minimum heap in the embodiment of the present application;

[0076] Figure 10 A schematic diagram of the structure of a data storage device according to an embodiment of the present application;

[0077] Figure 11 This is a schematic diagram of the composition structure of another data storage device in an embodiment of the present application;

[0078] Figure 12 This is a schematic diagram of the hardware structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0079] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of the technical solutions of this application, but not all of them. Based on the embodiments described in this application document, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the technical solutions of this application.

[0080] The terms "first," "second," and the like in the description and claims of the present invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, such that the embodiments of the present invention described herein can be practiced in orders other than those illustrated or described herein.

[0081] In embodiments of the present invention, the term "and / or" describes the association relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0082] The following explains some of the terms used in the embodiments of the present application to facilitate understanding by those skilled in the art.

[0083] 1. Cache: Data stored on slow physical devices is transferred to high-speed memory or flash memory devices through some mechanism to improve data access speed and system throughput.

[0084] 2. Write back: Storing data stored in high-speed memory or flash memory devices to slow physical devices.

[0085] 3. Balanced binary search tree: A binary search tree in which the absolute value of the difference between the left subtree depth and the right subtree depth of each node is no more than 1, and the key value of each node is greater than the primary key value of its left child and smaller than the primary key value of its right child.

[0086] 4. Minimum heap: A sorted complete binary tree in which the key value of the root node is the smallest of the key values ​​of all heap nodes.

[0087] 5. Cloud technology: refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data computing, storage, processing and sharing.

[0088] The embodiments of the present application mainly relate to cloud storage in cloud technology. Cloud storage is a new concept that extends and develops from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (storage devices are also called storage nodes) in the network through application software or application interfaces to work together and provide external data storage and business access functions.

[0089] Currently, storage systems utilize a method for creating logical volumes. When creating a logical volume, physical storage space is allocated for each logical volume. This physical storage space may consist of disks on a specific storage device or several storage devices. When a client stores data on a logical volume, it stores the data on a file system. The file system divides the data into multiple parts, each of which is an object. An object contains not only the data but also additional information such as the data identifier (ID entity). The file system writes each object to the physical storage space of the logical volume and records the storage location information for each object. Therefore, when a client requests access to data, the file system can provide access based on the storage location information for each object.

[0090] The storage system allocates physical storage space to logical volumes by pre-dividing the physical storage space into stripes based on the estimated capacity of the objects to be stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the Redundant Array of Independent Disks (RAID) groupings. A logical volume can be understood as a stripe, thereby allocating physical storage space to the logical volume.

[0091] The following is a brief introduction to the design concept of the embodiments of this application.

[0092] In order to improve data system performance, a common method is data caching, that is, transferring data stored on slow physical devices to high-speed memory or flash memory devices through some mechanism to increase data access speed and system throughput.

[0093] Page Cache is a common file caching solution, see Figure 1 As shown, the default page size of Page Cache is typically 4 kilobytes (KB). When the amount of data to be written exceeds 4KB, the data must be segmented into 4KB chunks and multiple page searches must be performed in the radix tree, resulting in low processing efficiency. Furthermore, writing data to multiple pages can lead to file fragmentation and holes, lacking flexibility. This is particularly true in distributed cloud storage systems, where large blocks are written sequentially and small blocks are written randomly. Using Page Cache can result in poor data storage and management performance.

[0094] Since Page Cache is used in a distributed cloud storage system, it is limited by a fixed page size, resulting in low data storage efficiency and causing problems such as file fragmentation and file holes. Therefore, in the embodiment of the present application, combined with the cloud storage technology in cloud technology, after each data write request is received, the first cache interval of the data to be written in the target cache area is obtained according to the offset carried in the data write request, and then the second cache interval with an overlapping area with the first cache interval is merged with the first cache interval, and then the data to be written is saved based on the merged cache area. In this way, after each data write request is received, the first cache interval can be accurately and quickly located according to the offset, and since cache intervals can be merged, it is no longer limited by the size of the page in the Page Cache solution. When facing data of any length to be written, only one search operation is required, which is very friendly to sequential writes and random writes, improves data storage efficiency, and reduces the time complexity of operations.

[0095] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application can be combined with each other if there is no conflict.

[0096] like Figure 2 As shown, it is a schematic diagram of the application scenario of the embodiment of the present application. It is a schematic diagram of the application scenario of the embodiment of the present application. The application scenario diagram includes two terminal devices 210 and a server 230, and the client 120 can be logged in through the terminal device 210. The terminal device 210 and the server 230 can communicate through the communication network. Each terminal device corresponds to a user, Figure 2 In the example, user A corresponds to terminal device 210 on the left, and user B corresponds to terminal device 210 on the right. In practice, the number of terminal devices is not limited. In some cases, the terminal devices may first communicate with each other through server 230, and then establish direct communication between terminal devices. This direct communication between terminal devices can be called point-to-point communication. In this case, some interactions between terminal devices 210 may not require the mediation of server 230.

[0097] In the embodiment of the present application, a client may be installed on the terminal device 210, and the client is used to receive operation instructions input by the user, such as write, read, etc. For example, the client may adopt a Filesystem in Userspace (FUSE) client. The server 230 is a server corresponding to the client.

[0098] In an optional embodiment, the communication network is a wired network or a wireless network. The terminal device 210 and the server 230 can be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0099] In the embodiment of the present application, the terminal device 210 is an electronic device used by the user, which can be a personal computer, mobile phone, tablet computer, notebook computer, e-book reader, or other computer device with certain computing capabilities and running instant messaging software and websites or social networking software and websites. Each terminal device 210 is connected to the server 230 via a wireless network. The server 230 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0100] In the embodiment of the present application, a user may log in to the client 220 through the terminal device 210 , and the terminal device 210 may respond to various operations triggered by the user on the client 220 and perform corresponding actions.

[0101] See Figure 3 As shown, Figure 3 This is a flowchart of a possible data storage method provided in an embodiment of the present application. The method can be applied to a terminal device or a server. The specific implementation process of the method is as follows:

[0102] S301: Receive a data write request triggered for a target file, where the data write request carries at least the to-be-written data, file identifier, and offset of the corresponding target file.

[0103] The data write request may be any data write request triggered for a target file, and the offset is used to indicate the starting position of the first cache interval corresponding to the data to be written in the target cache area corresponding to the file identifier.

[0104] S302: When determining to write the data to be written into the target cache area corresponding to the file identifier, a first cache interval corresponding to the data to be written is determined in the target cache area based on the area start position and offset of the target cache area.

[0105] It should be noted that in the embodiment of the present application, the cache interval in the target cache area that does not store the data to be written is called the first cache interval, and the cache interval in the target cache area that has the data to be written is called the second cache interval. The second cache interval stores valid and continuous data segments of the target file.

[0106] After receiving a data write request, there are but are not limited to the following two situations:

[0107] A first possible scenario: the data write request also carries the amount of data to be written.

[0108] Whether to write the data to be written into the target cache area corresponding to the file identifier may be determined by, but not limited to, the following two possible methods:

[0109] The first possible implementation method is to determine that the data to be written is written to the target cache area if the sum of the amount of data stored in the target cache area and the amount of data to be written is not greater than the preset data amount threshold value; otherwise, determine that the data to be written is not written to the target cache area.

[0110] For example, assuming that the amount of data to be written is 10KB, the amount of data stored in the target cache area is 11KB, and the preset data amount threshold is 20KB, at this time the sum of the amount of data stored in the target cache area and the amount of data to be written is greater than 20KB, and it is determined that the data to be written will not be written to the target cache area.

[0111] For another example, suppose that the amount of data to be written is 10KB, the amount of data stored in the target cache area is 10KB, and the preset data volume threshold is 20KB. At this time, the sum of the amount of data stored in the target cache area and the amount of data to be written is equal to 20KB, and it is determined that the data to be written will be written into the target cache area.

[0112] For another example, suppose that the amount of data to be written is 10KB, the amount of data stored in the target cache area is 9KB, and the preset data volume threshold is 20KB. At this time, the sum of the amount of data stored in the target cache area and the amount of data to be written is 19KB, which is less than 20KB. It is determined that the data to be written will be written into the target cache area.

[0113] In a second possible implementation, if the number of second cache intervals included in the target cache area is not greater than a preset interval number threshold, the data to be written is determined to be written into the target cache area; otherwise, the data to be written is determined not to be written into the target cache area.

[0114] For example, the number of second cache intervals included in the target cache area is 5, and the preset interval number threshold is 10. At this time, the number of second cache intervals included in the target cache area is not greater than 10, and it is determined that the data to be written is written into the target cache area.

[0115] After determining that the data to be written is to be written into the target cache area corresponding to the file identifier, first, based on the area starting position and offset of the target cache area, the area starting position of the first cache interval corresponding to the data to be written is determined within the target cache area; secondly, based on the amount of data to be written, the area ending position of the first cache interval corresponding to the data to be written is determined within the target cache area; then, based on the area starting position and the area ending position of the first cache interval, the first cache interval corresponding to the data to be written is determined.

[0116] For example, assuming that the offset is 2KB, the amount of data to be written is 3KB, and the area starting position of the target cache area is 0KB, first, based on the area starting position of the target cache area and the offset 2KB, the area starting position of the first cache interval corresponding to the data to be written is determined to be 2KB in the target cache area. Secondly, based on the amount of data to be written 3KB, the area ending position of the first cache interval corresponding to the data to be written is determined to be 5KB in the target cache area. Then, based on the area starting position of the first cache interval and the area ending position of the first cache interval, the first cache interval corresponding to the data to be written is determined to be [2, 5).

[0117] It should be noted that, in the embodiment of the present application, a left-closed and right-open interval may be used to represent the first cache interval and the second cache interval. For example, the first cache interval [2, 5) represents 2 to 4 KB.

[0118] A second possible situation: the data write request does not carry the amount of data to be written.

[0119] Using the second possible implementation method mentioned above, after determining to write the data to be written into the target cache area corresponding to the file identifier, first, based on the area starting position and offset of the target cache area, the area starting position of the first cache interval corresponding to the data to be written is determined within the target cache area. Secondly, based on the preset interval length, the area ending position of the first cache interval corresponding to the data to be written is determined within the target cache area. Then, based on the area starting position and the area ending position of the first cache interval, the first cache interval corresponding to the data to be written is determined.

[0120] For example, assuming the offset is 2KB, the preset interval length is 4KB, and the area starting position of the target cache area is 0KB, first, based on the area starting position of the target cache area and the offset 2KB, the area starting position of the first cache interval corresponding to the data to be written is determined to be 2KB in the target cache area. Secondly, based on the preset interval length of 4KB, the area ending position of the first cache interval corresponding to the data to be written is determined to be 6KB in the target cache area. Then, based on the area starting position of the first cache interval and the area ending position of the first cache interval, the first cache interval corresponding to the data to be written is determined to be [2, 6).

[0121] S303: If it is determined that there is an overlapping area between the first cache interval and the at least one second cache interval, the first cache interval and the at least one second cache interval are merged to obtain a merged cache area, wherein each second cache interval is used to store other to-be-written data of the corresponding target file before the first cache interval is determined.

[0122] Specifically, any one or more combinations of, but not limited to, the following operations may be used to determine whether an overlapping area exists between the first cache interval and one of the at least one second cache intervals:

[0123] A first possible operation: if the region start position or the region end position of a second cache interval is located between the region start position and the region end position of the first cache interval, it is determined that there is an overlapping region between the first cache interval and the second cache interval.

[0124] For example, see Figure 4a As shown, the first cache interval is [2, 5), the second cache interval is [4, 7), and the area starting position of the second cache interval [4, 7) is located between the area starting position and the area ending position of the first cache interval [2, 5), determining that there is an overlapping area between the first cache interval [2, 5) and the second cache interval [4, 7).

[0125] For example, see Figure 4b As shown, the first cache interval is [2, 5), the second cache interval is [4, 5), and the area starting position of the second cache interval [4, 5) is located between the area starting position and the area ending position of the first cache interval [2, 5), and it is determined that there is an overlapping area between the first cache interval [2, 5) and the second cache interval [4, 5).

[0126] For example, see Figure 4cAs shown, the first cache interval is [2, 5), the second cache interval is [0, 4), and the area end position of the second cache interval [0, 4) is located between the area start position and the area end position of the first cache interval [2, 5), determining that there is an overlapping area between the first cache interval [2, 5) and the second cache interval [0, 4).

[0127] A second possible operation: if the region start position or the region end position of the first cache interval is between the region start position and the region end position of a second cache interval, it is determined that there is an overlapping region between the first cache interval and the second cache interval.

[0128] For example, see Figure 4d As shown, the first cache interval is [2, 5), the second cache interval is [0, 6), and the starting position of the area of ​​the first cache interval [2, 5) is located between the starting position and the ending position of the area of ​​the second cache interval [0, 6). It is determined that there is an overlapping area between the first cache interval [2, 5) and the second cache interval [0, 6).

[0129] A third possible operation: if the region start position and the region end position of the first cache interval are the same as the region start position and the region end position of a second cache interval, it is determined that there is an overlapping region between the first cache interval and the second cache interval.

[0130] For example, see Figure 4e As shown, the first cache interval is [2, 5), the second cache interval is [2, 5), the area starting position and the area ending position of the first cache interval [2, 5) are the same as the area starting position and the area ending position of the second cache interval [2, 5), and it is determined that there is an overlapping area between the first cache interval [2, 5) and the second cache interval [2, 5).

[0131] A fourth possible operation: if the region start position of the first cache interval is the same as the region end position of a second cache interval, it is determined that there is an overlapping region between the first cache interval and the second cache interval.

[0132] For example, see Figure 4f As shown, the first cache interval is [2, 5), the second cache interval is [0, 2), the starting position of the area of ​​the first cache interval [2, 5) is the same as the ending position of the area of ​​the second cache interval [2, 5), and it is determined that there is an overlapping area between the first cache interval [2, 5) and the second cache interval [0, 2).

[0133] A fifth possible operation: if the end position of the first cache interval is the same as the start position of the second cache interval, it is determined that there is an overlapping area between the first cache interval and the second cache interval.

[0134] For example, see Figure 4g As shown, the first cache interval is [2, 5), the second cache interval is [5, 7), the end position of the area of ​​the first cache interval [2, 5) is the same as the starting position of the area of ​​the second cache interval [5, 7), and it is determined that there is an overlapping area between the first cache interval [2, 5) and the second cache interval [5, 7).

[0135] After determining that there is an overlapping area between the first cache interval and at least one second cache interval, the merged cache area may be obtained by, but not limited to, the following methods:

[0136] First, based on the area starting positions of at least one second cache interval and the first cache interval, the area starting position of the at least one second cache interval and the first cache interval that has the smallest distance from the area starting position of the target cache area is used as the area starting position of the merged cache area.

[0137] For example, assume that the first cache interval is [2, 6), the at least one second cache interval includes the second cache interval [0, 4) and the second cache interval [5, 8), and the region starting position of the target cache area is 0. Based on the region starting positions of the at least one second cache interval and the first cache interval, the region starting position 0 of the second cache interval [0, 4), the second cache interval [5, 8), and the first cache interval [2, 6], which has the smallest distance from the region starting position of the target cache area, is used as the region starting position of the merged cache area.

[0138] Secondly, based on the area end position of at least one second cache interval and the first cache interval, the area end position of the at least one second cache interval and the first cache interval that is the closest to the area start position of the target cache area is used as the area end position of the merged cache area.

[0139] For example, the first cache interval is [2, 6), at least one second cache interval includes the second cache interval [0, 4) and the second cache interval [5, 8), the area starting position of the target cache area is 0, and according to the area ending positions of the second cache interval [0, 4), the second cache interval [5, 8) and the first cache interval [2, 6), the area ending position 8 with the largest distance from the area starting position of the target cache area in the second cache interval [0, 4), the second cache interval [5, 8) and the first cache interval [2, 6) is used as the area ending position of the merged cache area.

[0140] Then, based on the region start position of the merged buffer area and the region end position of the merged buffer area, a merged buffer area is obtained.

[0141] For example, see Figure 4h As shown, based on the region start position 0 of the merged buffer region and the region end position 8 of the merged buffer region, the merged buffer region [0, 8) is obtained.

[0142] S304: The merged cache area is used as a new second cache interval, and the data to be written is saved in the new second cache interval.

[0143] Furthermore, if it is determined that there is no overlapping area between the first cache interval and any second cache interval, the data to be written is saved in the first cache interval, and the first cache interval after the data is written is used as a second cache interval in the target cache area.

[0144] For example, assuming that the first cache interval is [13, 15), the second cache interval includes the second cache interval [0, 4), the second cache interval [5, 8), and the second cache interval [10, 12), and there is no overlapping area between the first cache interval [13, 15) and the second cache interval [0, 4), the second cache interval [5, 8), and the second cache interval [10, 12), then the data to be written is saved in the first cache interval [13, 15), and the first cache interval [13, 15) is used as a second cache interval [13, 15) in the target cache area.

[0145] To improve data writing efficiency, see Figure 5 As shown in FIG, a flow chart of storing data in an embodiment of the present application specifically includes the following processes:

[0146] First, determine whether to write the data to be written into the target cache area corresponding to the file identifier. If it is determined that the data to be written is to be written into the target cache area corresponding to the file identifier, execute S302-S304 to write the data to be written into the target cache area.

[0147] If it is determined that the data to be written will not be written into the target cache area corresponding to the file identifier, a new cache area is generated corresponding to the file identifier, and the new cache area is used as the new target cache area. Then, based on the area starting position and offset of the new target cache area, the first cache interval corresponding to the data to be written is determined in the new target cache area. Then, the data to be written is saved in the first cache interval in the new target cache area, and the first cache interval after the data is written is used as a second cache interval in the new target cache area.

[0148] After determining that the data to be written will not be written into the target cache area corresponding to the file identifier, the area identifier of the target cache area can also be added to the frozen queue to change the state information of the target cache area from a changeable state to an immutable state (immutable state), and the immutable state is used to characterize that the target cache area is not writable. Accordingly, before saving the data to be written into the first cache interval in the new target cache area, if the queue waiting time of the area identifier of the previous target cache area is not greater than the preset time threshold, it is determined that the state information of the previous target cache area is changed to an immutable state, and based on the area starting position and offset of the new target cache area, the first cache interval corresponding to the data to be written is determined in the new target cache area, and the data to be written is saved in the first cache interval in the new target cache area, and the first cache interval after the data is written is used as a second cache interval in the new target cache area. Among them, the queue waiting time is used to characterize the time spent by the area identifier of the previous target cache area in the buffer area of ​​the frozen queue.

[0149] For example, assuming the preset duration threshold is 10 seconds, upon receiving a data write request x, if it is determined that the data to be written carried in the data write request x will not be stored in the target storage area corresponding to the file identifier, a new cache area is generated corresponding to the file identifier and the new cache area is used as the new target cache area. Simultaneously, the area identifier IDx of the previous target cache area is added to the freeze queue. If the enqueuing wait time for IDx is no longer than 10 seconds, the status information of the previous target cache area is determined to be changed to an unmodifiable state, and the data to be written is saved to the first cache interval within the new target cache area.

[0150] If the enqueue wait time of the area identifier of the previous target cache area is greater than the preset time threshold, S302-S304 are executed again. If the number of data storage retries exceeds the preset number of write retries, the write is determined to have failed and a write failure indication is returned. For example, the preset number of write retries may be 3 times.

[0151] In order to further improve data storage efficiency, a more flexible index structure is provided in an embodiment of the present application, hereinafter referred to as a File Range Tree (FTR). In the FTR, the offset carried in the data write request is used as an index, and each index corresponds to a second cache interval. A node in the FTR represents a second cache interval. Each FRT corresponds to a target cache area. It should be noted that, in an embodiment of the present application, the second cache interval can also be referred to as a range. Below, FRT is used as the index structure for explanation.

[0152] FTR can be implemented based on a balanced binary search tree. For each data write request received, after determining the first cache interval corresponding to the data to be written in the target cache area, at least one second cache interval that overlaps with the first cache interval can be determined based on FRT. Specifically, first locate the range corresponding to the maximum offset smaller than the offset in the FRT, and use this range as the starting range. If not found, use the range with the minimum offset of the range tree and the leftmost leaf node as the starting range, and then use offset+size as the termination condition to traverse the FTR in order to complete the data overwriting and deletion of the old range and the merging and insertion of the new range.

[0153] The data storage process is described below using a specific embodiment as an example.

[0154] See Figure 6 As shown, the first data write request (referred to as Data Write Request 1) is received. The offset of the data to be written carried in Data Write Request 1 is 0 and the amount of data to be written is 3KB. When determining to write the data to be written to the target cache area corresponding to the file identifier, based on the area starting position, offset, and amount of data to be written of the target cache area, the first cache interval corresponding to the data to be written is determined to be [0, 4) within the target cache area, i.e., 0 to 3KB. At this time, the FRT table is empty, the first cache interval is used as a second cache interval [0, 4), and the data to be written is saved in the second cache interval [0, 4).

[0155] A second data write request (referred to as Data Write Request 2) is received. The offset of the data to be written carried in Data Write Request 2 is 5, and the amount of data to be written is 3KB. When determining to write the data to be written to the target cache area corresponding to the file identifier, the first cache interval corresponding to the data to be written is determined within the target cache area to be [5, 8), i.e., 5 to 7KB. At this point, a second cache area [0, 4) exists in the FRT. Since there is no overlap between the first cache area [5, 8) and the second cache area [0, 4), the first cache area [5, 8) is treated as a second cache area [5, 8), and the data to be written is saved in the second cache area [5, 8).

[0156] A third data write request (referred to as Data Write Request 3) is received. The offset of the data to be written carried in Data Write Request 3 is 10, and the amount of data to be written is 2KB. When determining to write the data to the target cache area corresponding to the file identifier, the first cache interval corresponding to the data to be written is determined within the target cache area to be [10, 12), i.e., 10 to 11KB. At this point, the FRT contains a second cache area [0, 4) and a second cache interval [5, 8). There is no overlap between the first cache interval [10, 12) and the second cache areas [0, 4) and [5, 8]. The first cache interval [10, 12) is treated as a second cache interval [10, 12), and the data to be written is stored in the second cache interval [10, 12]. Since the primary key value of each node in the FRT is greater than the primary key value of its left child and smaller than the primary key value of its right child, the second cache interval [5, 8) is designated as the parent node.

[0157] …

[0158] A fifth data write request (referred to as data write request 5) is received. The offset of the data to be written carried in data write request 5 is 2, and the amount of data to be written is 4KB. When determining to write the data to be written into the target cache area corresponding to the file identifier, the first cache interval corresponding to the data to be written is determined to be [2, 6) within the target cache area, i.e., 2 to 5KB. At this time, the FRT contains the second cache area [0, 4), the second cache interval [5, 8), the second cache interval [10, 12), and the second cache interval [13, 15). There is an overlapping area between the first cache interval [2, 6) and the second cache area [0, 4) and the second cache interval [5, 8). Therefore, the first cache interval [2, 6) and the second cache area [0, 4) and the second cache interval [5, 8) are merged to obtain a merged cache area [0, 8). Then, the merged cache area [0, 8) is used as the new second cache area [0, 8), and the data to be written is saved in the new second cache area [0, 8). In order to keep the FRT balanced and reduce the time complexity of the tree operations, it is also necessary to adjust the tree structure by rotation. The adjusted FRT can be found in Figure 6 shown.

[0159] In order to maintain data consistency between the cache and the preset storage area, after the data to be written is saved in the target cache area, the data in the cache needs to be synchronized to the set first storage location. Figure 7FIG2 is a schematic diagram of a possible cache and first storage location provided in an embodiment of the present application, wherein the first storage location can be deployed in a device such as a hard disk or flash memory, or can be deployed in each storage node of a distributed cloud storage system. Exemplarily, the first storage location can use an object storage service (Cloud Object Storage, COS).

[0160] See Figure 8 As shown, it is a schematic diagram of the process of synchronizing cached data in an embodiment of the present application, which specifically includes the following processes:

[0161] First, after determining to write the data in the target cache area back to the first storage location, a new target cache area is generated corresponding to the file identifier of the target file, and the area identifier of the previous target cache area is added to the freezing queue.

[0162] As an example, after receiving the synchronization instruction, the data in the target cache area may be written back to a designated first storage location, where the first storage location may refer to an external storage device or a storage node.

[0163] As another example, the data in the target cache area is written back to the designated first storage location according to a set synchronization frequency. For example, the set synchronization frequency may be 30 seconds / time.

[0164] Secondly, it is determined whether the queue waiting time of the region identifier of the previous target cache region is greater than a preset time threshold. If the queue waiting time of the region identifier of the previous target cache region is not greater than the preset time threshold, it is further determined whether the metadata information of the previous target cache region is saved to the second storage location within the set submission time.

[0165] If the metadata information of the previous target cache area is saved to the second storage location within the set commit time, synchronization is determined to be successful. Otherwise, the cache data synchronization process described above is re-executed. If the number of data synchronization retries exceeds the preset number of synchronization retries, the write is determined to have failed and a synchronization failure indication is returned. For example, the preset number of synchronization retries can be 2.

[0166] In a file system, the target cache area contains not only the target file's file data but also metadata information. This metadata information includes, but is not limited to, any one or more of the following: the size of the target cache area, the physical storage address corresponding to each second cache interval in the target cache area, and access time. To maintain the consistency of the metadata information, after the data to be written is stored in the target cache area, the metadata information in the cache needs to be synchronized to a designated second storage location. The second storage location is used to manage the metadata information. Exemplarily, the second storage location is deployed in a metadata management module in a distributed cloud storage system.

[0167] Considering the situation of receiving multiple data write requests, in order to ensure the order of the multiple data write requests, after the new cache area is set as the new target cache area, and before the data to be written is saved in the first cache interval within the new target cache area, the following steps are further included:

[0168] Generate a sequence number corresponding to a new target cache area, where the sequence number is used to indicate the order in which metadata information of the new target cache area is saved;

[0169] After the first cache interval after the data is written is used as a second cache interval in the new target cache area, the method further includes:

[0170] generating metadata information of the new target cache area according to each second cache interval included in the new target cache area, wherein the metadata information at least includes a physical storage address corresponding to each second cache interval in the new target cache area;

[0171] The metadata information of the new target cache area is saved to the second storage location according to the sequence number.

[0172] For example, the metadata information of the new target cache area can be saved to the second storage location in sequence according to the sequence number based on the minimum heap. Among them, the ID is used to represent the sequence number corresponding to the target cache area, the key of each node of the minimum heap is the ID, and the value is the metadata information of the target cache area, such as the metadata information of the FRT. Specifically, the following steps are performed:

[0173] (1) Determine whether the ID of the top of the current minimum heap matches the logical ID of the expected order (expect_commit_id). If so, execute step (2); if not, execute step (3).

[0174] (2) Save the metadata information of the top of the current minimum heap to the second storage location, mark the metadata information submission completed, increase expect_commit_id by 1, then delete the metadata information at the top of the heap, and determine whether the minimum heap is empty. If it is empty, execute step (3); if it is not empty, repeat step (1);

[0175] (3) Wait for the next heap operation.

[0176] For example, see Figure 9a As shown, the IDs of the nodes in the current minimum heap are 1, 2, 3, and 4 respectively, where ID1, ID2, ID3, and ID4 correspond to the metadata information of FRT1, FRT2, FRT3, and FRT4 respectively. Assuming that the logical ID of the expected order is 1 and the ID of the top of the current minimum heap is ID1, which meets the logical ID of the expected order, the metadata information of FRT1 is saved to the second storage location, and then the metadata information of FRT1 is deleted. The minimum heap after deleting the metadata information of FRT1 is shown in FIG. Figure 9b shown.

[0177] In an embodiment of the present application, to facilitate fine-grained data block control, data may be stored in the first storage location in the form of data blocks. As a possible implementation, after writing the data to be written to the target cache area, the stored data in the target cache area may be segmented based on a preset data block size, and then at least one data block may be written back to the first storage location.

[0178] As another possible implementation, the data to be written can be segmented based on a preset data block size before being written to the target cache area. The segmented data to be written is then written to the target cache area. For example, the FRT used in this method is named FBRT (File Block Range Tree).

[0179] Based on the index structure of the FRT, an embodiment of the present application provides a possible data reading method. When a data read request for a target file is received, the method determines whether the data to be read exists in the FRT corresponding to the file identifier of the target file based on the offset of the data to be read and the file identifier carried in the data read request. If the data to be read exists in the FRT, the data to be read is read from the FRT; otherwise, the data to be read is read from the first storage location.

[0180] Based on the same inventive concept as the above method embodiment, the present application embodiment also provides a schematic diagram of the architecture of a data storage device, see Figure 10As shown, the data storage device 1000 includes a receiving unit 1001 and a processing unit 1002, wherein the processing unit 1002 includes a determining unit 1002a, a merging unit 1002b and a saving unit 1002c.

[0181] The receiving unit 1001 is configured to receive a data write request triggered for a target file, wherein the data write request carries at least the data to be written, the file identifier, and the offset of the corresponding target file;

[0182] A determining unit 1002a is configured to determine, when determining to write the data to be written into the target cache area corresponding to the file identifier, a first cache interval corresponding to the data to be written within the target cache area based on the area start position and offset of the target cache area;

[0183] a merging unit 1002b configured to merge the first cache interval and the at least one second cache interval to obtain a merged cache area if an overlapping area is determined between the first cache interval and the at least one second cache interval; wherein each second cache interval is used to store other to-be-written data of the corresponding target file before the first cache interval is determined;

[0184] The saving unit 1002c is configured to use the merged cache area as a new second cache interval and save the data to be written into the new second cache interval.

[0185] Optionally, the data write request also carries the amount of data to be written;

[0186] When determining to write the data to be written into the target cache area corresponding to the file identifier, the determining unit 1002a is configured to:

[0187] If the sum of the amount of data stored in the target cache area and the amount of data to be written is not greater than the preset data amount threshold, then determining to write the data to be written into the target cache area; or

[0188] If the number of second cache intervals included in the target cache area is not greater than a preset interval number threshold, it is determined to write the data to be written into the target cache area.

[0189] Optionally, when determining the first cache interval corresponding to the to-be-written data in the target cache area based on the area start position and the offset of the target cache area, the determining unit 1002a is configured to:

[0190] Determining, within the target cache area, a region start position of a first cache interval corresponding to the data to be written based on the region start position and the offset of the target cache area;

[0191] Based on the amount of data to be written, determining, within the target cache area, an end position of a first cache interval corresponding to the data to be written;

[0192] The first cache interval corresponding to the to-be-written data is determined based on a region start position of the first cache interval and a region end position of the first cache interval.

[0193] Optionally, when it is determined that there is an overlapping area between the first cache interval and one of the at least one second cache intervals, the determining unit 1002a is configured to perform any one or more combinations of the following operations:

[0194] If the region start position or the region end position of the first cache interval is between the region start position and the region end position of a second cache interval, it is determined that there is an overlapping region between the first cache interval and the second cache interval;

[0195] If the region start position or the region end position of a second cache interval is between the region start position and the region end position of the first cache interval, it is determined that there is an overlapping region between the first cache interval and the second cache interval;

[0196] If the region start position and the region end position of the first cache interval are the same as the region start position and the region end position of a second cache interval, it is determined that there is an overlapping region between the first cache interval and the second cache interval;

[0197] If the starting position of the first cache interval is the same as the ending position of the second cache interval, it is determined that there is an overlapping area between the first cache interval and the second cache interval;

[0198] If the end position of the first cache interval is the same as the start position of the second cache interval, it is determined that there is an overlapping area between the first cache interval and the second cache interval.

[0199] Optionally, when merging the first cache interval and at least one second cache interval to obtain a merged cache area, the merging unit 1002b is configured to:

[0200] Based on the region starting positions of the at least one second cache interval and the first cache interval, the region starting position of the at least one second cache interval and the first cache interval having the smallest distance from the region starting position of the target cache area is used as the region starting position of the merged cache area;

[0201] Based on the region end positions of the at least one second cache interval and the first cache interval, the region end position of the at least one second cache interval and the first cache interval that has the largest distance from the region start position of the target cache area is used as the region end position of the merged cache area;

[0202] A merged buffer area is obtained based on a region start position of the merged buffer area and a region end position of the merged buffer area.

[0203] Optionally, the data storage device 1000 further includes a thread pool unit 1004, which is used to:

[0204] Splitting the stored data in the target cache area according to a preset data block size to obtain at least one data block;

[0205] At least one data block is written back to the designated first storage location.

[0206] Optionally, the data storage device 1000 further includes an allocation unit 1003;

[0207] The allocating unit 1003 is configured to generate a new cache area corresponding to the file identifier if it is determined that the data to be written is not to be written into the target cache area corresponding to the file identifier, and use the new cache area as a new target cache area;

[0208] The determining unit 1002a is further configured to determine a first cache interval corresponding to the to-be-written data in the new target cache area based on the area start position and offset of the new target cache area;

[0209] The saving unit 1002c is further configured to save the data to be written into the first cache interval in the new target cache area, and use the first cache interval after the data is written as a second cache interval in the new target cache area.

[0210] Optionally, the data storage device 1000 further includes a submitting unit 1005;

[0211] After the new cache area is used as the new target cache area, the allocating unit 1003 is further configured to:

[0212] Generate a sequence number corresponding to a new target cache area, where the sequence number is used to indicate the order in which metadata information of the new target cache area is saved;

[0213] After the first cache interval after the data is written is used as a second cache interval in the new target cache area, the determining unit 1002a is further configured to:

[0214] generating metadata information of the new target cache area according to each second cache interval included in the new target cache area, wherein the metadata information at least includes a physical storage address corresponding to each second cache interval in the new target cache area;

[0215] The submitting unit 1005 is configured to save the metadata information of the new target cache area to a designated second storage location according to the sequence number.

[0216] Below, the interaction between the modules of the data storage device provided in the embodiment of the present application is described with reference to a specific embodiment.

[0217] See Figure 11 As shown, the data storage device 1100 includes the following modules: a receiving unit 1101, a processing unit 1102, an allocation unit 1103, a thread pool unit 1104 and a submission unit 1105. Among them, the receiving unit 1101 can also be called an ID allocator (Allocator), the thread pool unit 1104 can also be called a thread pool (Flush Worker), and the submission unit 1105 can also be called a committer. The first storage location is deployed in a storage node in the distributed cloud storage system, and the second storage location is deployed in a metadata management module in the distributed cloud storage system. It should be noted that the processing unit 1102 includes a determination unit, a merging unit, and a saving unit. Figure 11 Not shown in the figure.

[0218] First, after the receiving unit 1101 receives a data write request for the target file (referred to as request 1), the processing unit 1102 determines to write the data 1 to be written carried in request 1 into the target cache area corresponding to the file identifier (referred to as area 1), and then executes S302-S304 to save the data 1 to be written into area 1. The details are not repeated here.

[0219] See Figure 11 As shown, area 1 includes the second cache interval [0, 8), the second cache interval [10, 12), and the second cache interval [13, 15), namely 0-7KB, 10-11KB, and 13-14KB. After the receiving unit 1101 receives another data write request for the target file (referred to as request 2), when the processing unit 1102 determines not to write the to-be-written data 2 carried in request 2 into the target cache area (referred to as area 1) corresponding to the file identifier, the area identifier of area 1 is added to the freeze queue to change the status information of area 1 from a modifiable state to an unmodifiable state.

[0220] The allocator generates a new cache area corresponding to the region identifier of the target file and uses the new cache area as the new target cache area (referred to as region 2). Simultaneously, processing unit 1102 sends region 1 in the immutable state to the flush worker. The flush worker saves the data in region 1 to the first storage location and then sends the metadata information for region 1 to the committer. The committer commits the metadata information for region 1 to the second storage location.

[0221] During the process of the Flush Worker saving the data in area 1 to the first storage location, or the process of the committer saving the metadata information of area 1 to the second storage location, the processing unit 1102 can save the data to be written 2 to area 2. The process of saving the data to be written to area 2 is not repeated here.

[0222] In this way, users do not need to wait for all the data in the target cache area to be saved to the first storage location and the metadata information to be submitted to the second storage location before the next write, which greatly improves the write efficiency. In addition, the concurrency of flushing data can be controlled by Flush Worker and can be adjusted at any time, which is more flexible.

[0223] Furthermore, because the metadata of multiple immutable target cache areas arrives at the committer at random times, to ensure the order of data write requests, the allocator can assign a logical ID to each newly generated target cache area as a sequence number. This logical ID can be auto-incremental. Accordingly, the committer commits the metadata of the target cache areas to the secondary storage location for storage in the order of the logical IDs.

[0224] After introducing the data storage method and apparatus according to an exemplary embodiment of the present application, an electronic device according to another exemplary embodiment of the present application is introduced next.

[0225] Those skilled in the art will appreciate that various aspects of the present application can be implemented as systems, methods, or program products. Therefore, various aspects of the present application can be specifically implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation that combines hardware and software aspects, which may be collectively referred to herein as a "circuit," "module," or "system."

[0226] Based on the same inventive concept as the above method embodiment, an electronic device is also provided in the embodiment of the present application, Figure 12 As shown, the electronic device 1200 may include at least a processor 1201 and a memory 1202. The memory 1202 stores program codes, and when the program codes are executed by the processor 1201, the processor 1201 performs the steps of any one of the above data storage methods.

[0227] In some possible implementations, the computing device according to the present application may include at least one processor and at least one memory. The memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the data storage method according to various exemplary embodiments of the present application described above in this specification. For example, the processor may execute the following steps: Figure 3 、 Figure 5 or Figure 8 Follow the steps shown in .

[0228] Based on the same inventive concept as the above-mentioned method embodiment, various aspects of the data storage method provided by the present application can also be implemented in the form of a program product, which includes program code. When the program product is run on an electronic device, the program code is used to enable the electronic device to execute the steps of the data storage method according to various exemplary embodiments of the present application described above in this specification. For example, the electronic device can execute the following steps: Figure 3 、 Figure 5 or Figure 8 Follow the steps shown in .

[0229] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0230] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0231] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A data storage method, characterized in that: include: Receive a data write request triggered for a target file, wherein the data write request carries at least the to-be-written data, the file identifier, and the offset corresponding to the target file; When determining to write the data to be written into the target cache area corresponding to the file identifier, determining a first cache interval corresponding to the data to be written within the target cache area based on the area start position of the target cache area and the offset; If it is determined that there is an overlapping area between the first cache interval and at least one second cache interval, determining the regional starting position and regional ending position of a merged cache area based on the distance between the respective regional starting position and regional ending position of the at least one second cache interval and the first cache interval and the regional starting position of the target cache area; merging the first cache interval and the at least one second cache interval based on the regional starting position and regional ending position of the merged cache area to obtain the merged cache area; wherein each second cache interval is used to store other to-be-written data corresponding to the target file before the first cache interval is determined; The merged cache area is used as a new second cache interval, and the data to be written is saved in the new second cache interval.

2. The method according to claim 1, wherein The data writing request also carries the amount of data to be written; The determining to write the to-be-written data into the target cache area corresponding to the file identifier includes: If the sum of the amount of data stored in the target cache area and the amount of data to be written is not greater than a preset data amount threshold, determining to write the data to be written into the target cache area; or, If the number of second cache intervals included in the target cache area is not greater than a preset interval number threshold, it is determined to write the to-be-written data into the target cache area.

3. The method according to claim 2, wherein Determining a first cache interval corresponding to the to-be-written data within the target cache area based on the area starting position of the target cache area and the offset includes: Determining, within the target cache area, a region starting position of a first cache interval corresponding to the to-be-written data based on a region starting position of the target cache area and the offset; Based on the amount of data to be written, determining, within the target cache area, an end position of a first cache interval corresponding to the data to be written; The first cache interval corresponding to the to-be-written data is determined based on a region start position of the first cache interval and a region end position of the first cache interval.

4. The method according to claim 1, wherein Determining that there is an overlapping area between the first cache interval and one of the at least one second cache interval includes any one or more combinations of the following operations: If the region start position or the region end position of the first cache interval is between the region start position and the region end position of the second cache interval, it is determined that there is an overlapping region between the first cache interval and the second cache interval; If the region start position or the region end position of the second cache interval is between the region start position and the region end position of the first cache interval, it is determined that there is an overlapping region between the first cache interval and the second cache interval; If the region start position and the region end position of the first cache interval are the same as the region start position and the region end position of the second cache interval, it is determined that there is an overlapping area between the first cache interval and the second cache interval; If the starting position of the first cache interval is the same as the ending position of the second cache interval, it is determined that there is an overlapping area between the first cache interval and the second cache interval; If the end position of the first cache interval is the same as the start position of the second cache interval, it is determined that there is an overlapping area between the first cache interval and the second cache interval.

5. The method according to claim 1, wherein Determining the regional start position and the regional end position of the merged cache area according to the distance between the regional start position and the regional end position of each of the at least one second cache interval and the first cache interval and the regional start position of the target cache area includes: Based on the area starting positions of the at least one second cache interval and the first cache interval, the area starting position of the at least one second cache interval and the first cache interval that has the smallest distance from the area starting position of the target cache area is used as the area starting position of the merged cache area; According to the area end positions of the at least one second cache interval and the first cache interval, the area end position of the at least one second cache interval and the first cache interval, whichever is farther from the area start position of the target cache area, is used as the area end position of the merged cache area.

6. The method according to any one of claims 1 to 5, wherein: After the data to be written is saved in the new second cache interval, the method further includes: Splitting the stored data in the target cache area according to a preset data block size to obtain at least one data block; The at least one data block is written back to the designated first storage location.

7. The method according to any one of claims 1 to 5, wherein: Also includes: If it is determined that the data to be written is not to be written into the target cache area corresponding to the file identifier, a new cache area is generated corresponding to the file identifier, and the new cache area is used as a new target cache area; Determining a first cache interval corresponding to the to-be-written data within the new target cache area based on the area starting position of the new target cache area and the offset; The data to be written is saved in the first cache interval in the new target cache area.

8. A data storage device, characterized in that The data storage device comprises: A receiving unit, configured to receive a data write request triggered for a target file, wherein the data write request carries at least the data to be written, a file identifier, and an offset corresponding to the target file; a determining unit, configured to determine, when determining to write the data to be written into the target cache area corresponding to the file identifier, a first cache interval corresponding to the data to be written within the target cache area based on a starting position of the target cache area and the offset; a merging unit configured to, if it is determined that there is an overlapping area between the first cache interval and at least one second cache interval, determine a region start position and a region end position of a merged cache area based on the distance between the region start position and the region end position of each of the at least one second cache interval and the first cache interval and the region start position of the target cache area; and merge the first cache interval and the at least one second cache interval based on the region start position and the region end position of the merged cache area to obtain the merged cache area; wherein each second cache interval is used to store other to-be-written data corresponding to the target file before the first cache interval is determined; A saving unit is configured to use the merged cache area as a new second cache interval and save the to-be-written data into the new second cache interval.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor executes the steps of any one of the methods of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The method comprises a program code, and when the program code is run on an electronic device, the program code is used to enable the electronic device to execute the steps of any one of the methods of claims 1 to 7.

Images