Method, apparatus, and computer program product for managing a storage system

Abstract

Embodiments of the present disclosure relate to a method, an electronic device and a computer program product for managing a storage system. The method comprises, in response to a first write operation on a first data block on a persistent storage device, determining whether a first set of data corresponding to the first data block is included in a cache; if it is determined that the first set of data is included in the cache, updating the first set of data in the cache; and adding the first set of data as a first record in an associated data set of the cache. Embodiments of the present disclosure enable managing different types of cache data corresponding to data blocks in association, thereby optimizing system performance.

Description

Technical Field

[0001] Embodiments of this disclosure generally relate to the field of data storage, and more specifically to methods, apparatus, and computer program products for managing storage systems. Background Technology

[0002] Persistent storage devices in a storage system typically store large amounts of data. For example, they often store substantial amounts of metadata and user data. Storage systems typically utilize caching to process the data stored in persistent storage. During operation, the most frequently used data is temporarily stored in the cache to avoid redundant creation, processing, and transmission. By using caching to handle the most frequently used data, resources can be saved, effectively improving system performance. Therefore, effective cache management is a crucial issue. Summary of the Invention

[0003] Embodiments of this disclosure provide methods, apparatus, and computer program products for managing storage systems.

[0004] In a first aspect of this disclosure, a method for managing a storage system is provided. The method includes, in response to a first write operation on a first data block on a persistent storage device, determining whether a cache contains a first set of data corresponding to the first data block; if it is determined that the cache contains the first set of data, updating the first set of data in the cache; and adding the first set of data as a first record to an associated dataset of the cache.

[0005] In a second aspect of this disclosure, an electronic device is provided. The electronic device includes at least one processing unit and at least one memory. The at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the electronic device to perform an action including, in response to a first write operation on a first data block on a persistent storage device, determining whether a cache contains a first set of data corresponding to the first data block; if it is determined that the cache contains the first set of data, updating the first set of data in the cache; and adding the first set of data as a first record to an associated dataset in the cache.

[0006] In a third aspect of this disclosure, a computer program product is provided. The computer program product is tangibly stored in a non-transitory computer storage medium and includes machine-executable instructions. When executed by a device, the machine-executable instructions cause the device to perform any step of the method described in the first aspect of this disclosure.

[0007] The summary section is provided to present the chosen concepts in a simplified form, which will be further described in the detailed description below. The summary section is not intended to identify key or essential features of this disclosure, nor is it intended to limit the scope of this disclosure. Attached Figure Description

[0008] The above and other objects, features and advantages of this disclosure will become more apparent from the accompanying drawings, in which like reference numerals generally denote like parts.

[0009] Figure 1 A schematic diagram of an example system that can be implemented therein according to some embodiments of the present disclosure is shown;

[0010] Figure 2 A schematic diagram of an example architecture for caching according to some embodiments of the present disclosure is shown;

[0011] Figure 3 A schematic diagram illustrating an example method for managing a storage system according to some embodiments of the present disclosure is shown;

[0012] Figure 4 An example of updating the associated dataset 220 according to some embodiments of this disclosure is shown;

[0013] Figure 5 Examples of frequency information according to some embodiments of this disclosure are shown;

[0014] Figure 6 Examples of clue information according to some embodiments of this disclosure are shown;

[0015] Figure 7 Examples of grouping data in a record according to some embodiments of this disclosure are shown;

[0016] Figure 8 Example APIs according to some embodiments of this disclosure are shown;

[0017] Figure 9 Example handles according to some embodiments of this disclosure are shown;

[0018] Figure 10 Example procedures for the UMCOpenSG function according to some embodiments of this disclosure are shown;

[0019] Figure 11 An example procedure for the UMCCloseSG function according to some embodiments of this disclosure is shown;

[0020] Figure 12A schematic block diagram of an example device that can be used to implement embodiments of the present disclosure is shown.

[0021] In the various figures, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0022] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0023] The term "comprising" and its variations as used herein signify open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0024] As mentioned above, persistent storage devices in a storage system store a large amount of metadata and user data. Storage systems typically utilize caching to process the data stored in persistent storage devices. During use, the most frequently accessed data is temporarily stored in the cache to avoid repeated creation, processing, and transmission of data. The cache includes different types of caches for various types of data, such as metadata.

[0025] In conventional approaches, different types of caches are stored in different locations within the cache, such as storage tiers. However, this conventional approach lacks management of different types of caches. For example, each type of cache has its own storage budget, and conventional approaches struggle to achieve an optimal balance between different types of caches. This leads to a waste of cache resources, which in turn limits system performance.

[0026] On the other hand, performing operations on data typically requires accessing different cache types, i.e., different cache layers. There are scenarios where cached data of certain cache types can be found in their corresponding cache layers, while cached data of other types cannot. In such cases, it's necessary to wait for the cache to prepare all the corresponding cached data before the operation can be completed. Therefore, this conventional approach increases the waiting time for data operations, impacting system performance.

[0027] Embodiments of this disclosure propose a scheme for managing a storage system to address one or more of the aforementioned problems and other potential issues. In this scheme, a set of data corresponding to a single data operation on a data block (or data) on a persistent storage device is added as a record to an associated dataset in the cache. In this way, a set of cached data of different cache types corresponding to a single data operation on data on the persistent storage device can be added as a record to the associated dataset. Thus, the set of cached data of different cache types can be managed in association based on records in the associated dataset. This improves cache utilization and enhances the data processing performance of the storage system.

[0028] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0029] Figure 1 A schematic diagram of a storage system 100 in which embodiments of the present disclosure may be implemented is shown. The storage system 100 is used to provide data storage related tasks, including tasks such as storage, data access, and data protection (e.g., deduplication, backup, encryption, decryption, etc.). It should be understood that... Figure 1 The system shown is just an example.

[0030] like Figure 1 As shown, storage system 100 includes cache 110 and persistent storage device 120. It should be understood that, although... Figure 1 Only one cache 110 and one persistent storage device 120 are shown in the diagram, but this is merely illustrative. The storage system 100 may include any number of caches 110 and any number of persistent storage devices 120.

[0031] Persistent storage device 120 stores a large amount of data, such as metadata. When a user client accesses (such as reads or writes) data (or data blocks) in persistent storage device 120, it retrieves the data (or data blocks) by accessing cache 110. The cache 110 contains a set of cached data corresponding to the data (or data blocks) to be operated on in persistent storage device 120, which can belong to different cached data types. Common cached data types include, but are not limited to: Block Index (IB), Block Validation Method (BMD), and Usage Status (CG).

[0032] When data in persistent storage device 120 needs to be changed, storage system 100 maintains consistency between cache 110 and persistent storage device 120 through atomic operations (also known as operations or transactions).

[0033] Figure 2An example architecture diagram of cache 110 according to some embodiments of the present disclosure is shown. Figure 2 The cache 110 includes a dataset 210, which may be a dataset using the Least Recently Used (LRU) algorithm. The dataset 210 includes associated datasets 220 and non-associated datasets 230. The associated datasets 220 and non-associated datasets 230 may include various types of cached data as described above.

[0034] Each data point in the associated dataset 220 and the unassociated dataset 230 is associated with frequency information 240. Frequency information 240 includes frequency information associated with each data point. Frequency information can be determined, for example, based on records of data access over a recent period. This will be discussed in conjunction with... Figure 5 To describe the frequency information in more detail 240.

[0035] Figure 2 The diagram also shows a real cache pool 250. The individual data in the associated dataset 220 and the non-associated dataset 230, as well as information such as frequency information 240, are actually stored in their corresponding physical addresses within the real cache pool 250. The real cache pool 250 includes thread information 252 for different data types. Thread information 252 can include various information such as FSID, FSBN, and offset, which can identify the data. This will be discussed in conjunction with... Figure 6 A more detailed description of clue information 252 for the real cache pool 250.

[0036] Figure 2 The document also shows how to look up HASH 260 and aggregate HASH 270. (For example...) Figure 2 As shown, both lookup hash 260 and aggregate hash 270 are associated with the real cache pool 250. Each piece of data in the associated dataset 220 and the non-associated dataset 230 is associated with lookup hash 260. By using lookup hash 260, based on information that identifies the data such as FSID, FSBN, and offset, a hash algorithm is used to find the corresponding data in the associated dataset 220 and the non-associated dataset 230. Each piece of data in the associated dataset 220 and the non-associated dataset 230 is associated with aggregate hash 270. This will be discussed in conjunction with... Figure 7 A more detailed description of the aggregated hash 270.

[0037] It should be understood that Figure 2 The example architecture of cache 110 shown is merely exemplary. In some embodiments, cache 110 may have different architectures and may include more or fewer components.

[0038] Figure 3A flowchart of an example method 300 for managing a storage system 100 according to some embodiments of the present disclosure is shown. Method 300 may be, for example, by... Figure 1 The method is executed by the storage system 100 shown. It should be understood that method 300 can also be executed by other suitable devices or apparatuses. Method 300 may include additional actions not shown and / or the actions shown may be omitted; the scope of this disclosure is not limited in this respect. The following is in conjunction with… Figure 1 and Figure 2 Let me describe method 300 in detail.

[0039] like Figure 3 As shown, at 310, a first write operation is performed on a first data block on persistent storage device 120. For example, the first write operation can be performed on the first data block (or data) on persistent storage device 120 to modify the first data block.

[0040] At 320, it is determined whether cache 110 includes the first set of data corresponding to the first data block. For example, a lookup hash 260 can be used to search the actual cache pool 250 of cache 110 for the first set of data based on information such as the FSID, FSBN, or offset of the first data block. The first set of data can include various types of cached data, such as one or more types of cached data including IB, BMD, CG, etc.

[0041] In some embodiments, if it is determined at 320 that the cache 110 does not include a first set of data corresponding to the first data block, then the first set of data is added to the non-associative dataset 230 in the cache 110. Each piece of data in the non-associative dataset 230 is stored independently. In some embodiments, the data in the non-associative dataset 230 does not overlap with the data in the records of the associated dataset 220. In some embodiments, the non-associative dataset 230 may be a non-associative data queue. Each piece of data in the first set may be added sequentially to the end of the non-associative data queue. It should be understood that the non-associative dataset 230 may also employ other types of data structures, such as lists, stacks, etc.

[0042] If it is determined at 320 that cache 110 contains the first set of data corresponding to the first data block, then method 300 proceeds to 330. At 330, the first set of data in the cache is updated. For example, the first set of data in cache 110 is updated individually to ensure data consistency in cache 110.

[0043] At position 340, the first set of data is added as the first record in the associated dataset 220 of cache 110. It should be understood that... Figure 3Steps 330 and 340 shown can be executed in parallel or sequentially in any order. In some embodiments, the associated dataset 220 can be an associated data queue. A first set of data can be added as the first record to the end of the associated data queue. Additionally and alternatively, the individual records in the associated dataset 220 can also be a record queue. It should be understood that the associated dataset 220 and the records can also employ other types of data structures, such as lists, stacks, etc.

[0044] In some embodiments, the storage system 100 may further determine whether existing data in the associated dataset 220 includes first data in the first data group. Data in the existing data in the associated dataset 220 is updated in the cache 110 in response to a second write operation to a second data block on the persistent storage device 120. The second write operation is any write operation that occurs prior to the first write operation. If it is determined that the existing data includes the first data, the first data can be removed from the existing data.

[0045] Additionally or alternatively, in some embodiments, the storage system 100 may also determine whether the unrelated dataset 230 includes the first data from the first set of data. If it is determined that the unrelated dataset 230 includes the first data, then the first data may be removed from the unrelated dataset 230.

[0046] Figure 4 The example shown is adding the first set of data as the first record to the associated dataset 220. Figure 4 The associated dataset 220 originally included records 410-1, 420-2, 410-3, and 410-4. It should be understood that... Figure 4 Different shapes are used to represent different types of cached data. For example, circles, squares, pentagons, and hexagons can represent IB, BMD, CG, and other cached data, respectively. It should be understood that these different shapes can also be used to represent other types of cached data. Figure 4 As described above, record 410-1 includes 6 data items, and the others include data A; record 410-2 includes 5 data items, and the others include data B and D; record 410-3 includes 6 data items; and record 410-4 includes 8 data items, and the others include data C. As shown in the figure, each record in the associated dataset 220 can include different numbers of cached data of various types.

[0047] When the first set of data 420 is added to the associated dataset 220, the resulting associated dataset 220 can include records 430-1, 430-2, 430-3, 430-4, and 430-5. The first set of data 420 can include data of different types A, B, C, D, E, F, G, and H. The first set of data 420 can be added to the associated dataset 220 as a new record 430-5. For example... Figure 4 As shown, after the first set of data 420 is added to the associated dataset 220, the data A, B, C and D in the original records 410-1, 410-2, 410-3 and 410-4 that were included in the new record 430-5 are removed.

[0048] It should be understood that, although Figure 4 The unrelated dataset 230 is not shown, but one or more of the data E, F, G, and H from the first set of data 420 that were not included in the original records 410-1, 410-2, 410-3, and 410-4 may be included in the unrelated dataset 230. If, for example, data E is included in the unrelated dataset 230, then data E will be removed from the unrelated dataset 230.

[0049] In the above manner, a set of data corresponding to a single write operation on persistent storage device 120 in cache 110 can be added to the associated dataset 220. This allows for the unified association of related sets of data in cache 110, thereby improving the cache hit rate during data operations. This avoids resource waste and improves system data processing performance.

[0050] In some embodiments, if the cache space of the associated dataset 220 is insufficient, a second record in the associated dataset 220 can be removed. This second record is the earliest data record added to the associated dataset 220. For example, if the associated dataset 220 is an associated data queue, the second record is the first record in the associated data queue. Additionally, after removing the second record from the associated dataset 220, the data in the second record can be added to the non-associated dataset 230.

[0051] Additionally or alternatively, in some embodiments, if it is determined that the number of data entries accessed in the second record within a recent period does not exceed a threshold, the second record is removed from the associated dataset 220. Otherwise, the second record is retained in the associated dataset 220. Additionally, if the second record is retained in the associated dataset 220, the second earliest added data record in the associated dataset 220 may be removed.

[0052] In some embodiments, frequency information 240 can be used to determine whether the number of times each piece of data in the second record has been accessed in a recent period exceeds a threshold. For example, frequency information 240 can record the number of times each piece of data has been accessed, such as written to or read from, in a recent period (e.g., one minute or other time length). In some embodiments, frequency information 240 can be as follows: Figure 5 As shown.

[0053] like Figure 5 As shown, the frequency information 240 may include a high-frequency group 510, a medium-frequency group 520, and a low-frequency group 530. The high-frequency group 510, medium-frequency group 520, and low-frequency group 530 are determined by comparing the number of times each data point has been accessed in a recent period with a first threshold number and a second threshold number, respectively. If the number of times a data point has been accessed in a recent period is higher than both the first and second threshold numbers, it belongs to the high-frequency group 510. If the number of times a data point has been accessed in a recent period is lower than both the first and second threshold numbers, it belongs to the low-frequency group 510. If the number of times a data point has been accessed in a recent period is higher than the first threshold number and lower than the second threshold number (the second threshold number is higher than the first threshold number), it belongs to the medium-frequency group 510. The threshold numbers, the first threshold number, and the second threshold number mentioned above can be any suitable number.

[0054] use Figure 5 The frequency information 240 shown can be used to determine data belonging to the low-frequency group 530 as having been accessed no more than a threshold number in a recent period. Alternatively, data belonging to both the low-frequency group 530 and the medium-frequency group 520 can also be determined as having been accessed no more than a threshold number in a recent period.

[0055] In this way, it can be ensured that data that has been frequently accessed (e.g., read) recently is not removed from the associated dataset 220. This ensures that recently frequently manipulated data can be retained in the associated dataset 220 for subsequent operations.

[0056] In some embodiments, the data in the second record can be divided into at least one data group based on the address index of each piece of data in the cache 110. The address indices of data belonging to the same data group are close to each other. The address indexes of the data in the cache 110 can be stored in the clue information 252 in the physical cache pool 250.

[0057] Figure 6 Example thread information 252 in a real cache pool 250 according to some embodiments of this disclosure is shown. Figure 6As shown, clue information 252 for the data includes dataset link 610, which maps the data to one of the associated dataset 220 and the non-associated dataset 230. Clue information 252 also includes FSID 620, which can be an identifier for the data. Clue information 252 also includes frequency information link 620, which maps the data to a group in frequency information 240, such as high-frequency group 510. Clue information 252 also includes type 640, which indicates the type of data, such as IB, BMD, or CG. Clue information 252 also includes aggregation link 650, which maps the data to aggregation hash 270. Clue information 252 also includes index 660, which records the index information of the data. Clue information 252 also includes lookup link 670, which maps the data to lookup hash 260. Clue information 252 also includes real buffer pool lock (BUF RWLOCK) 680, which is used to represent a lock on the data. Clue information 252 also includes a real cache pool pointer 690, which is used to locate the data to its physical location in the real cache pool 250.

[0058] In some embodiments, the data in the second record may be divided into at least one data group based on, for example, index 680 in the clue information 252 mapped to the data.

[0059] Figure 7 The data partitioning process for the second record is described using the aggregated hash 270. For example... Figure 7 As shown, the second record 710 to be removed includes a set of data with address indices, such as hash index values ​​of 11, 3, 4, 7, 8, 1, 80, 14, and 77. The data in the second record 710 can be stored in the actual cache pool 250 according to the hash index, as shown in 720. In some embodiments, the data in the second record 710 can be grouped by shifting the hash index three bits to the right in binary. For example, as... Figure 7 As shown, in aggregated HASH 270, data with HASH indices 1, 3, 4, 7, and 8 are aggregated into a group 730-1, data with HASH index 14 is aggregated into a group 730-2, and data with HASH indices 77 and 80 are aggregated into a group 730-3. It should be understood that... Figure 7 The groupings 730-1, 730-2, and 730-3 of the second record 710 shown are merely illustrative; other grouping methods can also be used to group the second record 710.

[0060] In some embodiments, after grouping the second record 710, data belonging to the same data group in the second record 710 can be removed from the second record 710 in the same removal operation. For example, data in group 730-1 can be removed from the second record 710 in a single removal operation. Additionally or alternatively, in some embodiments, data belonging to the same data group in the second record 710 can also be added to the non-associative dataset 230 in the same addition operation.

[0061] In this way, data groups with similar addresses in cache 110 and the actual buffer pool 250 can be processed in a single removal and / or addition operation. This reduces the number of data operations, saves resources, and improves the performance of the storage system in processing data. This operation is also more friendly to lower-level buffers.

[0062] Figure 8 An application programming interface (API) 800 for an operation buffer 110 according to some embodiments of the present disclosure is shown. For example... Figure 8 As shown, API 800 may include the UMCOpen function, which returns a handle pointer. Figure 9 An example handle 910 according to some embodiments of this disclosure is shown. For example... Figure 9 As shown, handle 910 includes open type 920 and thread pointer 930. Open type 920 may include read type ("Read") and write type ("Write"). Thread pointer 930 may point to thread information 252 in the real buffer pool 250. Figure 9 As shown, the UMCOpen function requires an input address and an open type, where the address can be a combination of FSID620, type640, and index660.

[0063] Figure 9 The document also shows the UMCWrite function, which provides API 900 to perform changes on the content and scope of the changes. Figure 9 The document also shows the UMCClose function, which is called when the user completes data changes. If the open type is "Write", the program will put the changed data into the associated dataset 220.

[0064] Figure 9 The document also shows the UMCOpenSG function, which iterates through the input address Q, opens each required buffer, and adds each handle to handlerQHead. Figure 9 The hQHead can include an index-associated data queue Ref SG, which can be updated each time new data is found. This will be combined with... Figure 10The execution process of the UMCOpenSG function is described in more detail.

[0065] Figure 9 The document also illustrates the UMCCloseSG function, which iterates through the handle queue and closes each handle. During this process, thread information 252 is created as a record. This record is added to the end of the associated dataset 220 (the associated data queue in this example). This will be combined with... Figure 11 The execution process of the UMCOpenSG function is described in more detail.

[0066] Figure 10 An example procedure 1000 for the UMCOpenSG function according to some embodiments of this disclosure is shown. For example... Figure 10 As shown, the UMCOpenSG function begins at position 1005. At position 1010, it determines whether there is a next path. For example, the initial path is... Figure 8 The pathQHead parameter in the function UMCOpenSG determines whether there is a next path. If it is determined at 1010 that there is no next path, the process ends at 1070.

[0067] If a next path is determined at step 1010, proceed to step 1015. At step 1015, check the Ref SG queue to determine if the data for that path has already been stored in the Ref SG in a previous process. If it is determined at step 1015 that the data has already been included in the Ref SG queue, proceed to step 1065. At step 1065, add the data corresponding to that path to `handerQ`. Next, return to step 1010 to find the next path.

[0068] If it is determined at point 1015 that the data is not included in the Ref SG queue, proceed to point 1020. At point 1020, search for the data in lookup hash 260. If the data is found to be included in associated dataset 220 at point 1020, proceed to points 1060 and 1065. At point 1060, determine whether the data originates from existing data in associated dataset 220.

[0069] If at point 1060 it is determined that the data originates from existing data in associated dataset 220, the data is removed from associated dataset 220, and the process proceeds to point 1055. At point 1055, the removed data is added to a new record in associated dataset 1055 to update the associated dataset. If at point 1060 it is determined that the data does not originate from associated dataset 220, the process proceeds to point 1065. At point 1065, the data corresponding to that path is added to handlerQ. Next, the process returns to point 1010 to find the next path.

[0070] If the data is not found in the associated dataset 220 at step 1015, proceed to step 1025. At step 1025, determine if the real buffer pool 250 can be retrieved. If it is determined at step 1025 that the real buffer pool 250 can be retrieved, read its content, and add it to the lookup hash 260. If it is determined at step 1025 that the real buffer pool 250 cannot be retrieved, proceed to step 1030. At step 1030, determine if the data can be retrieved from the non-associated data 230.

[0071] If it is determined at 1030 that the data can be retrieved, proceed to 1050. At 1050, the content is read. Next, the data is added to the lookup hash 260. If it is determined at 1030 that the data cannot be retrieved, proceed to 1035. At 1035, the second record at the beginning of the associated dataset 220 (e.g., an associated data queue) is flushed from the associated dataset 220. In some embodiments, the second record to be flushed from the associated dataset 220 can also be determined based on frequency information 240. At 1040, the data of the second record is collected into the aggregation hash 270, and the aggregation hash 270 groups the data of the second record by index.

[0072] At position 1045, the data content of the second record is output according to the grouping determined by aggregate hash 270. Next, the data that was flushed at position 1045 is added to the non-associative dataset 230, and the process is repeated at position 1030.

[0073] Figure 11 An example procedure 1100 of the UMCCloseSG function according to some embodiments of this disclosure is shown. The UMCCloseSG function begins at 1110. At 1120, new record data at the current handle is determined. For example, the current handle may be the handle indicated by the function's input parameter handlerQHead, or it may be the next handle determined by the previous execution. At 1130, it is determined whether there is a next handle.

[0074] If it is determined at 1130 that there is no next handle, proceed to 1170. At 1170, a new record is added to the associated dataset 220. The new record can be obtained during the execution process prior to the UMCCloseSG function. For example, the new record can be added to the end of the associated dataset 220 (e.g., an associated data queue).

[0075] If a next handle is determined at 1130, proceed to 1140. At 1140, data is added to the determined new record. For example, this data can be added to the end of a four-year record (e.g., a record queue). At 1150, the current handle is closed, and the process returns to 1130 to determine if a next handle exists.

[0076] The above combination Figure 10 and Figure 11 The process of executing example API functions for managing buffer 110 according to some embodiments of this disclosure is described. By executing these processes, a set of interrelated data can be associated together to improve the cache hit rate when performing data operations. This saves resources and improves the performance of the storage system.

[0077] Figure 12 A schematic block diagram of an example device 1200 that can be used to implement embodiments of the present disclosure is shown. For example, such as Figure 1 The storage system 100 shown can be implemented by device 1200. For example... Figure 12 As shown, device 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 1202 or loaded from storage unit 1208 into random access memory (RAM) 1203. The RAM 1203 may also store various programs and data required for the operation of device 1200. The CPU 1201, ROM 1202, and RAM 1203 are interconnected via bus 1204. An input / output (I / O) interface 1205 is also connected to bus 1204.

[0078] Multiple components in device 1200 are connected to I / O interface 1205, including: input unit 1206, such as keyboard, mouse, etc.; output unit 1207, such as various types of monitors, speakers, etc.; storage unit 1208, such as disk, optical disk, etc.; and communication unit 1209, such as network card, modem, wireless transceiver, etc. Communication unit 1209 allows device 1200 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0079] The various processes and procedures described above, such as method 300, process 1000, or process 1100, may be executed by processing unit 1201. For example, in some embodiments, method 300, process 1000, or process 1100 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program may be loaded and / or installed on device 1200 via ROM 1202 and / or communication unit 1209. When the computer program is loaded into RAM 1203 and executed by CPU 1201, one or more actions of method 300, process 1000, or process 1100 described above may be performed.

[0080] This disclosure can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0081] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0082] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0083] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0084] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0085] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0086] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0087] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0088] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for managing a storage system, comprising: In response to a first write operation on a first data block on a persistent storage device, determine whether a first set of data corresponding to the first data block is in the cache; If it is determined that the first set of data is in the cache, update the first set of data in the cache; Add the first set of data as the first record to the associated dataset in the cache; In response to insufficient cache space in the associated dataset, a second record is removed from the associated dataset, the second record being the earliest added data record in the associated dataset; Add each piece of data included in the second record to the cached non-associative dataset; Based on the address index of each piece of data in the second record in the cache, the data in the second record is divided into at least one data group, wherein the address indexes of the data belonging to the same data group are close to each other; The removal of the second record includes: removing data belonging to the same data group from the second record in the same removal operation; and Adding the data included in the second record to the non-associative dataset includes adding the data belonging to the same data group in the second record to the non-associative dataset in the same addition operation.

2. The method according to claim 1, further comprising: Determine whether the existing data in the associated dataset includes the first data in the first group of data, the existing data being updated in the cache in response to a second write operation to a second data block on the persistent storage device, the second write operation occurring before the first write operation; as well as If it is determined that the existing data includes the first data, the first data is removed from the existing data.

3. The method according to claim 1, further comprising: If it is determined that the first set of data is not included in the cache, the first set of data is added to the non-associative dataset in the cache, wherein the data in the non-associative dataset does not overlap with the data in each record of the associated dataset.

4. The method according to claim 3, further comprising: Determine whether the non-associative dataset includes the first data from the first group of data; as well as If it is determined that the first data is included in the non-associative dataset: Remove the first data from the unrelated dataset.

5. The method of claim 1, wherein removing the second record further comprises: If it is determined that the number of times each piece of data in the second record has been accessed in a recent period does not exceed the threshold, then the second record is removed.

6. The method of claim 1, wherein the associated dataset is an associated data queue, and Adding the first set of data as the first record to the associated dataset includes: Add the first record to the end of the associated data queue.

7. The method according to claim 1, wherein the first set of data includes first data of a first cache type and fourth data of a second cache type, wherein the first cache type is different from the second cache type.

8. An electronic device, comprising: At least one processor; as well as At least one memory storing computer program instructions, the at least one memory and the computer program instructions being configured, together with the at least one processor, to cause the electronic device to perform actions, the actions including: In response to a first write operation on a first data block on a persistent storage device, determine whether a first set of data corresponding to the first data block is in the cache; If it is determined that the first set of data is in the cache, update the first set of data in the cache; Add the first set of data as the first record to the associated dataset in the cache; In response to insufficient cache space in the associated dataset, a second record is removed from the associated dataset, the second record being the earliest added data record in the associated dataset; Add each piece of data included in the second record to the cached non-associative dataset; Based on the address index of each piece of data in the second record in the cache, the data in the second record is divided into at least one data group, wherein the address indexes of the data belonging to the same data group are close to each other; The removal of the second record includes: removing data belonging to the same data group from the second record in the same removal operation; and Adding the data included in the second record to the non-associative dataset includes adding the data belonging to the same data group in the second record to the non-associative dataset in the same addition operation.

9. The electronic device according to claim 8, wherein the action further comprises: Determine whether the existing data in the associated dataset includes the first data in the first group of data, the existing data being updated in the cache in response to a second write operation to a second data block on the persistent storage device, the second write operation occurring before the first write operation; as well as If it is determined that the existing data includes the first data, the first data is removed from the existing data.

10. The electronic device according to claim 8, wherein the action further includes: If it is determined that the first set of data is not included in the cache, the first set of data is added to the non-associative dataset in the cache, wherein the data in the non-associative dataset does not overlap with the data in each data record of the associated dataset.

11. The electronic device of claim 10, further comprising: Determine whether the non-associative dataset includes the first data from the first group of data; as well as If it is determined that the first data is included in the non-associative dataset: Remove the first data from the unrelated dataset.

12. The electronic device of claim 8, wherein removing the second record further comprises: If it is determined that the number of times each piece of data in the second record has been accessed in a recent period does not exceed the threshold, then the second record is removed.

13. The electronic device of claim 8, wherein the associated dataset is an associated data queue, and Adding the first set of data as the first record to the associated dataset includes: Add the first record to the end of the associated data queue.

14. The electronic device of claim 8, wherein the first set of data includes first data of a first cache type and fourth data of a second cache type, the first cache type being different from the second cache type.

15. A computer program product tangibly stored on a non-volatile computer-readable medium and comprising machine-executable instructions that, when executed, cause a device to perform the method according to any one of claims 1-7.

Images