Cache data processing method and device, equipment and storage medium

Abstract

The invention provides a cache data processing method and device, equipment and a storage medium, relates to the field of distributed storage systems, and aims to solve the problem of how to ensure that cache data is partially available after a fault is recovered so as to realize read performance optimization after the fault is recovered. The method comprises the steps that in response to a read-write request for a first data storage device after failure recovery, whether a first modification range exists in the first data storage device after failure recovery or not is inquired, and the first modification range is used for representing the modification range of metadata corresponding to cache data during the failure period of the first data storage device; under the condition that the first modification range is queried, obtaining a storage distribution range of the cache data in the first data storage equipment; and under the condition that an overlapping range exists between the first modification range and the storage distribution range, performing failure operation on at least one part of cache data in the first data storage equipment after failure recovery.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of distributed storage systems, and specifically, to a cache data processing method, apparatus, device and storage medium. Background Art

[0002] With the rapid development of information technology, the importance of data has become increasingly prominent, which has promoted the progress of big data and high-performance computing. Distributed storage can meet the growing demand for data volume with its scalable system structure. At the same time, the reliability of stored data and the performance of storage systems have also received widespread attention.

[0003] In distributed storage, data reliability is achieved through replica mode or erasure mode. Under the premise of the same amount of data, the replica mode has a fast data read speed but consumes more storage space, while the erasure mode has a slow data read and write speed but consumes less storage space. In order to combine the advantages of the two modes and consider hardware costs and storage capacity, a replica pool is usually created on the SSD (Solid State Drive) as a cache pool, and an erasure pool is created on the HDD (Hard Disk Drive) as a data storage pool. The cache pool stores data information written to the storage system and aggregates the data and sends it to the data storage pool.

[0004] Since the cache pool only stores object attribute information but not specific data, in order to improve the performance of the storage system, a cache object is created in the cache pool OSD (Object Storage Device) to write the head object and store the object data in a single copy. However, since the cache object is a single copy, after the OSD fails and recovers, it is impossible to confirm whether the cache object has been modified, resulting in the cache object being unreliable, which affects the read performance in the failure recovery scenario. Summary of the invention

[0005] The embodiments of the present application provide a cache data processing method, apparatus, device and storage medium, aiming to solve the problem of how to ensure that part of the cache data is available after fault recovery to achieve read performance optimization after fault recovery.

[0006] A first aspect of an embodiment of the present application provides a cache data processing method, which is applied to a distributed storage system, wherein the distributed storage system includes at least a first data storage device, and the method includes: In response to a read / write request to the first data storage device after failure recovery, query whether a first modification range exists in the first data storage device after failure recovery, wherein the first modification range is used to represent a modification range of metadata corresponding to cached data during the period when the first data storage device fails; In the case where the first modification range is found, obtaining a storage distribution range of the cache data in the first data storage device before the first data storage device fails; In the case where there is an overlapping range between the first modification range and the storage distribution range, performing an invalidation operation on at least a portion of cached data in the first data storage device after failure recovery; Based on the first data storage device after the invalidation operation, the read and write operations corresponding to the read and write requests are performed.

[0007] In an optional implementation, when there is an overlapping range between the first modification range and the storage distribution range, performing an invalidation operation on at least a portion of cached data in the first data storage device after failure recovery includes: In the case where there is an overlapping range between the first modification range and the storage distribution range, determining that during the period when the first data storage device fails, cache data corresponding to the overlapping range in the first data storage device is modified; An invalidation operation is performed on cache data corresponding to the overlapping range in the first data storage device after failure recovery.

[0008] In an optional implementation, when there is an overlapping range between the first modification range and the storage distribution range, performing an invalidation operation on at least a portion of cached data in the first data storage device after failure recovery includes: In the case where there is an overlapping range between the first modification range and the storage distribution range, and the first modification range covers the storage distribution range, determining that during the period when the first data storage device fails, all cached data in the first data storage device are modified; The cached data in the first data storage device after the failure is recovered is deleted to implement an invalidation operation on the cached data in the first data storage device after the failure is recovered.

[0009] In an optional implementation, the distributed storage system further includes a second data storage device, and the method further includes: When the first data storage device that has failed is recovering from the failure, querying whether there is a second modification range in the second data storage device, the second modification range being a modification range of metadata in the second data storage device that has not failed during the period when the first data storage device fails; When the second modification range is found, the first modification range on the first data storage device is updated according to the second modification range, and the second modification range is cleared.

[0010] In an optional embodiment, the method further includes: During a period when the first data storage device fails, in response to a modification operation on metadata in the second data storage device, reading the second modification range from a database corresponding to the second data storage device; When the second modification range is empty, assigning the current modification range of the metadata in the second data storage device to the second modification range; When the second modification range is not empty, update the second modification range according to the following steps: When there is an overlapping range between the current modification range and the second modification range read from the database corresponding to the second data storage device, updating the second modification range to the current modification range; When the current modification range is not recorded in the second modification range read from the database corresponding to the second data storage device, the current modification range is added to the second modification range.

[0011] In an optional implementation, before querying whether the first modification range exists in the first data storage device after failure recovery, the method further includes: querying whether cached data in the first data storage device after failure recovery carries an invalidation mark, wherein the invalidation mark is added after an invalidation operation is performed during the first response to a read or write request of the first data storage device after failure recovery; In a case where the cache data in the first data storage device after failure recovery carries an invalidation mark, executing the read / write operation corresponding to the read / write request; Querying whether a first modification range exists in a first data storage device after failure recovery includes: In the case that the cache data in the first data storage device after the failure recovery does not carry an invalidation mark, it is queried whether the first modification range exists in the first data storage device after the failure recovery.

[0012] In an optional implementation, after performing a failure operation on the first data storage device after failure recovery in response to a read or write request to the first data storage device after failure recovery for the first time, the method further includes: adding an invalidation mark to the cache data that has been invalidated in the first data storage device after failure recovery, and deleting the first modification range; In response to a read / write request for the first data storage device after failure recovery, query whether there is a failure mark in the first data storage device after failure recovery; In the case where a failure mark exists in the first data storage device after failure recovery, the read / write operation corresponding to the read / write request is performed based on the first data storage device after failure recovery.

[0013] A second aspect of an embodiment of the present application provides a cache data processing apparatus, which is applied to a distributed storage system, wherein the distributed storage system includes at least a first data storage device, and the apparatus includes: A query module, configured to query, in response to a read / write request to the first data storage device after failure recovery, whether a first modification range exists in the first data storage device after failure recovery, wherein the first modification range is used to characterize a modification range of metadata corresponding to cached data during a period when the first data storage device fails; an acquisition module, configured to acquire, when the first modification range is found, a storage distribution range of the cached data in the first data storage device before the first data storage device fails; A processing module, configured to, when there is an overlapping range between the first modification range and the storage distribution range, perform an invalidation operation on at least a portion of cached data in the first data storage device after failure recovery; An execution module is used to execute the read / write operation corresponding to the read / write request based on the first data storage device after the invalidation operation.

[0014] A third aspect of an embodiment of the present application provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the cache data processing method of the first aspect of the embodiment of the present application when executing the computer program.

[0015] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the cache data processing method of the first aspect of the embodiment of the present application is implemented.

[0016] A fifth aspect of an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the cache data processing method of the first aspect of an embodiment of the present application.

[0017] In this embodiment, by querying whether there is a first modification range in the first data storage device after the failure recovery, the first modification range is used to characterize the modification range of the metadata corresponding to the cached data during the failure of the first data storage device; in the case of querying the first modification range, the storage distribution range of the cached data in the first data storage device before the failure of the first data storage device is obtained; in the case of overlapping ranges between the first modification range and the storage distribution range, at least a part of the cached data in the first data storage device after the failure recovery is invalidated, so as to ensure that the cached data is partially available after the failure recovery, realize the optimization of the read performance after the failure recovery, and improve the reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 is a flowchart of the steps of a cache data processing method proposed in an embodiment of the present application; Figure 2 is a flowchart of the processing steps of a cache data processing method proposed in an embodiment of the present application; Figure 3 is a schematic diagram of the structure of a cache data processing device proposed in an embodiment of the present application; Figure 4 It is a schematic diagram of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] With the rapid development of information technology, data, as an important part of information technology, has attracted more and more attention. By collecting data generated by people's various activities and analyzing and processing these data, very important information can be obtained, thereby creating huge economic value. This has also promoted the rapid development of big data and high-performance computing. How to make storage devices adapt to the rapid growth of data volume has become one of the key issues. Distributed storage implemented through software definition meets the requirements of growing data volume with its scalable system structure and is gradually recognized by more and more companies. At the same time, the reliability of stored data and the performance of storage systems are also receiving more and more attention.

[0022] In terms of reliability, distributed storage systems generally use replica mode or erasure mode to implement data redundancy mechanisms. In replica mode, each replica stores the same data. Read operations can read any replica, and write operations need to write data to each replica. In erasure mode, when data is written, the storage system divides the data written by the client into K parts according to the length of the erasure unit, and then uses the erasure algorithm to calculate M parts of verification data from these K parts of original data. Finally, these K+M parts of data are stored in different devices or nodes. When reading data, read any K parts of the K+M parts, and then restore the complete written data through the erasure algorithm.

[0023] PG (Placement Group) is a logical grouping mechanism in a distributed storage system, used to manage the storage location of objects. OSD (Object Storage Device) is the core component responsible for storing and managing data in a distributed storage system cluster. Each OSD is responsible for managing a disk or a storage device, and they work together to form a highly reliable data storage system. In a distributed storage system, PG is used to manage objects, and PG is mapped to OSD for storage through algorithm calculation. A PG usually selects a master OSD to interact with the client for IO. Within the distributed system, the master OSD writes or retrieves the data written or read by the client within the cluster.

[0024] Under the premise of the same data volume, the data read speed of the replica mode is fast but consumes more storage space, while the data read and write speed of the erasure mode is slow but consumes less storage space. Under the premise of ensuring the performance of the storage system and reducing the waste of space, in order to give full play to the advantages of the two storage pools, and taking into account factors such as the cost and storage capacity of different hardware, a replica pool is created on the SSD as the cache pool of the storage system, and an erasure pool is created on the HDD as the data storage pool of the storage system. The cache pool stores the data information written to the storage system, and aggregates the data written to the storage system and sends it to the data pool. The storage pool saves all the data.

[0025] Because the cache pool only stores object attribute information and does not store specific data, in order to improve the performance of the storage system, a cache object (cache data itself) is created in the cache pool OSD to write the head object (metadata), and the object data is stored in the cache pool OSD in a single copy to improve the read performance. After the head object is written, the cache control thread is notified to complete the writing of the cache object. Since the cache object is a single copy, after the OSD fails and recovers, it is impossible to confirm whether the cache object has been modified. As a result, the cache object is unreliable and the read performance in the failure recovery scenario is reduced.

[0026] The present application proposes a cache data processing method, which records the object modification range in the head object, and restores the object modification range attribute to a new attribute of the head object during fault recovery. When the cache object is read or written for the first time, the head object attribute and the cache data distribution range are obtained, the overlapping part is calculated, and whether the cache object is valid is determined based on the overlapping range, and invalid data is cleared, thereby improving the read performance of the storage system.

[0027] Reference Figure 1 , Figure 1 1 is a flowchart of a cache data processing method proposed in an embodiment of the present application. The method is applied to a distributed storage system, and the distributed storage system includes at least a first data storage device. Specifically, the method includes the following steps S11 to S14: Step S11: In response to a read or write request to the first data storage device after failure recovery, query whether there is a first modification range in the first data storage device after failure recovery, wherein the first modification range is used to characterize the modification range of the metadata corresponding to the cached data during the failure of the first data storage device.

[0028] OSD (Object Storage Device) is an object-based storage architecture. In OSD, data is stored in the form of objects, including cache data itself (cache object), metadata (head object, such as data identification, type, location, size, etc.), and may also include other information for data protection or management.

[0029] In response to a read or write request to the first data storage device (OSD1) after failure recovery, query whether there is a first modification range in the first data storage device by accessing the OSD metadata area or using a specific management command. This usually involves querying the metadata of the first data storage device to determine whether there is a dirty_range_recover attribute (first modification range), which can be used as the modification range of the metadata corresponding to the cached data during the failure.

[0030] Furthermore, the distributed storage system further includes a second data storage device, and the method further includes: When the first data storage device that has failed is recovering from the failure, querying whether there is a second modification range in the second data storage device, the second modification range being a modification range of metadata in the second data storage device that has not failed during the period when the first data storage device fails; When the second modification range is found, the first modification range on the first data storage device is updated according to the second modification range, and the second modification range is cleared.

[0031] In a distributed storage system, a replica pool is created on the SSD and used as a cache pool for the storage system. Each SSD can correspond to a cache pool OSD. When the cache pool OSD creates a cache object that writes the head object, the cache object stores the object data in a single copy. For example, the cache object is only stored on OSD1, and the head object is written on both OSD1 and OSD2.

[0032] The distributed storage system also includes a second data storage device (OSD2). When the first data storage device fails and the head object is restored, it queries whether there is a second modification range in the second data storage device. The second modification range is the modification range of the head object that is recorded and updated by the second data storage device that has not failed during the period when the first data storage device fails. Specifically, in the definition of the head object, a dirty_range attribute (second modification range) is added. This attribute is used to record the modification range of the object, that is, when the head object performs modification operations such as write, delete, zero, truncate, etc., the data range involved in these modification operations will be recorded. The write operation is the process of writing new data or updated data to the storage medium. The delete operation is the process of removing data from the storage medium. The zero operation is the process of setting the contents of a storage area to a zero value. The truncate operation is the process of shortening the contents of a file or data to zero length or a specified length.

[0033] When it is found that there is a second modification range in the second data storage device, that is, the head object in the second data storage device has a dirty_range attribute, on the first data storage device after the failure recovery, the head object adds the attribute dirty_range_recover, updates the dirty_range recovery to dirty_range_recover, and clears the dirty_range. By recording the modification range of the object in the object attribute, when the OSD of the storage system fails and data is recovered, the modification range attribute is restored, providing a data basis for the subsequent analysis of whether some cached data is available.

[0034] Furthermore, the method further comprises: During a period when the first data storage device fails, in response to a modification operation on metadata in the second data storage device, reading the second modification range from a database corresponding to the second data storage device; When the second modification range is empty, assigning the current modification range of the metadata in the second data storage device to the second modification range; When the second modification range is not empty, update the second modification range according to the following steps: When there is an overlapping range between the current modification range and the second modification range read from the database corresponding to the second data storage device, updating the second modification range to the current modification range; When the current modification range is not recorded in the second modification range read from the database corresponding to the second data storage device, the current modification range is added to the second modification range.

[0035] During the period when the first data storage device fails, in response to the modification operation of the metadata in the second data storage device, such as the modification operation of write, delete, zero, truncate, etc. on the head object in the second data storage device, the dirty_range attribute is read from the database corresponding to the second data storage device. If dirty_range is empty, the current modification range of the head object is assigned to dirty_range; if dirty_range is not empty, the current modification range is updated to dirty_range. The update strategy is to take the current modification as the basis, and when the current modification range has an overlapping part with the read dirty_range (there is an overlapping range), the dirty_range is updated to the current modification range; when the current modification range is not recorded in dirty_range, the current modification range is increased to dirty_range.

[0036] Step S12: when the first modification range is found, obtaining a storage distribution range of the cache data in the first data storage device before the first data storage device fails.

[0037] If the first modification range is found, that is, the dirty_range_recover attribute of the head object is obtained, it means that the cache object has undergone data modification operations during the failure of the first data storage device. Then, the storage distribution range of the cache object in the first data storage device is obtained through the fiemap interface of the storage system. The fiemap interface is a functional interface provided by the storage system, which can return the data distribution range information of the cache object on the storage device, including the location and size of the data block. If the first modification range is not found, it means that the cache object has not undergone data modification operations during the failure of the first data storage device. Continue the cache object read and write operations. Step S13: in the case where there is an overlapping range between the first modification range and the storage distribution range, invalidating at least a portion of cached data in the first data storage device after failure recovery.

[0038] The first modification range and the storage distribution range are compared. When there is an overlapping range between the first modification range and the storage distribution range, at least a portion of the cached data in the first data storage device after fault recovery is invalidated, thereby ensuring that the cached data portion after fault recovery is available to achieve post-fault recovery read performance optimization. When there is no overlapping range between the first modification range and the storage distribution range, it can be determined that during the period when the first data storage device fails, the cached data corresponding to the overlapping range in the first data storage device has not been modified, and the cache object does not need to be invalidated. Based on the first data storage device after fault recovery, the read and write operations corresponding to the read and write requests are executed.

[0039] Further, in the case where there is an overlapping range between the first modification range and the storage distribution range, performing an invalidation operation on at least a portion of cached data in the first data storage device after failure recovery includes: In the case where there is an overlapping range between the first modification range and the storage distribution range, determining that during the period when the first data storage device fails, cache data corresponding to the overlapping range in the first data storage device is modified; An invalidation operation is performed on cache data corresponding to the overlapping range in the first data storage device after failure recovery.

[0040] When there is an overlapping range between the first modification range and the storage distribution range, it can be determined that during the failure of the first data storage device, the cache data corresponding to the overlapping range in the first data storage device is modified. The cache data corresponding to the overlapping range in the first data storage device is invalidated, and this part of the data is invalidated by performing a zero operation, that is, all the data in the cache objects of the overlapping part is set to zero. The cache data corresponding to the determined overlapping range is modified, and the invalidation operation is performed on this part of the data, thereby ensuring that the cache data is partially available after the failure recovery, so as to achieve read performance optimization after the failure recovery.

[0041] Further, in the case where there is an overlapping range between the first modification range and the storage distribution range, performing an invalidation operation on at least a portion of cached data in the first data storage device after failure recovery includes: In the case where there is an overlapping range between the first modification range and the storage distribution range, and the first modification range covers the storage distribution range, determining that during the period when the first data storage device fails, all cached data in the first data storage device are modified; The cached data in the first data storage device after the failure is recovered is deleted to implement an invalidation operation on the cached data in the first data storage device after the failure is recovered.

[0042] When there is an overlapping range between the first modification range and the storage distribution range, and the first modification range covers the storage distribution range, that is, when the first modification range includes the entire storage distribution range, it can be determined that during the period when the first data storage device fails, the cached data in the first data storage device are all modified, and the cache object no longer has valid data. In this case, the cache object is deleted from the first data storage device to avoid using untrusted data, while freeing up storage space, avoiding unnecessary waste of storage resources, and improving the performance and efficiency of the storage system.

[0043] Step S14: executing the read / write operation corresponding to the read / write request based on the first data storage device after the invalidation operation.

[0044] After the invalidation operation is completed, the read and write operations corresponding to the read and write requests are performed based on the first data storage device after the invalidation operation. At this time, the read and write operations will be performed based on the updated cache state and stored data to ensure the accuracy of the data.

[0045] Furthermore, before querying whether the first modification range exists in the first data storage device after failure recovery, the method further includes: querying whether cached data in the first data storage device after failure recovery carries an invalidation mark, wherein the invalidation mark is added after an invalidation operation is performed during the first response to a read or write request of the first data storage device after failure recovery; In a case where the cache data in the first data storage device after failure recovery carries an invalidation mark, executing the read / write operation corresponding to the read / write request; Querying whether a first modification range exists in a first data storage device after failure recovery includes: In the case that the cache data in the first data storage device after the failure recovery does not carry an invalidation mark, it is queried whether the first modification range exists in the first data storage device after the failure recovery.

[0046] In order to improve the read performance, the cache object index information is fully cached in the memory, and an invalidation mark indicating whether the cache data invalidation operation has been performed is added to the cache object index information. The invalidation mark is added after the invalidation operation is performed in the process of responding to the read and write request of the first data storage device after the fault recovery for the first time. Before responding to the read and write request of the first data storage device after the fault recovery and querying whether there is a first modification range in the first data storage device after the fault recovery, first query whether the cache data in the first data storage device carries the invalidation mark to determine whether the cache data invalidation operation has been performed. If the cache data carries the invalidation mark, it means that the invalidation operation has been performed, and the read and write operation corresponding to the read and write request continues to be performed; if the cache data does not carry the invalidation mark, it means that the invalidation operation has not been performed, and query whether there is a first modification range in the first data storage device after the fault recovery. By querying whether the cache data carries the invalidation mark to determine whether the invalidation operation has been performed, it can be ensured that only one cache data invalidation operation is performed when responding to the read and write request, avoiding repeated invalidation operations to shorten the response time of the read and write request and improve the efficiency of the system.

[0047] Further, after performing a failure operation on the first data storage device after failure recovery in response to a read or write request to the first data storage device after failure recovery for the first time, the method further includes: adding an invalidation mark to the cache data that has been invalidated in the first data storage device after failure recovery, and deleting the first modification range; In response to a read / write request for the first data storage device after failure recovery, query whether there is a failure mark in the first data storage device after failure recovery; In the case where a failure mark exists in the first data storage device after failure recovery, the read / write operation corresponding to the read / write request is performed based on the first data storage device after failure recovery.

[0048] In response to the first read / write request to the first data storage device after the fault recovery, after the invalidation operation is performed on the first data storage device after the fault recovery, an invalidation mark is added to the cached data in the first data storage device after the fault recovery that has been invalidated, and the first modification range is deleted, that is, the cache object is marked as having been invalidated, and the dirty_range_recover attribute of the head object is deleted. In response to the second read / write request to the first data storage device after the fault recovery, it is queried whether there is an invalidation mark in the first data storage device after the fault recovery; if there is an invalidation mark in the first data storage device after the fault recovery, the read / write operation corresponding to the second read / write request is performed based on the first data storage device after the fault recovery. By adding an invalidation mark to the cached data that has been invalidated and deleting the first modification range, it is helpful to perform only one cached data invalidation operation in response to the read / write request, avoid repeated invalidation operations, and avoid mistakenly invalidating valid data in subsequent read / write operations, thereby improving the efficiency and performance of the system.

[0049] In this embodiment, by querying whether there is a first modification range in the first data storage device after the failure recovery, the first modification range is used to characterize the modification range of the metadata corresponding to the cached data during the failure of the first data storage device; in the case of querying the first modification range, the storage distribution range of the cached data in the first data storage device before the failure of the first data storage device is obtained; in the case of overlapping ranges between the first modification range and the storage distribution range, at least a part of the cached data in the first data storage device after the failure recovery is invalidated, so as to ensure that the cached data is partially available after the failure recovery, realize the optimization of the read performance after the failure recovery, and improve the reliability of the system.

[0050] Reference Figure 2 , Figure 2 1 is a flowchart of the processing steps of the cache data processing method proposed in one embodiment of the present application. Figure 2 As shown, the specific processing process is: Before responding to cache data (cache object) reading and writing, first determine whether the cache object has undergone a cache invalidation operation; when the cache object has undergone a cache invalidation operation, perform cache object reading and writing operations; when the cache object has not undergone a cache invalidation operation, determine whether the data modification range (first modification range) of the head object corresponding to the cache object has a value; if the data modification range has no value, perform cache object reading and writing operations; if the data modification range has a value, obtain the storage distribution range of the cache object, and calculate the overlapping range of the data modification range and the storage distribution range; if the data modification range does not overlap with the storage distribution range, do not perform an invalidation operation; if the data modification range partially overlaps with the storage distribution range, clear the overlapping part to 0, complete the cache object invalidation operation, and mark the cache object as having undergone a cache invalidation operation; if the data modification range includes the entire storage distribution range, clear the overlapping part to 0, delete the cache object, complete the cache object invalidation operation, and mark the cache object as having undergone a cache invalidation operation; perform cache object reading and writing operations.

[0051] By first determining whether the cache object has undergone a cache invalidation operation, and performing an invalidation operation based on the overlap between the cache object storage distribution range and the data modification range, when the cache object is read or written, only one cache data invalidation operation is performed, thereby ensuring that part of the cache data is available after fault recovery to achieve the purpose of optimizing read performance after fault recovery.

[0052] Reference Figure 3 , Figure 3 1 is a schematic diagram of the structure of a cache data processing device proposed in an embodiment of the present application. The device is applied to a distributed storage system. The distributed storage system includes at least a first data storage device. The device includes: A query module, configured to query, in response to a read / write request to the first data storage device after failure recovery, whether a first modification range exists in the first data storage device after failure recovery, wherein the first modification range is used to characterize a modification range of metadata corresponding to cached data during a period when the first data storage device fails; an acquisition module, configured to acquire, when the first modification range is found, a storage distribution range of the cached data in the first data storage device before the first data storage device fails; A processing module, configured to, when there is an overlapping range between the first modification range and the storage distribution range, perform an invalidation operation on at least a portion of cached data in the first data storage device after failure recovery; An execution module is used to execute the read / write operation corresponding to the read / write request based on the first data storage device after the invalidation operation.

[0053] In an optional implementation, the processing module is specifically used for: In the case where there is an overlapping range between the first modification range and the storage distribution range, determining that during the period when the first data storage device fails, cache data corresponding to the overlapping range in the first data storage device is modified; An invalidation operation is performed on cache data corresponding to the overlapping range in the first data storage device after failure recovery.

[0054] In an optional implementation, the processing module is specifically used for: In the case where there is an overlapping range between the first modification range and the storage distribution range, and the first modification range covers the storage distribution range, determining that during the period when the first data storage device fails, all cached data in the first data storage device are modified; The cached data in the first data storage device after the failure is recovered is deleted to implement an invalidation operation on the cached data in the first data storage device after the failure is recovered.

[0055] In an optional embodiment, the device further comprises: A recovery module, used for querying whether there is a second modification range in the second data storage device when the first data storage device that has failed is recovering from the failure, wherein the second modification range is a modification range of metadata in the second data storage device that has not failed during the period when the first data storage device fails; When the second modification range is found, the first modification range on the first data storage device is updated according to the second modification range, and the second modification range is cleared.

[0056] In an optional embodiment, the device further comprises: a record update module, configured to read the second modification range from a database corresponding to the second data storage device in response to a modification operation on the metadata in the second data storage device during a failure of the first data storage device; When the second modification range is empty, assigning the current modification range of the metadata in the second data storage device to the second modification range; When the second modification range is not empty, update the second modification range according to the following steps: When there is an overlapping range between the current modification range and the second modification range read from the database corresponding to the second data storage device, updating the second modification range to the current modification range; When the current modification range is not recorded in the second modification range read from the database corresponding to the second data storage device, the current modification range is added to the second modification range.

[0057] In an optional embodiment, the device further comprises: a judgment module, used for querying whether the cached data in the first data storage device after the failure recovery carries an invalidation mark, wherein the invalidation mark is added after the invalidation operation is performed in the process of responding to the read and write request of the first data storage device after the failure recovery for the first time; In a case where the cache data in the first data storage device after failure recovery carries an invalidation mark, executing the read / write operation corresponding to the read / write request; Querying whether a first modification range exists in a first data storage device after failure recovery includes: In the case that the cache data in the first data storage device after the failure recovery does not carry an invalidation mark, it is queried whether the first modification range exists in the first data storage device after the failure recovery.

[0058] In an optional implementation, after performing a failure operation on the first data storage device after failure recovery in response to a read or write request to the first data storage device after failure recovery for the first time, the apparatus further includes: An adding module, used for adding an invalidation mark to the cache data that has been invalidated in the first data storage device after failure recovery, and deleting the first modification range; In response to a read / write request for the first data storage device after failure recovery, query whether there is a failure mark in the first data storage device after failure recovery; In the case where a failure mark exists in the first data storage device after failure recovery, the read / write operation corresponding to the read / write request is performed based on the first data storage device after failure recovery.

[0059] Based on the same inventive concept, another embodiment of the present application provides an electronic device, referring to Figure 4 , Figure 4 FIG. 1 is a schematic diagram of an electronic device shown in an embodiment of the present application. Figure 4 As shown, the electronic device 100 includes: a memory 110 and a processor 120. The memory 110 and the processor 120 are connected via a bus communication. A computer program is stored in the memory 110. The computer program can be run on the processor 120 to implement the steps in the cache data processing method disclosed in the embodiment of the present disclosure.

[0060] Based on the same inventive concept, an embodiment of the present disclosure also provides a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor of a computer device, the computer device can execute the steps in the cache data processing method described in any of the above embodiments of the present application.

[0061] Based on the same inventive concept, an embodiment of the present disclosure further provides a computer program product, including a computer program. When the computer program is executed by a processor of a computer device, the computer program can execute the steps in the cache data processing method disclosed in the embodiment of the present disclosure.

[0062] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0063] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application may adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0064] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0065] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0066] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0067] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0068] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0069] The above is a detailed introduction to a cache data processing method, device, equipment and storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A cache data processing method, characterized in that: Applied to a distributed storage system, the distributed storage system includes at least a first data storage device, and the method includes: In response to a read / write request to the first data storage device after failure recovery, query whether a first modification range exists in the first data storage device after failure recovery, wherein the first modification range is used to represent a modification range of metadata corresponding to cached data during the period when the first data storage device fails; In the case where the first modification range is found, obtaining a storage distribution range of the cache data in the first data storage device before the first data storage device fails; In the case where there is an overlapping range between the first modification range and the storage distribution range, performing an invalidation operation on at least a portion of cached data in the first data storage device after failure recovery; Based on the first data storage device after the invalidation operation, the read and write operations corresponding to the read and write requests are performed.

2. The cache data processing method according to claim 1, characterized in that: In the case where there is an overlapping range between the first modification range and the storage distribution range, performing an invalidation operation on at least a portion of cached data in the first data storage device after failure recovery includes: In the case where there is an overlapping range between the first modification range and the storage distribution range, determining that during the period when the first data storage device fails, cache data corresponding to the overlapping range in the first data storage device is modified; An invalidation operation is performed on cache data corresponding to the overlapping range in the first data storage device after failure recovery.

3. The cache data processing method according to claim 1, characterized in that: In the case where there is an overlapping range between the first modification range and the storage distribution range, performing an invalidation operation on at least a portion of cached data in the first data storage device after failure recovery includes: In the case where there is an overlapping range between the first modification range and the storage distribution range, and the first modification range covers the storage distribution range, determining that during the period when the first data storage device fails, all cached data in the first data storage device are modified; The cached data in the first data storage device after the failure is recovered is deleted to implement an invalidation operation on the cached data in the first data storage device after the failure is recovered.

4. The cache data processing method according to claim 1, characterized in that: The distributed storage system further includes a second data storage device, and the method further includes: When the first data storage device that has failed is recovering from the failure, querying whether there is a second modification range in the second data storage device, the second modification range being a modification range of metadata in the second data storage device that has not failed during the period when the first data storage device fails; When the second modification range is found, the first modification range on the first data storage device is updated according to the second modification range, and the second modification range is cleared.

5. The cache data processing method according to claim 4, characterized in that: The method further comprises: During a period when the first data storage device fails, in response to a modification operation on metadata in the second data storage device, reading the second modification range from a database corresponding to the second data storage device; When the second modification range is empty, assigning the current modification range of the metadata in the second data storage device to the second modification range; When the second modification range is not empty, update the second modification range according to the following steps: When there is an overlapping range between the current modification range and the second modification range read from the database corresponding to the second data storage device, updating the second modification range to the current modification range; When the current modification range is not recorded in the second modification range read from the database corresponding to the second data storage device, the current modification range is added to the second modification range.

6. The cache data processing method according to claim 1, characterized in that: Before querying whether the first modification range exists in the first data storage device after failure recovery, the method further includes: querying whether cached data in the first data storage device after failure recovery carries an invalidation mark, wherein the invalidation mark is added after an invalidation operation is performed during the first response to a read or write request of the first data storage device after failure recovery; In a case where the cache data in the first data storage device after failure recovery carries an invalidation mark, executing the read / write operation corresponding to the read / write request; Querying whether a first modification range exists in a first data storage device after failure recovery includes: In the case that the cache data in the first data storage device after the failure recovery does not carry an invalidation mark, it is queried whether the first modification range exists in the first data storage device after the failure recovery.

7. The cache data processing method according to claim 6, characterized in that: After performing a failure operation on the first data storage device after failure recovery in response to a read / write request to the first data storage device after failure recovery for the first time, the method further includes: adding an invalidation mark to the cache data that has been invalidated in the first data storage device after failure recovery, and deleting the first modification range; In response to a read / write request for the first data storage device after failure recovery, query whether there is a failure mark in the first data storage device after failure recovery; In the case where a failure mark exists in the first data storage device after failure recovery, the read / write operation corresponding to the read / write request is performed based on the first data storage device after failure recovery.

8. A cache data processing device, characterized in that: Applied to a distributed storage system, the distributed storage system includes at least a first data storage device, and the apparatus includes: A query module, configured to query, in response to a read / write request to the first data storage device after failure recovery, whether a first modification range exists in the first data storage device after failure recovery, wherein the first modification range is used to characterize a modification range of metadata corresponding to cached data during a period when the first data storage device fails; an acquisition module, configured to acquire, when the first modification range is found, a storage distribution range of the cached data in the first data storage device before the first data storage device fails; A processing module, configured to, when there is an overlapping range between the first modification range and the storage distribution range, perform an invalidation operation on at least a portion of cached data in the first data storage device after failure recovery; An execution module is used to execute the read / write operation corresponding to the read / write request based on the first data storage device after the invalidation operation.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the cache data processing method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the cache data processing method according to any one of claims 1 to 7 is implemented.

Images