A log space recovery optimization method for cache

Abstract

The present invention discloses a log space recovery optimization method for cache, comprising the following steps: S1: writing log 1 into the head of the log space; S2: recovering the log space. The present invention formats the logs that block the release of the log space into new logs after aggregation, rewrites the log space head, and releases the blocked tail space without flushing. The present invention can significantly reduce the data flushing to the back-end disk when the business volume is large and the log space is almost exhausted, thereby improving the performance of the log system in this scenario. At the same time, the present invention does not require additional log space, and reuses the previous log process, which can improve the utilization rate of the log space to a certain extent. In addition, aggregating multiple old logs belonging to the same file into a new log write can save log space consumption to a certain extent.

Description

Technical Field

[0001] The present invention relates to the technical field of computer storage, and in particular to a log space recovery optimization method applied to cache. Background Art

[0002] Cache: In order to speed up read and write requests in storage systems, faster-response media is usually used as cache. Write requests are first written to the cache media, and then written to the slower backend media at the right time. If a read request can hit the cache, the data is directly retrieved from the cache and returned; otherwise, it is read from the backend. Common open source cache products in block storage include flashcache, bcache, etc.

[0003] Log: In order to speed up read and write requests in the storage system, or if the backend HDD disk is not very sensitive to random writes, log disks are usually configured to speed up or improve random write performance. When the system crashes and restarts due to an error, the log system ensures that the data on the backend disk can be restored to the state before the crash, ensuring data integrity.

[0004] The write request in the storage system will be formatted into a log (including all the request information, the write request can be recovered from the log, and if it is a log in the database, it will contain redo and undo information) and written to the log space first. After success, it will return success to the client and start writing to the backend disk. After the backend disk is successfully written, the log space occupied by the previous log can be released. The log space is usually independent of the backend hard disk. It is a circular log space composed of a continuous disk area. New logs are always written from the head and old logs are released from the tail, so that the log is always appended forward. The cache and log can also be used in combination, using memory as cache and faster media as logs. Write requests are written to the log, and data is directly stored in the memory. If the read hit is directly returned from the memory, the reading will be faster. At the same time, converting random writes into sequential writes and writing them to the log can improve the random write performance of the system.

[0005] Under the existing technology, when the storage system using cache logs is under high pressure and the logs are not flushed in time, the log space may be used up, and the newly generated logs cannot be written without log space, thus blocking the front-end request. The usual practice is to enable forced flushing when a certain water level is reached, and write the log data that blocks the log space to the back-end disk. If the back-end disk is busy at this time, frequent flushing will increase the system busyness, resulting in lower system performance and even blocking the front-end request. Summary of the invention

[0006] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0007] In view of the above problems existing in the prior art, the present invention is proposed.

[0008] Therefore, the purpose of the present invention is to provide a method for optimizing the recovery of log space applied to cache, in order to solve the problem that "frequent flushing will aggravate the system busyness, resulting in lower system performance and even blocking foreground requests".

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0010] A method for optimizing the recovery of log space applied to a cache includes the following steps:

[0011] S1: write log 1 to the head of the log space;

[0012] S2: Reclaim the log space.

[0013] As a preferred solution of the log space recovery optimization method applied to cache described in the present invention, in step S1, the system receives a write request from the client, and after formatting the request into log 1, log 1 is written into the log space header.

[0014] As a preferred solution of the log space recovery optimization method applied to cache described in the present invention, in the step S1, after log 1 is successfully written into the log space header, file A in the memory is updated and success is returned to the client.

[0015] As a preferred solution of the log space recovery optimization method applied to cache described in the present invention, in the step S1, the log number that has been written to the file is recorded in the memory file.

[0016] As a preferred solution of the log space recovery optimization method applied to the cache described in the present invention, in the step S2, when the forced flush is triggered, the log list of the file to which the log record stuck at the tail of the log space belongs is taken out and aggregated to generate a new log 9.

[0017] As a preferred solution of the log space recovery optimization method applied to cache described in the present invention, after the new log 9 is aggregated and generated, the log 9 is written into the current log space header.

[0018] As a preferred solution of the log space recovery optimization method applied to cache described in the present invention, after log 9 is written successfully, the effective starting position of the update log is log 9.

[0019] As a preferred solution of the log space recovery optimization method applied to cache described in the present invention, after the effective starting position of the new log, the log list of file A is updated to log 9.

[0020] As a preferred solution of the log space recovery optimization method applied to cache described in the present invention, the log list of the file to which the log record stuck at the tail of the log space belongs is composed of log 1, log 2 and log 7.

[0021] As a preferred solution of the log space recovery optimization method applied to cache described in the present invention, when the effective starting position of the update log is log 9, log 1, log 2 and log 7 become invalid.

[0022] Beneficial effects of the present invention:

[0023] The present invention formats the logs that block the release of log space into new logs after aggregation, rewrites the log space header, and releases the blocked tail space without flushing. The present invention can significantly reduce the data flushing to the back-end disk when the business volume is large and the log space is almost exhausted, thereby improving the performance of the log system in this scenario. At the same time, the present invention does not require additional log space, and reuses the previous log process, which can improve the utilization rate of the log space to a certain extent. In addition, aggregating multiple old logs belonging to the same file into a new log for writing can save log space consumption to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0025] Figure 1 Write a schematic diagram for the log;

[0026] Figure 2 This is a schematic diagram of log space recovery of the present invention;

[0027] Figure 3 This is a schematic diagram of log space recovery of the present invention;

[0028] Figure 4 The figure is a schematic diagram of log space recovery under the prior art. DETAILED DESCRIPTION

[0029] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0030] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0031] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0032] Secondly, the present invention is described in detail with reference to the schematic diagram. When describing the embodiments of the present invention in detail, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0033] Reference Figure 1-4 The present invention provides a method for optimizing the recovery of log space applied to a cache, comprising the following steps:

[0034] S1: Write log 1 to the head of the log space:

[0035] The system receives a write request from the client, formats the request into log 1, and writes log 1 into the log space header. After log 1 is successfully written into the log space header, file A in the memory is updated, and success is returned to the client (Note: the file here is an abstract concept of data management collection, similar to files in the file system and objects in object storage). The log number that has been written to this file is recorded in the memory file. The purpose of recording the written log number is to facilitate the flushing process.

[0036] S2: Reclaim the log space:

[0037] When forced flushing is triggered, the log list (log 1, log 2, and log 7) of the file to which the log record stuck at the tail of the log space belongs is taken out, aggregated to generate a new log 9, and log 9 is written to the head of the current log space. After log 9 is written successfully, the effective starting position of the update log is log 9, and log 1, log 2, and log 7 are invalidated. The log list of file A is updated to log 9.

[0038] Appendix: The log writing process under the prior art is: 1. The system receives a write request from the client, formats the request into log 1, and writes it into the head of the log space; 2. After writing log 1 successfully, the file A in the memory is updated, and success is returned to the client. The existing log writing process is basically the same as S1 of the present invention, except that the operation of recording the log number that has been written to the file in the memory file is omitted.

[0039] The log space recovery process under the existing technology is: 1. File A is written and flushed to the backend disk; 2. After file A is successfully written, the effective starting position of the update log is log 2 (log 1 is invalid).

[0040] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for optimizing the recovery of log space applied to cache, characterized in that: The following steps are involved: S1: Write log 1 to the log space header. After the system receives the client's write request, it formats the request into log 1, and then writes log 1 to the log space header. After log 1 is successfully written to the log space header, file A in the memory is updated, and success is returned to the client. S2: Reclaim the log space. When a forced flush is triggered, the log list of the file to which the log record stuck at the tail of the log space belongs is taken out and aggregated to generate a new log 9. After the new log 9 is aggregated and generated, log 9 is written to the head of the current log space. After log 9 is written successfully, the effective starting position of the update log is log 9.

2. The method for optimizing the recovery of log space applied to cache according to claim 1, characterized in that: In the step S1, the log number of the file that has been written is recorded in the memory file.

3. The method for optimizing the recovery of log space applied to cache according to claim 1, characterized in that: After the effective start position of the update log, the log list of file A is updated to log 9.

4. The method for optimizing the recovery of log space applied to cache according to claim 1, characterized in that: The log list of the file to which the log record stuck at the end of the log space belongs consists of log 1, log 2, and log 7.

5. The method for optimizing the recovery of log space applied to cache according to claim 4, characterized in that: When the effective starting position of the update log is log 9, logs 1, 2, and 7 become invalid.

Images