A pre-write log storage method, system and device

Abstract

The application discloses a pre-write log storage method, system and device, and belongs to the computer storage field. In view of the low safety and low efficiency of the prior art, the application provides a pre-write log storage method, system and device, which calculates the physical position of each pre-write log by managing the storage medium space segment, realizes the pre-write log sequential writing and sequential elimination. When the pre-write log becomes invalid, the invalid pre-write log is recycled according to the position sequence of the pre-write log in the storage medium, the invalid log is not erased in the storage medium when the space of the invalid log is recycled, and the new pre-write log is directly written into the recycled space to cover the recycled invalid pre-write log. The application can realize efficient management of the data storage pre-write log, supports fast data writing, and realizes fast elimination and recycling.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer storage, more particularly, to a pre-write log storage method, system and device. BACKGROUND

[0002] In the field of data storage, in order to ensure the consistency of data, the associated data needs to be written into a pre-write log before being actually stored in the storage medium, so as to ensure that the associated data can be modified or not modified at the same time when the machine fails, and the situation of being modified in part does not occur. Since this function is an indispensable function in the storage system, the pre-write log occurs before the data is actually persisted, and its performance and availability also seriously affect the performance and availability of the entire storage system, so efficient and reliable management of the pre-write log also becomes particularly important.

[0003] When a set of systems modify one or more related data in one or more storage media or related data between multiple nodes, we want these related data to be modified successfully or not to be modified, and do not want to appear the situation of being modified in part. If the data in the storage medium is directly modified, the biggest risk is that if the node crashes during the modification process, only part of the modification will occur. When the node restarts, we cannot repair it. Therefore, in order to ensure data write consistency, a pre-write log is often performed before the associated data is actually written, and the pre-write log also needs to be persisted to the storage medium. When the pre-write log is successfully written to the storage medium, the associated data in the storage medium is modified.

[0004] In the prior art, the pre-write log function is often set in the storage engine of the database, such as Chinese patent application No. 202210158482.8, published on June 3, 2022, which discloses a method, device and system for processing pre-write logs, wherein the method for processing pre-write logs comprises: providing a publishing interface and a subscribing interface based on a message queue communication protocol; receiving a pre-write log write request through the publishing interface; obtaining a pre-write log record according to the data carried by the pre-write log write request; storing the pre-write log record into a message queue, so that the subscribing end consumes the log record through the subscribing interface; wherein the log sequence number in the pre-write log record is determined based on the corresponding rule between the message offset and the log sequence number, the message offset is the message offset of the pre-write log record in the message queue, and the log sequence number is used to make the subscribing end determine the message offset corresponding to the pre-write log record that needs to be played back based on the corresponding rule between the message offset and the log sequence number. However, there is still a possibility of server downtime and data loss, and the read-write speed is not high. SUMMARY

[0005] 1. Technical problem to be solved

[0006] In view of the problems of low safety and low efficiency in the prior art, the present application provides a pre-write log storage method, system and device, which can realize efficient management of data storage pre-write logs, support fast data writing and fast elimination and recycling.

[0007] 2. Technical solution

[0008] The object of the present application is achieved by the following technical solutions.

[0009] A method of a pre-write log storage system, steps of which are as follows,

[0010] The pre-write log is directly written into the storage medium without passing through any intermediate system such as a file system, a database or an operating system cache, etc. The pre-write log needs to be sequentially written when being written into the storage medium, and the sequence is written according to the position sequence of the storage medium. When the pre-write log becomes invalid, the storage space occupied by the invalid log needs to be recycled: the recycling strategy of the invalid log is recycled according to the position sequence in the storage medium, that is, the pre-write log written into the storage medium earliest needs to be recycled first; when recycling the space of the invalid log, the invalid log does not need to be erased in the storage medium, but when a new pre-write log needs to be written into the storage medium, the new pre-write log is written into the recycled space, that is, the storage medium position corresponding to the recycled invalid log, and the invalid log is directly overwritten.

[0011] Further, the size of the pre-write log is chunkWriteSize, the writePosition is the position of the new pre-write log allocated in the storage medium, the readPosition is the position of the first submitted but valid pre-write log in the storage medium, and the capacity is the size of the storage medium space occupied by the content to be stored. The specific moving start position is as follows,

[0012] Write log: move writePosition, and the moving method is writePosition=writePosition+chunkWriteSize or writePosition=writePosition+chunkWriteSize–capacity;

[0013] Recycle log: move readPosition, and the moving method is readPosition=readPosition+chunkReadSize or readPosition=readPosition+chunkReadSize–capacity.

[0014] Further, the storage medium is divided into two areas, NewRingBuffer and OldRingBuffer. When recycling the log, if the first submitted but not discarded pre-write log cannot be recycled, it is determined whether the creation time of the pre-write log exceeds a threshold or the available space of the entire NewRingBuffer is lower than a threshold. If either condition is met, a space is applied in the OldRingBuffer, the pre-write log is copied from the NewRingBuffer to the space managed by the OldRingBuffer, and the pre-write log in the NewRingBuffer is discarded. This solves the problem of long-time non-recyclable pre-write logs.

[0015] Further, the pre-write log outputs a pre-write result after being written to the storage medium. The pre-write result refers to a corresponding response output to the user program according to whether the pre-write log is successfully written to the storage medium, and the user program will perform subsequent operations.

[0016] Further, the readPosition and writePosition data are saved according to a threshold. The data is corresponding position data, and the specific threshold can be a time value, such as being saved at a fixed time or being saved at a set time sequence. Alternatively, the threshold can be a space value, such as being solidified when WritePosition-readPosition is greater than a certain threshold, or being solidified when the size of a space interval is exceeded. Alternatively, a solidified writePosition is recorded in the memory, and new writePosition minus the solidified writePosition is greater than a certain threshold to be solidified. The saving or solidification is essentially a backup process.

[0017] Further, when a restart occurs, the readPosition and writePosition data are reacquired, the effective pre-write logs between the readPosition and the writePosition are read out and submitted to the user program for processing. As long as the corresponding readPosition and writePosition data, i.e., the corresponding positions, are obtained, it can be known where the previously written pre-write logs are, which is an addressing process. Because the specific content of the corresponding pre-write logs will not be lost during the restart, it is not necessary to write again, but only to find.

[0018] Further, when encountering a pre-write log that cannot be eliminated, the current log is determined whether it needs to be moved from the NewRingBuffer to the OldRingBuffer. If not, the task is terminated and waits for the next execution; if yes, the log is moved from the NewRingBuffer to the OldRingBuffer, and the readPosition of the NewRingBuffer is moved.

[0019] A pre-write log storage system comprises:

[0020] A space management module maps the storage medium space into a head-to-tail data structure, and manages three parameters of the data structure, capacity, writePosition and readPosition, wherein the capacity represents the size of the storage medium space, the readPosition represents the position of the first submitted but valid pre-write log in the storage medium, and the writePosition represents the position of the latest valid pre-write log in the storage medium; the space management module calculates the writePosition of the new log, so as to realize that the new pre-write log can be sequentially written into the storage medium;

[0021] A new log writing module

[0022] The storage medium position for writing the pre-write log is applied through the space management module, and the new pre-write log is written into the storage medium according to the returned result;

[0023] A space recycling module

[0024] The readPosition is obtained through the space management module, the first submitted but not eliminated pre-write log is obtained, it is confirmed whether the corresponding pre-write log can be eliminated, and the space management module is notified to move the readPosition if the pre-write log can be eliminated.

[0025] Further, the effective space management module is further included, the readPosition and the writePosition of the space management module are recorded in the persistent storage medium by the effective space management module at regular intervals, and the readPosition and the writePosition of the space management module are obtained from the persistent storage medium after the machine restarts.

[0026] Further, the recovery module is further included, the readPosition and the writePosition of the space management module are read after the user process is reloaded and started after the machine restarts, then the valid pre-write logs between the readPosition and the writePosition are read out and submitted to the user program for processing by the user program.

[0027] Further, the space management module manages the storage medium into two areas, NewRingBuffer and OldRingBuffer, when a user newly submits a pre-write log, first applies space from NewRingBuffer, and when NewRingBuffer recycles space, if the first submitted but not discarded pre-write log cannot be recycled, it is judged whether the creation time of the pre-write log exceeds a certain threshold or the available space of the entire NewRingBuffer is lower than a threshold, if any of the judgment conditions is established, the pre-write log is defined as a timeout pre-write log. For the timeout pre-write log, we need to apply a space in OldRingBuffer, copy this pre-write log from NewRingBuffer to the space managed by OldRingBuffer, and the timeout pre-write log in NewRingBuffer is discarded.

[0028] Further, the space recycling module further includes the following functions, when encountering a pre-write log that cannot be discarded, it is judged whether the current log needs to be moved from NewRingBuffer to OldRingBuffer, if not, the task is terminated, and waits for the next execution, if yes, the log is moved from NewRingBuffer to OldRingBuffer, and the space management module is notified to move readPosition.

[0029] Further, it further includes a user response module, which outputs according to the result of the pre-write log written to the storage medium, and responds to the user request.

[0030] An electronic device, comprising a processor, an input device, an output device and a memory, the processor, the input device, the output device and the memory are connected in sequence, the memory is used to store a computer program, the computer program includes program instructions, the processor is configured to call the program instructions, and the method described above is executed.

[0031] 3. Advantages

[0032] Compared with the prior art, the advantages of the present application are:

[0033] Based on the pre-write log read and written to the storage medium, without passing through any memory in the middle, the memory loss caused by server restart is prevented, data loss is avoided, data is safer, the storage of the pre-write log directly operates the storage medium, sequential IO is used, performance is higher, the recycling of the pre-write log does not need to erase data, reduces the IO operation of the storage medium, improves the performance of the storage medium, is more efficient, and a certain pre-write log does not recycle for a long time, which will not affect the recycling of the space of the subsequent pre-write log and the writing of the new pre-write log, ensures that the available storage space is sufficient, and is more practical. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of the write-ahead log storage area managed by the space management module;

[0035] Figure 2 This is a flowchart of the write-ahead log of the present invention;

[0036] Figure 3 Flowchart for write-ahead log recovery;

[0037] Figure 4 Recovery flow chart for write-ahead log. DETAILED DESCRIPTION

[0038] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] Example 1

[0040] Based on the requirement for fast and stable writing of the above-mentioned write-ahead log, this solution designs the following system and method. This solution uses a storage medium to directly store the write-ahead log. The write-ahead log is written directly to the disk in a direct manner without going through the cache of the file system and the storage medium, which solves the problem of data loss due to server crashes, while ensuring higher performance of reading and writing the write-ahead log, which is safer and more efficient. The storage medium may include but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or components, or any combination of the above. More specific examples of storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0041] We have a pre-set rule that allows us to perform different operations for different situations. For storage, especially distributed storage, where multiple storage devices are available, we require that each independent block of storage store identical content. For example, if there are three storage media, one serving as the primary disk and two serving as backup disks, but all are required to be written to, there are multiple possibilities for writing. For distributed storage, we need to write the write-ahead log directly to the raw disk. Of course, for other systems, this can also be done using one or a combination of the aforementioned storage media.

[0042] 1. If the machine crashes before the write-ahead log is successfully written, the write-ahead log will be discarded after the machine restarts, but the associated data in the storage medium will not be modified, meeting our expectation of no modification.

[0043] 2. If the pre-write log is successfully written, the associated data modification part of the machine is down, and after restarting, we can continue to modify the associated data according to the content of the pre-write log to achieve the expected modification of the associated data.

[0044] 3. If the associated data is successfully modified, the pre-write log is discarded, and the space is recycled.

[0045] We designed a pre-write log storage method. The present scheme designs a pre-write log storage method without using cache, and directly reads and writes pre-write logs based on storage media. Since we directly manage the storage media, we will encounter a major problem, that is, we need to decide where the new log needs to be written to the storage medium. We initialize the storage medium, map the storage medium space into a ring buffer in the memory space, and calculate the physical address of the data on the storage medium through the ring buffer to achieve sequential writing to the storage medium.

[0046] Specifically, as shown in Figure 1 , we maintain three parameters: capacity, writePosition, and readPosition. The capacity represents the size of the storage medium, the readPosition represents the position of the first submitted but valid pre-write log on the storage medium, and the writePosition represents the position of the latest valid pre-write log on the storage medium. Therefore, the space between readPosition and writePosition is the location of all valid logs.

[0047] As shown in Figure 2 , a new pre-write log needs to be allocated a new storage medium space. By calculating the size of the newly written log, that is, chunkWriteSize, the current writePosition is the starting position of the newly written log allocated on the storage medium. Then move the writePosition, that is, the new writePosition = writePosition + chunkWriteSize, to prepare for the next allocation. This allocation method determines that each new pre-write log is appended to the disk, that is, append write.

[0048] The oldest valid log is discarded. If the pre-write log corresponding to readPosition can be discarded, move the new readPosition = readPosition + chunkReadSize. It should be noted that Ringbuffer can only sequentially discard, as shown in Figure 1If chunk1 cannot be recycled, readPosition cannot be moved no matter whether chunk2 can be recycled or not. The elimination does not need to really erase the data of the storage medium, but only moves the readPosition. When there is data in this part later, it is a kind of covering mode, so the process of erasing the data of the storage medium can be omitted, and the efficiency will be higher.

[0049] Since the space between readPosition and writePosition is the position where all valid logs are located, when writePosition+chunkWriteSize>readPosition indicates that the available space cannot carry the pre-written log this time, the pre-written log submission fails.

[0050] During the continuous movement of writePosition, the condition of >capacity will always occur. If writePosition+chunkWriteSize>capacity, then the new writePosition=writePosition+chunkWriteSize–capacity. The purpose of this is to make the storage medium space "look like" a ring structure, and readPosition also needs the same processing, readPosition=readPosition+chunkReadSize–capacity.

[0051] Of course, we can also assume that the same storage medium is connected at the end and infinitely extended, so the new writePosition=writePosition+chunkWriteSize, and the new readPosition=readPosition+chunkReadSize. However, when actually reading and writing the storage medium, writePositionInDisk=writePosition%capacity, and readPositionInDisk=readPosition%capacity. The % symbol is the remainder, writePositionInDisk is the position of writing the storage medium, and readPositionInDisk indicates the position of reading the storage medium. For example, the size of the storage medium is 1G, but at this time, due to the movement of the position to 1.2G, the remainder is 0.2G. In fact, the position of writing and reading the storage medium is at the position of 200M.

[0052] When applying for space, because the RingBuffer structure is actually in the "chasing" mode, therefore, writePosition + chunkWriteSize > readPosition indicates that the available space cannot bear this pre-write log. Because it is infinite extension, therefore, writePosition + chunkWriteSize - readPosition > capacity indicates that the available space cannot bear this pre-write log.

[0053] The above space management module constructs a ring or infinite extension of the storage medium space for storing pre-write logs. According to the introduction of the space management module, we know that the elimination of RingBuffer is strictly sequential elimination, that is, the pre-write log submitted first must be eliminated first, and if the pre-write log submitted first cannot be eliminated, the entire RingBuffer cannot allocate new space. However, in the real scene, there may be some pre-write logs corresponding to the associated data modification which is more complex and requires longer processing time and cannot be eliminated in a short time. In order to solve this problem, we introduce the method of partition management.

[0054] When initializing the storage medium, we logically divide it into two areas. For example, a 200G storage medium, we give 0~180G space to NewRingBuffer for management, NewRingBuffer is a new ring data structure, and 180G~200G space is given to OldRingBuffer for management, that is, the old ring data structure. The space of NewRingBuffer and OldRingBuffer is managed independently and does not affect each other. And add a creation time to each pre-write log. When recycling space in NewRingBuffer, if the pre-write log submitted first but not eliminated cannot be recycled, it is judged whether the creation time of this pre-write log exceeds a certain threshold or the available space of the entire NewRingBuffer is lower than a threshold. If any of the judgment conditions is met, a space is applied for in OldRingBuffer, and this pre-write log is copied from NewRingBuffer to the space managed by OldRingBuffer. The timeout pre-write log in NewRingBuffer can be eliminated. It is guaranteed that the log which cannot be eliminated in a short time due to longer processing time will not affect the writing and recycling of subsequent pre-write logs.

[0055] For the above scheme, we design a system based on the above pre-write log storage method, which includes a space management module, a new log writing module, a space recycling module, an effective space solidification module, a recovery module and a user response module.

[0056] The above module is a function module divided according to functions, each function can be set in an independent device to run, the device can be an entity or a virtual module, or a plurality of functions in any combination in one device, and is not completely limited, as long as the function module of the above method can be within the scope of our protection.

[0057] Space management module: for managing and allocating the space of the above storage medium.

[0058] New log writing module: after receiving the pre-written log of the user program, the pre-written log should be written to the storage medium position through the space management module RingBuffer, and the result is written to the storage medium according to the returned result, and the corresponding result is returned through the response module.

[0059] Space recycling module: as shown in Figure 3 The space recycling module is a task module for recycling. The readPosition is obtained through the RingBuffer to obtain the first submitted but not eliminated pre-written log, and the user program is inquired whether the pre-written log can be eliminated. If it can, notify the RingBuffer to move the readPosition. Until the pre-written log that cannot be eliminated is encountered. When the log that cannot be eliminated is encountered, it is judged whether the current pre-written log needs to be moved from the NewRingBuffer to the OldRingBuffer. If not, terminate the task and wait for the next execution. If it is needed, the pre-written log is moved from the NewRingBuffer to the OldRingBuffer, and the NewRingBuffer is notified to move the readPosition.

[0060] Effective space management module: effective space management has two functions: one, as an independent task. The readPosition and writePosition of RingBuffer can be recorded in a certain place of the storage medium according to the threshold value. This storage is not disturbed by restart and power failure, and is a persistent save. Two, the readPosition and writePosition of RingBuffer are obtained through the effective space management module after the machine restarts. It effectively ensures that it can be quickly recovered when a machine failure occurs, which is equivalent to a backup and recovery process. This process can be set to a certain time, such as 5 minutes once, or other times. The persistent save here is placed in a persistent storage medium, which can be part of the original storage medium, or can be independently set, such as a database, redis, etc.

[0061] Recovery module: as shown in Figure 4As shown, the recovery module is only used when the user process is reloaded after the machine restarts, and the recovery module should be loaded first.

[0062] The recovery module first reads the readPosition and writePosition of the space management module, and then reads the valid pre-write logs between the readPosition and writePosition and submits them to the user process for processing by the user process.

[0063] Preferably, a user response module is further included to output corresponding responses to the user program according to the results of the pre-write logs written to the storage medium. The corresponding system and method designed according to the present scheme can ensure that the allocation and recovery of the storage medium space are performed using a data structure similar to a Ringbuffer when storing in the specific storage medium. This data structure allows all pre-write logs to be written to the storage medium in an append write manner, improving performance and facilitating management. Once the data related to the pre-write logs is successfully submitted, i.e., the associated data of the storage medium in the system is modified, these pre-write logs can be discarded. The space recovery process is the process of recovering the discarded logs, i.e., recovering the space of these logs for use by new pre-write logs.

[0064] Since we write new pre-write logs while recovering the space occupied by discarded pre-write logs, the space recovery process does not require erasing data. When writing new pre-write logs, it is only necessary to overwrite the discarded pre-write logs. The pre-write logs are stored in separate zones according to their survival time, and the zones are discarded. Long-time unsubmitted pre-write logs are stored separately to prevent the pre-write logs from being discarded due to space occupation, thereby preventing subsequent pre-write logs from being submitted.

[0065] Since the validity of the logs needs to be determined when the storage system is restarted, the usual method requires reading all the logs and checking their status to determine their validity. This method requires loading too many invalid logs and is too slow. In the process of writing logs and log space recovery, we record the location of the valid logs on the storage medium, i.e., their start and end positions, in a certain place in the storage medium, which is persistent. When restarting, the positions of all valid logs are first obtained, and then only the data from the start position to the end position of the limited range is read from the storage medium to avoid reading the entire storage medium, and then the logs in the limited range are read to determine their validity, which greatly reduces the number of logs read and improves the time for the system to recover to a normal state. The present scheme can ensure that the pre-write log storage occupies less space, runs quickly, and has good safety through the above method.

[0066] Correspondingly, we can set an electronic device, including a processor, an input device, an output device and a memory, the processor, the input device, the output device and the memory are connected in turn, the memory is used for storing a computer program, the computer program includes program instructions, the processor is configured to call the program instructions, and the method described above is executed. This device is a specific device, and the specific form is not limited as long as it can run the above method.

[0067] The above describes the present application and its embodiments schematically, which is not limited, and the present application can be realized in other specific forms without departing from the spirit or essential characteristics of the present application. The embodiments shown in the drawings are only one of the embodiments of the present application, and the actual structure is not limited thereto, and any reference signs in the claims should not limit the claims involved. Therefore, if a person skilled in the art is inspired thereby, without departing from the spirit of the present application, similar structural forms and embodiments can be designed without creativity, which should belong to the protection scope of the present patent. In addition, the word "comprising" does not exclude other elements or steps, and the word "one" before an element does not exclude "multiple" elements. The multiple elements stated in the product claim can also be realized by one element through software or hardware. The words "first", "second" and the like are used to represent names, and do not represent any specific order.

Claims

1. A write-ahead log storage method, the steps of which are as follows: After receiving the write-ahead log from the user program, the system calculates the size of the newly written write-ahead log and applies for the non-volatile storage medium location to which the write-ahead log is to be written. The write-ahead log is written directly to the non-volatile storage medium in the order of its location, without passing through any intermediate system. When the write-ahead log becomes invalid, it is recycled in order according to its location in the non-volatile storage medium. When the space of the invalid log is reclaimed, the non-volatile storage medium does not erase the invalid log. Instead, when a new write-ahead log is written, the new write-ahead log overwrites the location of the recycled invalid write-ahead log in the non-volatile storage medium.

2. A write-ahead log storage method according to claim 1, characterized in that: The size of the write-ahead log is chunkwriteSize, writePosition is the position of the new write-ahead log on the non-volatile storage medium, readPosition is the position of the first valid write-ahead log on the non-volatile storage medium, and capacity is the amount of space on the non-volatile storage medium that the content needs to be stored. The specific method of moving the starting position is as follows: Write log: After writePosition is applied, we need to move writePosition to prepare for the next write. The moving method is writePosition = writePosition + chunkWriteSize or writePosition = writePosition + chunkWriteSize – capacity. Reclaim log: Move readPosition, used for space reclamation of non-volatile storage media. The movement method is readPosition = readPosition + chunkReadSize or readPosition = readPosition + chunkReadSize – capacity.

3. A write-ahead log storage method according to claim 1 or 2, characterized in that: The non-volatile storage medium is divided into two areas: NewRingBuffer and OldRingBuffer. When recycling logs, if the earliest written-ahead log submitted but not eliminated cannot be recycled, it is determined whether the creation time of this written-ahead log exceeds the threshold or whether the available space of the entire NewRingBuffer is less than the threshold. If either judgment condition is met, a space is allocated in the OldRingBuffer, and the written-ahead log is copied from the NewRingBuffer to the space managed by the OldRingBuffer. The timed-out written-ahead log in the NewRingBuffer is then eliminated.

4. A write-ahead log storage method according to claim 1, characterized in that: The write-ahead log outputs the pre-written results after writing them to the non-volatile storage medium.

5. The write-ahead log storage method according to claim 1, wherein: According to the set threshold, save the readPosition and writePosition information data.

6. A write-ahead log storage method according to claim 5, characterized in that: When a restart occurs, the readPosition and writePosition information is retrieved, and the valid write-ahead log between the readPosition and writePosition is read out and submitted to the user program for processing.

7. A write-ahead log storage method according to claim 3, characterized in that: When encountering a pre-written log that cannot be eliminated, determine whether the current log needs to be moved from NewRingBuffer to OldRingBuffer. If not, terminate the task and wait for the next execution; If necessary, move the log from NewRingBuffer to OldRingBuffer and move the readPosition of NewRingBuffer.

8. A write-ahead log storage system, characterized in that: include: The space management module maps the non-volatile storage medium space into a data structure connected end to end, and manages the three parameters of the data structure: capacity, writePosition, and readPosition. Capacity represents the space size of the non-volatile storage medium, readPosition represents the position of the first valid write-ahead log on the non-volatile storage medium, and writePosition represents the position of the latest valid write-ahead log on the non-volatile storage medium. The space management module calculates the writePosition of the new log to ensure that the new write-ahead log can be written to the non-volatile storage medium in sequence. New log writing module Request the non-volatile storage medium location for writing the write-ahead log through the space management module, and write the new write-ahead log to the non-volatile storage medium based on the returned result; Space Reclamation Module The readPosition is obtained through the space management module, and the earliest written-ahead log submitted but not eliminated is obtained. The corresponding written-ahead log is confirmed to be eligible for elimination. If it is eligible, the space management module is notified to move the readPosition.

9. A write-ahead log storage system according to claim 8, characterized in that: It also includes an effective space management module, which records the readPosition and writePosition of the space management module in a persistent non-volatile storage medium according to a set threshold, and obtains the readPosition and writePosition of the space management module from the persistent non-volatile storage medium after the machine is restarted.

10. A write-ahead log storage system according to claim 8 or 9, characterized in that: It also includes a recovery module. After the machine restarts, the user process starts when it is reloaded. It reads the readPosition and writePosition of the space management module, and then reads the valid pre-write log between the readPosition and writePosition and submits it to the user program for processing.

11. A write-ahead log storage system according to claim 10, characterized in that: The space management module manages non-volatile storage media into two areas: NewRingBuffer and OldRingBuffer. When a user submits a newly submitted write-ahead log, space is first requested from NewRingBuffer. During space reclaim in NewRingBuffer, if the earliest submitted but not eliminated write-ahead log cannot be reclaimed, the module determines whether the creation time of the write-ahead log exceeds a certain threshold or whether the available space in the entire NewRingBuffer is below a threshold. If any of these conditions are met, the write-ahead log is defined as a timed-out write-ahead log. For timed-out write-ahead logs, space is requested in OldRingBuffer and the write-ahead log is copied from NewRingBuffer to the space managed by OldRingBuffer. The timed-out write-ahead log in NewRingBuffer is then eliminated.

12. A write-ahead log storage system according to claim 11, characterized in that: The space recycling module also includes the following functions. When encountering a pre-written log that cannot be eliminated, it determines whether the current log needs to be moved from NewRingBuffer to OldRingBuffer. If not, it terminates the task and waits for the next execution. If necessary, it moves the log from NewRingBuffer to OldRingBuffer and notifies the space management module to move the readPosition.

13. A write-ahead log storage system according to claim 8, characterized in that: It also includes a user response module, which outputs the result of writing the pre-write log to the non-volatile storage medium and responds to user requests.

14. An electronic device, characterized in that: The method comprises a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are connected in sequence, the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 7.

Images