Write-Ahead Log Service Method, Apparatus, Device, and Storage Medium
By generating and managing logical log sequence numbers and physical log sequence numbers on the log server, and using the logical log sequence sliding window for log playback, the problem of inconsistent submission order and storage order of concurrent write logs in the distributed storage engine is solved, and the system's fault tolerance and throughput capabilities are improved.
Patent Information
- Application Number
- CN202510288006.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-12
- Publication Date
- 2025-05-27
- Estimated Expiration
- 2045-03-12
AI Technical Summary
In the distributed storage engine, the commit order and storage order of concurrent write logs in the pre-write log service are inconsistent, which leads to the system being unable to ensure the order and uniqueness of log records during recovery, affecting the atomicity and persistence of data.
On the log server, it receives the log record request submitted concurrently by the state machine, generates the logical log sequence number and the physical log sequence number, obtains the mapping relationship of the log record through the logical log sequence sliding window, and correctly plays back the log record when the state machine restarts.
Through the mapping relationship between logical log sequence numbers and physical log sequence numbers, the uniqueness and order of target logs are ensured, and the fault tolerance and throughput capability of the system are improved.
Smart Images

Figure CN119807154B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of log service, and in particular to a pre-written log service method, device, equipment and storage medium. Background Art
[0002] Write-ahead logging (WAL) is a series of technologies used by data management systems to provide atomicity and durability, which can ensure that the system can fully recover the committed transaction data after any failure. The core idea of write-ahead logging is to write data to the log before writing it to the database, and then change the log record to the storage. This mechanism ensures the atomicity and durability of data, that is, in the event of a system crash or other abnormal situation, the write-ahead log can be used to recover the data to ensure that the data will not be lost. For example, transaction A inserts two rows of data and commits it, and transaction B inserts two rows of data and is then rolled back. At this time, the system fails and recovers and restarts. The system should ensure that the complete data of transaction A is recorded in the system, and the rolled back transaction B has no data record.
[0003] Traditional stand-alone storage engines write pre-written log files to local disks to optimize random changes to multiple data files into sequential writes of a single file. Combined with group commits, a single FSYNC (operation to synchronize all modified file data in memory to disk) log file can commit multiple transactions. Each log record is identified by a unique log sequence number LSN (Logging Sequence Number), which increases monotonically globally. When the system is restored, the log records are replayed one by one in the order of LSN and applied to the memory state machine.
[0004] Compared with traditional stand-alone storage engines, the new distributed storage engine no longer chooses to write to local write-ahead log files sequentially to meet the reliability of write-ahead log data, data throughput requirements, and simplicity of architecture. Instead, it makes writing logs a service. However, writing logs faces some technical challenges:
[0005] On the one hand, the traditional write-ahead log writes log records sequentially to a file. In order to improve the throughput of the log service, multiple concurrent write log records are required to the log service.
[0006] On the other hand, the LSN of each log record in the traditional write-ahead log is globally monotonically increasing, which is consistent with the monotonicity of the physical location of the log record in the log file. This is also guaranteed by the behavior of sequentially writing a single file. When the system recovers, it also relies on the monotonicity of this LSN. However, when writing log services concurrently, it is impossible to guarantee that the LSN of the log record maintains the same monotonicity as its physical storage location. Moreover, if the log record with a larger LSN of the traditional write-ahead log is submitted successfully, it can be guaranteed that the log records with the previous LSN have been submitted successfully. However, in the concurrent write log service scenario, some requests cannot be written successfully due to network reasons. At this time, even if the log record with a higher LSN is submitted successfully, it cannot be guaranteed that all previous log records have been submitted successfully.
[0007] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention
[0008] The main purpose of the present application is to provide a write-ahead log service method, apparatus, device and storage medium, aiming to solve the technical problem of inconsistent submission order and storage order of concurrent write logs in the write-ahead log service.
[0009] To achieve the above objectives, the present application proposes a pre-written log service method, which is applied to a log service end, and the method includes:
[0010] Receive logging requests for target logs submitted concurrently by the state machine;
[0011] Based on the log record request, generate a logical log sequence number and a physical log sequence number of the target log;
[0012] In response to the restart playback request sent by the state machine, obtaining a mapping relationship between the logical log sequence number and the physical log sequence number;
[0013] Based on the mapping relationship between the logical log sequence number and the physical log sequence number, the target log playback is applied to the state machine through a logical log sequence sliding window.
[0014] In one embodiment, the step of generating the logical log sequence number and the physical log sequence number of the target log based on the log record request includes:
[0015] Generate a logical log sequence number of the target log according to the log record request submission order of the target log;
[0016] A physical log sequence number of the target log is generated according to the physical storage location of the target log in the log service end.
[0017] In one embodiment, the step of applying the target log playback to the state machine through a logical log sequence sliding window based on the mapping relationship between the logical log sequence number and the physical log sequence number includes:
[0018] Starting from an initial physical sequence number and an initial logical sequence number, the physical log sequence number is compared with the logical log sequence number;
[0019] When the order of the physical log sequence number corresponds to the order of the logical log sequence number, replaying and applying the target log corresponding to the physical log sequence number to the state machine;
[0020] When the order of the physical log sequence number and the order of the logical log sequence number do not correspond, the window sliding side of the logical log sequence sliding window is slid toward the target direction until a physical log sequence number corresponding to the logical log sequence number appears, and the target log corresponding to the physical log sequence number and the physical log sequence number before it is replayed and applied to the state machine.
[0021] In one embodiment, the method further comprises:
[0022] If any of the log record requests concurrently submitted by the state machine fails to be submitted, a rollback log of the target log is generated, and the rollback logical sequence number of the log record request is recorded in the rollback log;
[0023] Based on the rollback logical sequence number, the logical log sequence sliding window and the target log are adjusted accordingly.
[0024] In one embodiment, the method is applied to a state machine, and the method comprises:
[0025] Concurrently submitting a log record request for a target log to a log service end, so that the log service end generates a logical log sequence number and a physical log sequence number of the target log based on the log record request;
[0026] A restart playback request is sent to the log server so that the log server obtains the mapping relationship between the logical log sequence number and the physical log sequence number, and based on the mapping relationship between the logical log sequence number and the physical log sequence number, the target log playback is applied to the state machine through the logical log sequence sliding window.
[0027] In one embodiment, the step of concurrently submitting a log record request for a target log to the log service end includes:
[0028] Submit a logging request to the target log through the concurrent log writing window;
[0029] When the current number of the log record requests exceeds the maximum window value of the concurrent log write window, the log record requests outside the concurrent log write window are blocked.
[0030] In one embodiment, the method further comprises:
[0031] Initialize the physical log sequence number and the logical log sequence number, and determine the initial physical sequence number of the physical log sequence number and the initial logical sequence number of the logical log sequence number;
[0032] Initialize a concurrent log write window and configure a maximum window value of the concurrent log write window.
[0033] In addition, to achieve the above-mentioned purpose, the present application also proposes a pre-written log service device, the pre-written log service device comprising:
[0034] A request receiving module, used for receiving log record requests for target logs submitted concurrently by the state machine;
[0035] A sequence generation module, used for generating a logical log sequence number and a physical log sequence number of the target log based on the log record request;
[0036] A mapping acquisition module, configured to acquire a mapping relationship between the logical log sequence number and the physical log sequence number in response to a restart playback request sent by the state machine;
[0037] A log playback module is used to apply the target log playback to the state machine through a logical log sequence sliding window based on the mapping relationship between the logical log sequence number and the physical log sequence number.
[0038] In addition, to achieve the above-mentioned purpose, the present application also proposes a pre-written log service device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the pre-written log service method as described above.
[0039] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the pre-written log service method as described above are implemented.
[0040] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the write-ahead log service method as described above are implemented.
[0041] One or more technical solutions proposed in this application have at least the following technical effects:
[0042] The embodiment of the present application proposes a write-ahead log service method, apparatus, device and storage medium, which receives a log record request for a target log concurrently submitted by a state machine; based on the log record request, generates a logical log sequence number and a physical log sequence number of the target log; in response to a restart playback request sent by the state machine, obtains a mapping relationship between the logical log sequence number and the physical log sequence number; based on the mapping relationship between the logical log sequence number and the physical log sequence number, applies the target log playback to the state machine through a logical log sequence sliding window. The present application ensures the uniqueness and sequentiality of the target log through the logical log sequence number and the physical log sequence number, and improves fault tolerance and throughput. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0045] Figure 1 A flowchart of the first embodiment of the pre-written log service method of the present application is provided;
[0046] Figure 2 An example diagram of possible state distribution of logical log sequence numbers and physical log sequence numbers when the write-ahead log service method of this application receives three concurrent log record requests;
[0047] Figure 3 A window sliding example diagram of a logical log sequence sliding window provided by the write-ahead log service method of the present application;
[0048] Figure 4 This is an example diagram of the log server when a request fails to be submitted in the log record request;
[0049] Figure 5 This is an example diagram of the process of replaying the target log to the state machine when a request fails to be submitted in the log recording request;
[0050] Figure 6 This is a schematic diagram of the module structure of the pre-written log service device according to an embodiment of the present application;
[0051] Figure 7This is a schematic diagram of the device structure of the hardware operating environment involved in the pre-written log service method in the embodiment of the present application.
[0052] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0053] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.
[0054] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.
[0055] The main solution of the embodiment of the present application is: applied to the log service end, including: receiving a log record request for a target log concurrently submitted by a state machine; based on the log record request, generating a logical log sequence number and a physical log sequence number of the target log; in response to a restart playback request sent by the state machine, obtaining a mapping relationship between the logical log sequence number and the physical log sequence number; based on the mapping relationship between the logical log sequence number and the physical log sequence number, applying the target log playback to the state machine through a logical log sequence sliding window.
[0056] In this embodiment, for ease of description, the following description is made by taking the identification of a pre-written log service device as the execution subject.
[0057] Write-ahead logging (WAL) is a series of technologies used by data management systems to provide atomicity and durability, which can ensure that the system can fully recover the committed transaction data after any failure. The core idea of write-ahead logging is to write data to the log before writing it to the database, and then change the log record to the storage. This mechanism ensures the atomicity and durability of data, that is, in the event of a system crash or other abnormal situation, the write-ahead log can be used to recover the data to ensure that the data will not be lost. For example, transaction A inserts two rows of data and commits it, and transaction B inserts two rows of data and is then rolled back. At this time, the system fails and recovers and restarts. The system should ensure that the complete data of transaction A is recorded in the system, and the rolled back transaction B has no data record.
[0058] Traditional stand-alone storage engines write pre-written log files to local disks to optimize random changes to multiple data files into sequential writes of a single file. Combined with group commits, a single FSYNC (operation to synchronize all modified file data in memory to disk) log file can commit multiple transactions. Each log record is identified by a unique log sequence number LSN (Logging Sequence Number), which increases monotonically globally. When the system is restored, the log records are replayed one by one in the order of LSN and applied to the memory state machine.
[0059] Compared with traditional stand-alone storage engines, the new distributed storage engine no longer chooses to write to local write-ahead log files sequentially to meet the reliability of write-ahead log data, data throughput requirements, and simplicity of architecture. Instead, it makes writing logs a service. However, writing logs faces some technical challenges:
[0060] On the one hand, the traditional write-ahead log writes log records sequentially to a file. In order to improve the throughput of the log service, multiple concurrent write log records are required to the log service.
[0061] On the other hand, the LSN of each log record in the traditional write-ahead log is globally monotonically increasing, which is consistent with the monotonicity of the physical location of the log record in the log file. This is also guaranteed by the behavior of sequentially writing a single file. When the system recovers, it also relies on the monotonicity of this LSN. However, when writing log services concurrently, it is impossible to guarantee that the LSN of the log record maintains the same monotonicity as its physical storage location. Moreover, if the log record with a larger LSN of the traditional write-ahead log is submitted successfully, it can be guaranteed that the log records with the previous LSN have been submitted successfully. However, in the concurrent write log service scenario, some requests cannot be written successfully due to network reasons. At this time, even if the log record with a higher LSN is submitted successfully, it cannot be guaranteed that all previous log records have been submitted successfully.
[0062] The application ensures the uniqueness and order of the target log through the logical log sequence number and the physical log sequence number, thereby improving the fault tolerance and throughput.
[0063] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a pre-written log service device, etc. The following takes the pre-written log service device as an example to illustrate this embodiment and the following embodiments.
[0064] Based on this, the present application embodiment provides a pre-written log service method, referring to Figure 1 , Figure 1This is a flowchart of the first embodiment of the write-ahead log service method of the present application.
[0065] In this embodiment, the write-ahead log service method is applied to a log service end, and the write-ahead log service method includes steps S10 to S40:
[0066] Step S10: receiving a log record request for a target log concurrently submitted by the state machine.
[0067] It should be noted that the log service end is used to receive, store and manage the log record request submitted from the state machine of the distributed storage engine. The logical log sequence number and the physical log sequence number of the target log are set at the log service end, and the logical log sequence number is used to represent the order of transaction submission, and the physical log sequence number is used to represent the physical location of the log record in the log service. The log record request submitted concurrently can be processed, and the log record is played back in the correct order when the state machine is restarted, thereby ensuring the consistency and the reliability of the distributed storage engine. In one embodiment of the application, the log service end is located on the disk and is used to manage the log.
[0068] In addition, it should be noted that a state machine refers to a computing model that determines its next state based on the input and current state. In one embodiment of the present application, the state machine is responsible for receiving and processing transaction requests from clients or other nodes, or error requests when the state machine is running. These transaction requests may include operations such as reading, writing, updating or deleting data, and submitting log record requests to the log server through the state machine.
[0069] In addition, it should be noted that the target log refers to the write-ahead log WAL (Write-ahead Logging), which is written to the log before the data is written to the database, and then the log record is changed to the memory. In one embodiment of the present application, the target log is used to record transaction requests processed when the state machine is running. The transaction request may include operations such as reading, writing, updating or deleting data. Write-ahead logging is a technology that the storage engine uses to ensure data atomicity and persistence. The modification of the data file is first persisted in the log as a log record, and then the dirty pages of the memory data are asynchronously written to the data file through the checkpoint, and the old log record is destroyed. The log can be regarded as a sliding window on the timeline. The checkpoint pushes this window to slide forward continuously, and the log on the left side of the window will be cleared.
[0070] In addition, it should be noted that a logging request refers to a request submitted by the state machine to the log server, which is used to record the transaction information, state changes or other key events of the state machine during operation. When the state machine executes a transaction, some information that needs to be persistently recorded will be generated, such as data modification, state transition, etc. In order to ensure that this information can be correctly played back and applied when the system is restarted or fault recovered, the state machine will submit this information as a logging request to the log server.
[0071] Specifically, the logging request submitted by the receiving state machine is continuously monitored, and the target log is requested to be recorded.
[0072] Step S20: Generate a logical log sequence number and a physical log sequence number of the target log based on the log record request.
[0073] It should be noted that the logical log sequence number is a monotonically continuously increasing sequence number used to indicate the order in which log record requests are submitted, and is generated according to the order in which log record requests are submitted.
[0074] In addition, it should be noted that the physical log sequence number is used to indicate the physical storage location of the target log corresponding to the log record request in the log server, and is generated according to the physical storage order of the target log.
[0075] Specifically, according to the log record request submission order of the target log, the logical log sequence number of the target log is generated; according to the physical storage location of the target log in the log service end, the physical log sequence number of the target log is generated.
[0076] Step S30, in response to the restart playback request sent by the state machine, obtaining a mapping relationship between the logical log sequence number and the physical log sequence number.
[0077] It should be noted that the restart replay request refers to a request sent by the state machine to the log server to apply the target log replay to the state machine.
[0078] Specifically, when the state machine restarts, it will send a restart playback request to the log server, requesting the log server to play back the target log to the state machine, but requiring the order of the target logs to be played back to be consistent with the order of the log record requests submitted by the transaction. However, the order of log playback depends on the physical position of the target log in the physical storage medium, that is, it depends on the physical log sequence number of the target log, and the two may not be the same. Therefore, after receiving the restart playback request sent by the state machine, in response to the restart state request, the mapping relationship between the logical log sequence number and the physical log sequence number is obtained.
[0079] Step S40: Based on the mapping relationship between the logical log sequence number and the physical log sequence number, the target log is replayed and applied to the state machine through a logical log sequence sliding window.
[0080] It should be noted that the logical log sequence sliding window is a mechanism for ensuring the correctness and continuity of logs when replaying logs after the state machine is restarted. The logical log sequence sliding window slides on the logical log sequence number to control which logs can be applied to the state machine.
[0081] Specifically, starting from an initial physical sequence number and an initial logical sequence number, the physical log sequence number and the logical log sequence number are compared; when the sorting of the physical log sequence number corresponds to the sorting of the logical log sequence number, the target log playback corresponding to the physical log sequence number is applied to the state machine; when the sorting of the physical log sequence number and the sorting of the logical log sequence number do not correspond, the window sliding side of the logical log sequence sliding window is slid toward the target direction until a physical log sequence number corresponding to the logical log sequence number appears, and the target log playback corresponding to the physical log sequence number and the physical log sequence number before it is applied to the state machine.
[0082] The embodiment of the present application adopts the above scheme, by receiving the log record request for the target log submitted concurrently by the state machine; based on the log record request, generating the logical log sequence number and the physical log sequence number of the target log; in response to the restart playback request sent by the state machine, obtaining the mapping relationship between the logical log sequence number and the physical log sequence number; based on the mapping relationship between the logical log sequence number and the physical log sequence number, applying the target log playback to the state machine through the logical log sequence sliding window. The present application ensures the uniqueness and sequentiality of the target log through the logical log sequence number and the physical log sequence number, and improves fault tolerance and throughput.
[0083] Based on the above implementation scheme, in a feasible implementation manner, the step of generating the logical log sequence number and the physical log sequence number of the target log based on the log record request includes S21-S22:
[0084] Step S21, generating a logical log sequence number of the target log according to the log record request submission order of the target log.
[0085] Specifically, the log server monitors and records the log record requests of each transaction corresponding to the target log, and assigns a unique logical log sequence number to each transaction in the order in which the transactions are submitted. The logical log sequence number is an increasing integer, starting from the initial logical sequence number and increasing each time a transaction is submitted. The generated logical log sequence number is stored in the log file together with the corresponding log record. This ensures that each target log has a unique logical log sequence number to identify the submission order of its log record requests.
[0086] Step S22: Generate a physical log sequence number of the target log according to the physical storage location of the target log in the log server.
[0087] Specifically, first determine the layout of the log on the physical storage medium where the log server is located; then determine its physical log sequence number based on the location of the target log stored on the physical storage medium (such as the starting sector number, offset, etc.). This sequence number can be a code based on the physical location, which can uniquely identify the location of each log record on the physical storage medium.
[0088] For better understanding, please refer to Figure 2 , Figure 2 The following is an example of the possible state distribution of the logical log sequence number and the physical log sequence number for receiving three concurrent log record requests. When the state machine submits three concurrent log record requests, each log record request corresponds to the target log of four transactions, and the sequence numbers of the four target logs corresponding to each request are monotonically increasing. Figure 2 In the example, the first log record request contains transactions 1 to 4, the second log record request contains transactions 5 to 8, and the third log record request contains transactions 9 to 12. The first, second, and third do not represent order, but only indicate distinction.
[0089] If all log record requests are submitted and processed successfully, but the state of the physical storage order of the target log on the log server is uncertain, because the request order sent by the state machine can no longer maintain the submission order of the requests on the log server. When there are three log record requests, the physical storage location of the target log on the log server is certain, that is, the physical log sequence number of the target log is certain, then there will be 6 possible states of the logical log sequence number, such as Figure 2 As shown, in Figure 2 "LSN" here means log sequence number.
[0090] Based on the above implementation scheme, in a feasible implementation manner, the step of applying the target log playback to the state machine through the logical log sequence sliding window based on the mapping relationship between the logical log sequence number and the physical log sequence number includes S41~S43:
[0091] Step S41, starting from the initial physical sequence number and the initial logical sequence number, the physical log sequence number and the logical log sequence number are compared.
[0092] It should be noted that the initial physical sequence number refers to the starting sequence number according to the storage order of the target log on the physical storage medium, that is, the starting point of the physical log sequence number.
[0093] In addition, it should be noted that the initial logical sequence number refers to the starting sequence number according to the log record request submission order of the target log, that is, the starting point of the logical log sequence number.
[0094] Specifically, the initial physical sequence number and the initial logical sequence number are used as initial positions, and the sorting order of the physical log sequence number and the sorting order of the logical log sequence number are compared.
[0095] Step S42: When the order of the physical log sequence number corresponds to the order of the logical log sequence number, the target log corresponding to the physical log sequence number is replayed and applied to the state machine.
[0096] Specifically, when the sorting order of the physical log sequence number matches the sorting order of the logical log sequence number, that is, when the order in which the target log is stored on the physical storage medium is the same as the submission order of the log record request corresponding to the target log, the target log stored on the physical log sequence number is replayed and applied to the state machine. Then, the next physical log sequence number and the next logical log sequence number are updated for comparison until all target logs that need to be replayed are replayed and applied to the state machine.
[0097] Step S43, when the sorting of the physical log sequence number and the sorting of the logical log sequence number do not correspond, slide the window sliding side of the logical log sequence sliding window toward the target direction until a physical log sequence number corresponding to the logical log sequence number appears, and then replay the target log corresponding to the physical log sequence number and the physical log sequence number before it and apply it to the state machine.
[0098] Specifically, when there is a mismatch between the sorting order of the physical log sequence number and the sorting order of the logical log sequence number, that is, when the order in which the target logs are stored on the physical storage medium is different from the submission order of the log record requests corresponding to the target log, the logical log sequence number is not monotonically increasing. At this time, the window sliding side of the logical log sequence sliding window will not be updated, and no target log will be applied to the state machine. The window sliding side of the logical log sequence sliding window will continue to slide toward the target direction on the logical log sequence number until the logical log sequence number can form a continuous monotonically increasing sequence. At this time, the target log corresponding to the physical log sequence number and the physical log sequence number before it will be replayed and applied to the state machine.
[0099] For better understanding, please refer to Figure 3 , Figure 3 This is an example diagram of a window sliding for a logical log sequence sliding window. Figure 3 The order of the physical log sequence number and the logical log sequence number do not correspond, that is, the storage order of the target logs in the log server is not arranged in the order in which the log record requests are submitted. When replaying to replay point 1 on the physical log sequence number, because the logical log sequence number in the current logical log sequence sliding window is continuously increasing, the four target logs corresponding to the physical log sequence number "LSN1->4" will be replayed in sequence and applied to the state machine; when replaying to replay point 2, the largest logical log sequence number is LSN12, but the space of "LSN4->12" still lacks four target logs "LSN5->8", which does not meet the constraint that the logical log sequence number in the logical log sequence sliding window needs to increase monotonically and continuously. At this time, the right side of the logical log sequence sliding window will not be updated, and no target log will be applied to the state machine; when replaying to replay point 3, the largest logical log sequence number is LSN12, and all the sequence numbers of "LSN4->12" have entered the logical log sequence sliding window. At this time, the right side of the logical log sequence sliding window is raised to LSN12, and the physical records of the target logs corresponding to the logical log sequence number "LSN1->12" will be replayed and applied to the state machine.
[0100] Based on the above implementation, in a feasible implementation, the method further includes steps S50 to S60:
[0101] Step S50: If any of the log record requests concurrently submitted by the state machine fails to be submitted, a rollback log of the target log is generated, and the rollback logical sequence number of the log record request is recorded in the rollback log.
[0102] It should be noted that the rollback log is a special type of log that records the state that the system needs to roll back to when encountering an abnormal situation. In a distributed system, when a transaction or a series of transactions cannot determine its commit status due to some reasons (such as network failure, log loss, etc.), the system may need to roll back to a stable state before these transactions. The rollback log contains all the information required for the rollback operation, including the target state to be rolled back (that is, the rollback logical sequence number) and the specific rollback operation that may need to be performed.
[0103] In addition, it should be noted that the rollback logic sequence number is a key value recorded in the rollback log, which indicates the target state to which the state machine needs to roll back.
[0104] Specifically, when the state machine concurrently submits multiple log record requests to the log server, it continuously monitors the request responses. When the middle log record request is submitted but no successful submission response is received, and retrying the submission does not work, the requests before and after the unresponsive log record request have received successful submission responses. At this time, the transaction submission corresponding to the unresponsive log record request will be blocked all the time, and the request after the unresponsive log record request is successfully submitted, but it is blocked because the previous transaction submission is blocked, and the corresponding transaction can only be submitted after the unresponsive log record request receives a successful submission response and completes the transaction submission.
[0105] More specifically, when an intermediate log record request is submitted but no successful submission response is received, this does not mean that the request submission is unsuccessful. There may be two situations. The first situation is that the unresponded log record request has been successfully submitted, but due to network and other reasons, the state machine did not receive a successful submission response. Because it is impossible to know whether the submission is successful, the state machine chooses to alarm and kill the current process. After restarting, the log server has a full amount of data, and the corresponding logical log sequence number is a continuous and complete record, which means that all previous transactions were successfully submitted. The second situation is that the unresponded log record request was not successfully submitted. Because it is impossible to know whether the submission is successful, the state machine chooses to alarm and kill the current process. After restarting, the log server lacks the target log corresponding to the unresponded log record request. According to the sliding principle of the logical log sequence sliding window, it is determined that the target log before the unresponded log record request is successfully submitted, and the target logs corresponding to this request and the requests after this request are invalid submissions.
[0106] Furthermore, when the unresponded logging request is a commit failure, the first state machine restart finds that the logical log sequence number is discontinuous, obtains the valid commits and invalid commits in the logical log sequence number, generates a rollback log, stores the rollback log and the target log on the log server, and records the last logical log sequence number of the valid commit in the rollback log as the rollback logical sequence number, that is, the logical log sequence number with the largest sequence number in the valid commit is used as the rollback logical sequence number.
[0107] Step S60: Based on the rollback logic sequence number, the logic log sequence sliding window and the target log are adjusted accordingly.
[0108] Specifically, the fixed side of the logical log sequence sliding window is fixed on the initial logical sequence number of the logical log sequence number, and the sliding side of the logical log sequence sliding window is slid. During the sliding process, the logical log sequence number and the physical log sequence number are compared, and the valid submitted target log is replayed and applied to the state machine. When sliding to the rollback log, the rollback logical sequence number is obtained from the rollback log, and the sliding side of the logical log sequence sliding window is adjusted to the rollback logical sequence number, and the target log that is successfully submitted but invalidly submitted is discarded.
[0109] For better understanding, please refer to Figure 4 and Figure 5 , Figure 4 This is an example diagram of the log server when a request fails to be submitted in the log record request. Figure 5 This is a flow chart showing an example of how to replay the target log to the state machine when a request fails to be submitted in a logging request. Figure 4 The state machine submits three concurrent log record requests to the log server. Each log record request corresponds to the target logs of four transactions. The sequence numbers of the four target logs corresponding to each request are monotonically increasing. Among them, the second log record request fails to be submitted. Therefore, the logical log sequence number of the log server only has the logical log sequence number "LSN1->4" and the corresponding physical log sequence number "LSN1->4" corresponding to the first log record request, and the logical log sequence number "LSN9->12" and the physical log sequence number "LSN5->8" corresponding to the third log record request. According to the sliding principle of the logical log sequence sliding window, because the logical log sequence number is not monotonically increasing, it is determined that the logical log sequence number "LSN1->4" is submitted successfully, and the logical log sequence number "LSN9->12" is submitted unsuccessfully.
[0110] Figure 5The state machine submitted three concurrent log record requests to the log server. Each log record request corresponds to the target log of four transactions. The second log record request failed to be submitted. The first state machine restart found that the logical log sequence number was discontinuous. The valid submission sequence number "LSN1->4" and the invalid submission sequence number "LSN5->12" in the logical log sequence number were obtained, and a rollback log was generated. The rollback log and the target log were stored on the log server. The last valid submission logical log sequence number was recorded in the rollback log as the rollback logical sequence number. Figure 5 In the example, the rollback log is a log with a physical log sequence number of "LSN9". The rollback logical sequence number "LSN4" is recorded in the rollback log. After 8 transactions are subsequently submitted and the log record requests corresponding to the transactions are successfully submitted, the state machine is restarted again. When replaying to playback point 1, the sliding side (right side) of the logical log sequence sliding window is at the logical log sequence number "LSN4". At this time, the previous physical log sequence number and the logical log sequence number are corresponding, so the "LSN1->4" in the physical log sequence number is replayed and applied to the state machine; when replaying to playback point 2 "LSN9", it is determined that this is a rollback log, and the rollback logical sequence number "LSN4" in the rollback log is obtained. The right side of the logical log sequence sliding window is shrunk to "LSN4", and the physical log record corresponding to the logical log sequence number "LSN9->12" that has been replayed by the first state machine restart is replayed. (that is, the target log corresponding to the physical log sequence number "LSN5->8") is discarded; when replaying to replay point 3, the continuous logical log sequence number "LSN5->12" cannot be obtained, and the right side of the logical log sequence sliding window is maintained at the logical log sequence number LSN4; when replaying to replay point 4, the continuous logical log sequence number "LSN5->12" is obtained, and the right side of the logical log sequence sliding window is pushed to the logical log sequence number LSN12, and the target log playback on the physical log sequence number "LSN10->17" corresponding to the logical log sequence number "LSN5->12" is applied to the state machine.
[0111] Based on the above implementation scheme, in a feasible implementation mode, the method is applied to a state machine, and the method includes steps S10' to S20':
[0112] Step S10', concurrently submitting a log record request for the target log to the log service end, so that the log service end generates a logical log sequence number and a physical log sequence number of the target log based on the log record request.
[0113] Specifically, a log record request for a target log is concurrently submitted to a log service end. After receiving the log record request, the log service end generates a logical log sequence number and a physical log sequence number of the target log based on the log record request. The log service end generates the logical log sequence number of the target log according to the order in which the log record request of the target log is submitted; and generates the physical log sequence number of the target log according to the physical storage location of the target log in the log service end.
[0114] Step S20', sending a restart playback request to the log server so that the log server obtains the mapping relationship between the logical log sequence number and the physical log sequence number, and based on the mapping relationship between the logical log sequence number and the physical log sequence number, applies the target log playback to the state machine through the logical log sequence sliding window.
[0115] Specifically, when the state machine restarts, a restart playback request is sent to the log server, requesting the log server to play back the target log to the state machine. After receiving the restart playback request, the log server obtains the mapping relationship between the logical log sequence number and the physical log sequence number. Starting from the initial physical sequence number and the initial logical sequence number, the physical log sequence number and the logical log sequence number are compared; when the order of the physical log sequence number corresponds to the order of the logical log sequence number, the target log playback corresponding to the physical log sequence number is applied to the state machine; when the order of the physical log sequence number does not correspond to the order of the logical log sequence number, the window sliding side of the logical log sequence sliding window is slid toward the target direction until the physical log sequence number corresponding to the logical log sequence number appears, and the target log playback corresponding to the physical log sequence number and the physical log sequence number before it is applied to the state machine.
[0116] Based on the above implementation scheme, in a feasible implementation scheme, the step of concurrently submitting a log record request for a target log to the log service end includes S11'-S12':
[0117] Step S11 ′: submitting a log record request to a target log through a concurrent log writing window.
[0118] It should be noted that the concurrent log write window is used to control the number of log record requests that the state machine can submit. The window ensures that the state machine will not be blocked indefinitely due to waiting for a log record request that has not been responded to for a long time. The number of subsequent log record requests processed is limited by setting a threshold (maximum window value).
[0119] Specifically, the transaction execution of the state machine is obtained, a log record request is generated, and it is checked whether the number of log record requests in the current concurrent log write window has reached the maximum window value. If the current window is not full (that is, the number of log record requests has not reached the maximum window value), the log record request is added to the window, and the log record request is sent to the log server through the concurrent log write window. After processing a log record request, the status of the concurrent log write window is updated to release the occupied space; if the current window is full, a blocking operation is performed.
[0120] Step S12': when the current number of the log record requests exceeds the maximum window value of the concurrent log write window, the log record requests outside the concurrent log write window are blocked.
[0121] Specifically, the status of the concurrent log write window is monitored. When the current number of log record requests in the window exceeds the maximum window value of the concurrent log write window, the blocking mechanism is activated to block new log record requests that exceed the maximum capacity of the concurrent log write window. The problem of physical log sequence number and logical log sequence number not corresponding is solved by the logical log sequence sliding window. The span of disorder needs to be not too large. Therefore, the concurrent log write window and the blocking mechanism are combined to ensure that the span is not too large.
[0122] Through the above solution, this embodiment can better manage and allocate resources by configuring the maximum window value in the concurrent log writing window and combining the blocking mechanism, which helps to avoid resource overload, thereby improving the overall performance of log recording; the blocking mechanism can prevent log loss or disorder when resources are tight, thereby maintaining the consistency and reliability of the log; limiting the number of concurrent requests helps reduce the risk of system crash or instability caused by overload. This can improve the overall stability and reliability of the system.
[0123] Based on the above implementation, in a feasible implementation, the method further includes steps S70 to S80:
[0124] Step S70, initializing the physical log sequence number and the logical log sequence number, and determining an initial physical sequence number of the physical log sequence number and an initial logical sequence number of the logical log sequence number.
[0125] Specifically, the physical log sequence number and the logical log sequence number are initialized, and a starting sequence number (initial physical sequence number) is assigned to the storage of the target log on the physical storage medium; the starting sequence number (initial logical sequence number) of the logical log sequence number is determined according to the submission order of the log record request of the target log.
[0126] Step S80, initializing a concurrent log writing window, and configuring a maximum window value of the concurrent log writing window.
[0127] Specifically, the concurrent log write window is initialized, and a reasonable maximum window value is set according to system performance, resource limitations, and log processing requirements to limit the number of concurrently submitted log record requests. In one embodiment of the present application, a data structure (such as a queue or buffer) is initialized to store the log record requests waiting to be processed. This data structure will serve as the entity of the concurrent log write window.
[0128] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the pre-written log service method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.
[0129] This application also provides a pre-written log service device, please refer to Figure 6 , the pre-written log service device comprises:
[0130] The request receiving module 601 is used to receive the log record request for the target log submitted concurrently by the state machine;
[0131] A sequence generation module 602, configured to generate a logical log sequence number and a physical log sequence number of the target log based on the log record request;
[0132] A mapping acquisition module 603 is used to obtain a mapping relationship between the logical log sequence number and the physical log sequence number in response to a restart playback request sent by the state machine;
[0133] The log playback module 604 is used to apply the target log playback to the state machine through a logical log sequence sliding window based on the mapping relationship between the logical log sequence number and the physical log sequence number.
[0134] The pre-written log service device provided by the present application adopts the pre-written log service method in the above embodiment, which can solve the technical problem of inconsistent submission order and storage order of concurrent log writing in the pre-written log service. Compared with the prior art, the beneficial effects of the pre-written log service device provided by the present application are the same as the beneficial effects of the pre-written log service method provided by the above embodiment, and other technical features in the pre-written log service device are the same as the features disclosed in the above embodiment method, which will not be repeated here.
[0135] The present application provides a pre-written log service device, which includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the pre-written log service method in the above-mentioned embodiment 1.
[0136] Reference below Figure 7 , which shows a schematic diagram of the structure of a pre-written log service device suitable for implementing the embodiment of the present application. The pre-written log service device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The written-ahead log service device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0137] like Figure 7 As shown, the pre-written log service device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the pre-written log service device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the pre-written log service device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a pre-written log service device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided alternatively.
[0138] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.
[0139] The pre-written log service device provided by the present application adopts the pre-written log service method in the above embodiment, which can solve the technical problem of inconsistent submission order and storage order of concurrent log writing in the pre-written log service. Compared with the prior art, the beneficial effects of the pre-written log service device provided by the present application are the same as the beneficial effects of the pre-written log service method provided by the above embodiment, and other technical features in the pre-written log service device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.
[0140] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.
[0141] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0142] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, the computer-readable program instructions being used to execute the write-ahead log service method in the above-mentioned embodiment.
[0143] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.
[0144] The computer-readable storage medium may be included in the pre-written log service device; or may exist independently without being assembled into the pre-written log service device.
[0145] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a write-ahead log service device, the write-ahead log service device: receives a log record request for a target log concurrently submitted by a state machine; generates a logical log sequence number and a physical log sequence number of the target log based on the log record request; obtains a mapping relationship between the logical log sequence number and the physical log sequence number in response to a restart playback request sent by the state machine; and applies the target log playback to the state machine through a logical log sequence sliding window based on the mapping relationship between the logical log sequence number and the physical log sequence number.
[0146] Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0147] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0148] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.
[0149] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned write-ahead log service method, and can solve the technical problem of inconsistent submission order and storage order of concurrent write logs in the write-ahead log service. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the write-ahead log service method provided by the above-mentioned embodiment, and will not be repeated here.
[0150] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned write-ahead log service method when executed by a processor.
[0151] The computer program product provided by the present application can solve the technical problem of inconsistent submission order and storage order of concurrent log writing in the write-ahead log service. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the write-ahead log service method provided by the above embodiment, which will not be repeated here.
[0152] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.
Claims
1. A write-ahead log service method, characterized in that: The method is applied to a log service end, and the method includes: Receive logging requests for target logs submitted concurrently by the state machine; Based on the log record request, generate a logical log sequence number and a physical log sequence number of the target log; In response to the restart playback request sent by the state machine, obtaining a mapping relationship between the logical log sequence number and the physical log sequence number; Based on the mapping relationship between the logical log sequence number and the physical log sequence number, the target log playback is applied to the state machine through a logical log sequence sliding window; Wherein, the step of generating the logical log sequence number and the physical log sequence number of the target log based on the log record request includes: Generate a logical log sequence number of the target log according to the log record request submission order of the target log; Generate a physical log sequence number of the target log according to the physical storage location of the target log in the log server; Wherein, the step of applying the target log playback to the state machine through the logical log sequence sliding window based on the mapping relationship between the logical log sequence number and the physical log sequence number includes: Starting from an initial physical sequence number and an initial logical sequence number, the physical log sequence number is compared with the logical log sequence number; When the order of the physical log sequence number corresponds to the order of the logical log sequence number, replaying and applying the target log corresponding to the physical log sequence number to the state machine; When the order of the physical log sequence number and the order of the logical log sequence number do not correspond, the window sliding side of the logical log sequence sliding window is slid toward the target direction until a physical log sequence number corresponding to the logical log sequence number appears, and the target log corresponding to the physical log sequence number and the physical log sequence number before it is replayed and applied to the state machine.
2. The method according to claim 1, characterized in that The method further comprises: If any of the log record requests concurrently submitted by the state machine fails to be submitted, a rollback log of the target log is generated, and the rollback logical sequence number of the log record request is recorded in the rollback log; Based on the rollback logical sequence number, the logical log sequence sliding window and the target log are adjusted accordingly.
3. A pre-written log service method, characterized in that: The method is applied to a state machine, and the method comprises: Concurrently submitting a log record request for a target log to a log service end, so that the log service end generates a logical log sequence number and a physical log sequence number of the target log based on the log record request; Sending a restart playback request to the log service end so that the log service end obtains the mapping relationship between the logical log sequence number and the physical log sequence number, and based on the mapping relationship between the logical log sequence number and the physical log sequence number, applying the target log playback to the state machine through the logical log sequence sliding window; The step of concurrently submitting a log record request for a target log to a log service end so that the log service end generates a logical log sequence number and a physical log sequence number of the target log based on the log record request includes: Concurrently submitting a log record request for a target log to a log service end, and generating a logical log sequence number of the target log by the log service end according to the order in which the log record request of the target log is submitted; Generate a physical log sequence number of the target log according to the physical storage location of the target log in the log service end through the log service end; The step of sending a restart playback request to the log server so that the log server obtains the mapping relationship between the logical log sequence number and the physical log sequence number, and applying the target log playback to the state machine through the logical log sequence sliding window based on the mapping relationship between the logical log sequence number and the physical log sequence number includes: Sending a restart playback request to the log service end, so that the log service end compares the physical log sequence number with the logical log sequence number starting from the initial physical sequence number and the initial logical sequence number; When the order of the physical log sequence number corresponds to the order of the logical log sequence number, replaying and applying the target log corresponding to the physical log sequence number to the state machine; When the order of the physical log sequence number and the order of the logical log sequence number do not correspond, the window sliding side of the logical log sequence sliding window is slid toward the target direction until a physical log sequence number corresponding to the logical log sequence number appears, and the target log corresponding to the physical log sequence number and the physical log sequence number before it is replayed and applied to the state machine.
4. The method according to claim 3, characterized in that The step of concurrently submitting a log record request for a target log to the log service end includes: Submit a logging request to the target log through the concurrent log writing window; When the current number of the log record requests exceeds the maximum window value of the concurrent log write window, the log record requests outside the concurrent log write window are blocked.
5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Initialize the physical log sequence number and the logical log sequence number, and determine the initial physical sequence number of the physical log sequence number and the initial logical sequence number of the logical log sequence number; Initialize a concurrent log write window and configure a maximum window value of the concurrent log write window.
6. A pre-written log service device, characterized in that: The device comprises: A request receiving module, used for receiving log record requests for target logs submitted concurrently by the state machine; A sequence generation module, used for generating a logical log sequence number and a physical log sequence number of the target log based on the log record request; A mapping acquisition module, configured to acquire a mapping relationship between the logical log sequence number and the physical log sequence number in response to a restart playback request sent by the state machine; A log playback module, configured to apply the target log playback to the state machine through a logical log sequence sliding window based on a mapping relationship between the logical log sequence number and the physical log sequence number; Wherein, the sequence generation module is further used to generate a logical log sequence number of the target log according to the log record request submission order of the target log; Generate a physical log sequence number of the target log according to the physical storage location of the target log in the log server; The log playback module is further used to compare the physical log sequence number with the logical log sequence number starting from the initial physical sequence number and the initial logical sequence number; When the order of the physical log sequence number corresponds to the order of the logical log sequence number, replaying and applying the target log corresponding to the physical log sequence number to the state machine; When the order of the physical log sequence number and the order of the logical log sequence number do not correspond, the window sliding side of the logical log sequence sliding window is slid toward the target direction until a physical log sequence number corresponding to the logical log sequence number appears, and the target log corresponding to the physical log sequence number and the physical log sequence number before it is replayed and applied to the state machine.
7. A pre-written log service device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the write-ahead log service method according to any one of claims 1 to 5.
8. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the write-ahead log service method according to any one of claims 1 to 5 are implemented.
Citation Information
Patent Citations
Optimization method for writing batch group serialization logic
CN113590036A
Synchronization method of logic replication slot and related device
CN118673078A