Log packet replay method, device, electronic device and storage medium
By initializing node memory objects in a shared storage cluster and associating log dependency information, the replay order of log packages is optimized, and the problem of inefficient log replay caused by incomplete archive log files is solved, and efficient log package replay is achieved.
Patent Information
- Application Number
- CN202111587144.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-23
- Publication Date
- 2025-06-06
- Estimated Expiration
- 2041-12-23
AI Technical Summary
In a shared storage cluster, the archived log files in the node may be incomplete, resulting in the process of repeating the logs being unable to complete smoothly. The existing method requires parsing a large number of archived log files, which is not efficient.
By initializing the memory object of each node in the cluster, loading the log package and associating the log dependency information, scanning the memory objects in turn to determine the optimal reenactment log package, and reenacting the optimal reenactment log package after all memory objects are scanned.
This method improves the efficiency of log package replay by optimizing the replay sequence and process of log packages, ensuring that log packages can be replayed in sequence according to log dependency information.
Smart Images

Figure CN114297043B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the technical field of database management, and in particular to a log packet replay method, device, electronic device and storage medium. Background Art
[0002] Multiple instances in a shared storage cluster can access and modify data pages, and the redo logs generated by each instance modifying the data pages are generated and saved to the online log files and archive log files of the current instance. When the shared storage cluster replays the redo logs and other logs, the archive log files associated with the replay logs in the node may be incomplete, such as the archive log files in the node are missing or deleted, which makes the replay log process unable to complete smoothly.
[0003] In order to ensure that the process of replaying logs can be completed smoothly, when replaying logs, it is necessary to check and judge the integrity of the archive log files in the node. At present, the method for checking the integrity of the archive log files in the node is to scan and parse all the archive log files to check whether the archive log files associated with the replay log exist and are complete. However, the above method needs to parse a large number of archive log files, which is not efficient. Summary of the invention
[0004] The embodiments of the present invention provide a log packet replay method, device, electronic device and storage medium to improve the efficiency of log packet replay.
[0005] In a first aspect, an embodiment of the present invention provides a log packet replay method, comprising:
[0006] Initialize a memory object of each node in the cluster, wherein the memory object is loaded with a log package, and the log package is associated with the log dependency information;
[0007] Scanning each of the memory objects in sequence, and in each scanning process, determining the optimal log package to be replayed according to the log dependency information corresponding to the currently scanned memory object;
[0008] After scanning of each of the memory objects is completed, the optimal log package to be replayed is replayed.
[0009] In a second aspect, an embodiment of the present invention further provides a log packet replay device, comprising:
[0010] A memory object initialization module, configured to initialize a memory object of each node in the cluster, wherein the memory object is loaded with a log package, and the log package is associated with log dependency information;
[0011] A determination module is configured to scan each of the memory objects in sequence, and during each scanning process, determine the optimal log package to be replayed according to the log dependency information corresponding to the currently scanned memory object;
[0012] The replay module is configured to replay the optimal log package to be replayed after scanning of each memory object is completed.
[0013] In a third aspect, an embodiment of the present invention further provides an electronic device, including:
[0014] one or more processors;
[0015] A storage device for storing one or more programs;
[0016] The one or more programs are executed by the one or more processors, so that the one or more processors implement the log packet replay method provided in the embodiment of the present invention.
[0017] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the log packet replay method provided by an embodiment of the present invention is implemented.
[0018] The embodiment of the present invention provides a log packet replay method, device, electronic device and storage medium, which initializes the memory object of each node in the cluster, the memory object is loaded with a log packet, and the log packet is associated with log dependency information; on this basis, each memory object is scanned in sequence, and in each scanning process, the optimal log packet to be replayed is determined according to the log dependency information corresponding to the currently scanned memory object; after each memory object is scanned, the optimal log packet to be replayed is replayed. The method initializes the memory object of each node in the cluster, and each log packet can be correspondingly associated with a log dependency information, on this basis, the optimal log packet to be replayed is determined according to the log dependency information, and after each memory object is scanned, the optimal log packet to be replayed is replayed, which can ensure that the log packets are replayed in sequence according to the log dependency information, thereby improving the efficiency of log packet replay. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 A flow chart of a log packet replay method provided in Embodiment 1 of the present invention;
[0020] Figure 2 A schematic diagram of an adjusted RLOG_PKG format provided in the first embodiment of the present invention;
[0021] Figure 3 A schematic diagram of memory object initialization provided in Embodiment 1 of the present invention;
[0022] Figure 4A schematic diagram of the structure of a log packet replay device provided in Embodiment 2 of the present invention;
[0023] Figure 5 A schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION
[0024] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.
[0025] It should be mentioned before discussing exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe various operations (or steps) as sequential processes, many operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of various operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to methods, functions, procedures, subroutines, subprograms, etc. In addition, the embodiments in the present invention and the features in the embodiments can be combined with each other without conflict.
[0026] The term “including” and its variations used in the present invention are open inclusions, that is, “including but not limited to.” The term “based on” means “based at least in part on.” The term “one embodiment” means “at least one embodiment.”
[0027] It should be noted that the concepts such as "first" and "second" mentioned in the present invention are only used to distinguish the corresponding contents, and are not used to limit the order or interdependence.
[0028] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0029] In order to better understand the embodiments of the present invention, relevant terms are introduced below.
[0030] Redo log: also known as REDO log, faithfully records all physical page modifications. When the system fails and restarts, the database can be restored to the state at the time of failure by redoing the REDO log.
[0031] Physical Transaction (PTX): A collection of operations that modify physical data pages within the database.
[0032] LSN: The full name of LSN is Log Sequence Number, which is the log sequence number that uniquely identifies each operation or transaction that modifies a data page. LSN is a Bigint value that is automatically maintained by the database system. It has the characteristics of automatic increment and global uniqueness. Each LSN value represents a physical transaction generated within the database system. For example, if the LSN value corresponding to the data page modified this time is 1000, then the LSN value corresponding to the record of the next data page modification is 1001.
[0033] Log package (which can be expressed as RLOG_PKG): refers to the data unit where the database saves logs. Among them, REDO log package refers to the data unit where the database saves REDO logs. A log package can save REDO logs generated by one or more PTXs. Each log record in the REDO log corresponds to an LSN.
[0034] Log package sequence number (which can be expressed as PKG_SEQNO): Each RLOG_PKG has a corresponding sequence number attribute, called the log package sequence number. When the log package is generated, it is continuously increased according to the log package sequence number.
[0035] Multiple instances of a shared storage cluster can access and modify data pages, and the REDO logs generated by each instance modifying the data page are generated and saved to the online log file and archive log file of the current instance. There is a FRESH_EP field in the data page header. When an instance modifies a data page, it will record the instance number in the FRESH_EP field of the data page. Whether the data page is loaded from disk or has been flushed, the FRESH_EP information can be read from the data page header to clarify the previous modification node number of the data page.
[0036] When a data page is modified by multiple instances, when the fault restart or standby database replays the REDO logs of each instance, the REDO logs modified by each instance on the data page must be replayed in sequence in strict accordance with the generation order of the REDO logs in order to correctly restore the data page content. To this end, when modifying a data page, the shared storage cluster generates a data page synchronization log (i.e., PAGE_SYNC log) to determine the instance and LSN value of the previous modified data page to ensure the replay order of the REDO logs.
[0037] The PAGE_SYNC log records the FRESH_EP and FRESH_LSN information, where FRESH_EP can represent the node number of the last modified data page, and FRESH_LSN can represent the LSN value corresponding to the last modified data page. When replaying the REDO log, the log records are parsed in sequence, and the PAGE_SYNC log will be replayed first, and then the REDO log of the modified data page will be replayed. When replaying the PAGE_SYNC log, the PAGE_LSN of the data page (that is, the LSN corresponding to the data page, which can be understood as the maximum LSN value of the data page that has been replayed) will be read and compared with the FRESH_LSN recorded in the PAGE_SYNC log. If the PAGE_LSN of the data page is greater than or equal to the FRESH_LSN, it means that the REDO log corresponding to the FRESH_LSN recorded in the current PAGE_SYNC log has been replayed, and the REDO logs of subsequent modified data pages can continue to be replayed. If the PAGE_LSN of the data page is less than the FRESH_LSN, it means that the REDO log corresponding to the FRESH_LSN recorded in the current PAGE_SYNC log has not been replayed yet. At this time, you need to wait for the corresponding node to replay the REDO log corresponding to the current FRESH_LSN first, and then continue to replay the REDO log of the subsequent modified data page.
[0038] When the standby database of the shared storage cluster replays the master database REDO log, the archive on the FRESH_EP node recorded in the master database PAGE_SYNC log may be missing (for example, the archive log file is deleted), or the shared storage cluster master database does not send other logs that the PAGE_SYNC log depends on to the standby database, resulting in the standby database replaying the PAGE_SYNC log and waiting timeout due to the lack of dependent logs, and the replay cannot be completed. Similarly, the same situation may occur when the shared storage cluster is restored.
[0039] To address the above issues, when synchronizing logs to an asynchronous standby database or restoring a shared storage cluster, the shared storage cluster needs to check the integrity of archived logs in all nodes in advance. An optional integrity check solution is to scan all archived log files, parse the REDO logs, and check in turn whether the REDO logs that the PAGE_SYNC log depends on exist. However, this solution requires parsing a large number of REDO logs, which is inefficient.
[0040] The embodiment of the present invention provides an efficient log integrity verification method, which sets log dependency information in a REDO log package (ie, RLOG_PKG), and determines whether the global log is complete based on the log dependency information, ensuring that all logs to be replayed are sent for replay in sequence according to the specified order.
[0041] It should be noted that the PAGE_SYNC log is a type of REDO log, which is generated when PTX is submitted (ie, PTX COMMIT) and is inserted before the REDO log of the modified data page.
[0042] The format of the PAGE_SYNC log record can be expressed as:
[0043] [TS_ID, FILE_ID, PAGE_NO, FRESH_EP, FRESH_LSN],
[0044] Among them, FRESH_EP represents the node number of the last modified data page, FRESH_LSN represents the LSN value corresponding to the last modified data page, and the other attributes are irrelevant to the present invention and are not introduced here.
[0045] Since the PAGE_SYNC log is generated when PTX COMMIT is executed, the log content will be first written into the REDO log package RLOG_PKG when PTX COMMIT is executed. Therefore, the embodiment of the present invention can first transfer the log dependency information in the PAGE_SYNC log to RLOG_PKG, and then write RLOG_PKG into the log file.
[0046] Embodiment 1
[0047] Figure 1 A flow chart of a log packet replay method provided in Embodiment 1 of the present invention is provided. The method is applicable to the case where the log packets that have not been replayed in each node are replayed. The method can be executed by a log packet replay device, wherein the device can be implemented by software and / or hardware and is generally integrated on an electronic device. In this embodiment, the electronic device includes but is not limited to: desktop computers, laptop computers, servers and other devices.
[0048] like Figure 1 As shown, a log packet replay method provided by Embodiment 1 of the present invention comprises the following steps:
[0049] S110: Initialize a memory object of each node in the cluster, wherein a log package is loaded in the memory object, and the log package is associated with log dependency information.
[0050] In this embodiment, the cluster may refer to a shared storage cluster; a shared storage cluster may be understood as a single database, multi-instance cluster system, whose database is deployed on shared storage for access by all nodes. A memory object may refer to a data packet used to load a corresponding log packet in each node. A log packet may refer to a data unit in which a database stores logs. Log dependency information may be understood as other log information that a certain log depends on.
[0051] In this embodiment, before initializing the memory object of each node in the cluster, the archived log file from which the log package can be loaded into the memory object in each node can be determined through the unreplayed log package.
[0052] Optionally, before initializing the memory object of each node, it also includes: for each node, recording the first maximum log packet sequence number and the first maximum LSN that the node has repeated; scanning the archived log files on the node, and the header information of each archived log file includes the second maximum log packet sequence number and the second maximum LSN of the log packet in the archived log file; if the second maximum log packet sequence number is greater than the first maximum log packet sequence number, and the second maximum LSN is greater than the first maximum LSN, then adding the archived log file to the archived log file linked list of the node.
[0053] Wherein, the archive log file linked list may refer to a data structure for recording archive log files. In the present embodiment, the archive log file linked list may be used to record archive log files that meet specific conditions, wherein the specific condition may be that the second largest log package sequence number is greater than the first largest log package sequence number, and the second largest LSN is greater than the first largest LSN. In the database, the redo log file needs to be archived in time, i.e., the backup is retained, and the archive file formed in this process may be referred to as an archive log file. A node may include multiple archive log files; an archive log file may include multiple log packages, and the log package sequence number increases successively; a log package may include multiple logs (i.e., log records), and a log record may correspond to an LSN, and the LSN value increases successively.
[0054] In a shared storage cluster, each node can independently maintain an APPLY_SEQ and an APPLY_LSN, that is, [APPLY_SEQ, APPLY_LSN], where APPLY_SEQ can represent the maximum log packet sequence number PKG_SEQNO (i.e., the first maximum log packet sequence number) that the node has currently sent for replay, and APPLY_LSN can represent the maximum LSN value (i.e., the first maximum LSN) that the node has currently sent for replay. That is, before initializing the memory object of each node, first for each node, the first maximum log packet sequence number and the first maximum LSN that the node has replayed can be recorded.
[0055] Then, the archive log files on the node are scanned in sequence. The header information of each archive log file may include information [NEXT_SEQ, CLSN], where NEXT_SEQ may represent the maximum log packet sequence number (i.e., the second largest log packet sequence number) of the log packets in the archive log file; CLSN may represent the maximum LSN value (i.e., the second largest LSN) of the log packets in the archive log file.
[0056] Finally, by comparing [NEXT_SEQ, CLSN] and [APPLY_SEQ, APPLY_LSN], the archived log files that may have unreplayed log packages on the node can be added to an archived log file linked list. Exemplarily, for a certain archived log file, if NEXT_SEQ is greater than APPLY_SEQ (i.e., the second largest log package sequence number is greater than the first largest log package sequence number), and CLSN is greater than APPLY_LSN (i.e., the second largest LSN is greater than the first largest LSN), then the archived log file can be added to the archived log file linked list of the current corresponding node. For example, assuming that the maximum log packet sequence number (i.e., the first maximum log packet sequence number) that a node has currently sent for replay is 10, and the maximum LSN value (i.e., the first maximum LSN) is 1001, it can indicate that before this, in the log packets with log packet sequence numbers less than or equal to 10 on the node, log records with LSN values less than or equal to 1001 have been replayed; in this case, if the second maximum log packet sequence number of a log packet of an archived log file is 11 (11>10), and the second maximum LSN value is 1010 (1010>1001), it can be explained that there are log records in the log packet of the archived log file that have not been replayed. At this time, the archived log file can be added to the archived log file linked list of the node to facilitate the subsequent loading of the unreplayed log packets from the archived log file linked list into the memory object.
[0057] It should be noted that, in the process of adding the archived log file to the corresponding archived log file linked list, the continuity of the archived log file can also be verified; if the verification is successful, the subsequent operations can be continued; if the verification fails and the archived log file is not continuous, an error can be directly reported and returned without performing any subsequent operations. The specific method of verifying the archived log file is not limited here.
[0058] Optionally, it also includes: determining the log dependency information of each log package according to the node number of the modified data page recorded in the log package and the maximum LSN value corresponding to the corresponding node.
[0059] The data page synchronization log may refer to a PAGE_SYNC log. The PAGE_SYNC log records FRESH_EP and FRESH_LSN information, where FRESH_EP may indicate the node number of the last modified data page, and FRESH_LSN may indicate the LSN value corresponding to the last modified data page.
[0060] Exemplarily, when PTX COMMIT generates a PAGE_SYNC log, the FRESH_LSN in the PAGE_SYNC log can be registered in the DPS_LSN_ARR array of the current PTX, and the array subscript can be FRESH_EP in the PAGE_SYNC log. When registering, the larger one of the current FRESH_LSN and the registered LSN value in the array item with the subscript FRESH_EP can be registered.
[0061] PTX COMMIT can trigger a log write action, at which time the PAGE_SYNC log generated by the PTX will be written to the log package (i.e., RLOG_PKG) cache. When writing logs, the content of the DPS_LSN_ARR array of the current PTX can be registered in the DPS_LSN_ARR array of the RLOG_PKG cache. Like the DPS_LSN_ARR array on the PTX, the subscript of the DPS_LSN_ARR array in the RLOG_PKG cache is also the node number. When registering, the larger LSN value of the corresponding node of the DPS_LSN_ARR array in the PTX and RLOG_PKG log package cache can be registered.
[0062] Figure 2 This is a schematic diagram of an adjusted RLOG_PKG format provided in the first embodiment of the present invention. Figure 2 As shown, the RLOG_PKG format can be adjusted to increase log dependency information. The adjustment process can be: RLOG_PKG can be composed of a log header and a log body, and a field RLOG_PKG_N_DEPENDS (where RLOG_PKG_N_DEPENDS can also be expressed as N_DPS) is added to the log header to record the number of shared storage cluster nodes that the RLOG_PKG depends on; the tail of the log body can be used to record the log dependency information of the RLOG_PKG. Figure 2 In the log dependency information, the log dependency information may include multiple [DPS_EP, DPS_LSN] data groups, where DPS_EP may represent the node number corresponding to the node that modified the data page in the previous order; DPS_LSN may represent the maximum LSN value corresponding to each node modifying the data page. That is, according to the node number of the modified data page recorded in RLOG_PKG and the maximum LSN value corresponding to the corresponding node, the log dependency information of each log package can be determined.
[0063] When PTX COMMIT or checkpoint is generated, RLOG_PKG flushing is triggered (that is, the DPS_LSN_ARR array content in the RLOG_PKG cache is written to the disk). When flushing, you can follow Figure 2The RLOG_PKG format in the log packet is used to write the number of shared storage cluster nodes that the RLOG_PKG depends on into the RLOG_PKG_N_DEPENDS field in the log packet header, and write the DPS_LSN_ARR array content in the RLOG_PKG cache into the log dependency information at the end of the log packet. The length of the DPS_LSN_ARR array of the RLOG_PKG actually written can be increased by adjusting the effective log length DATA_OFF of the log packet.
[0064] In this embodiment, after determining the log dependency information corresponding to the memory object and the log package loaded by the memory object, the memory object of each node in the cluster can be initialized, and the corresponding log package is loaded in the memory object.
[0065] Optionally, initializing the memory object of each node in the cluster includes: for each node, scanning the log packets in the archived log files in the archived log file linked list of the node in turn; loading the first log packet that meets the following conditions into the memory object of the node: the log packet sequence number of the log packet is greater than the first maximum log packet sequence number that has been replayed, and the maximum LSN value in the log packet is greater than the first maximum LSN that has been replayed.
[0066] Among them, the memory object of each node is initialized (the memory object can be expressed as rarch_ep) to manage the log synchronization / recovery information of the node. First, for each node, the archived log files in the archived log file linked list of the node can be scanned in sequence, and for each archived log file, the log packages therein are scanned in sequence. Then, the first log package that meets the conditions on the node is loaded into the rarch_ep of the node; wherein the above conditions that need to be met can be: the log package sequence number of the log package (i.e. PKG_SEQNO) is greater than the APPLY_SEQ of the node (i.e. the first maximum log package sequence number that has been repeated), and the maximum LSN value in the log package is greater than the APPLY_LSN of the node (i.e. the first maximum LSN that has been repeated). For example, suppose that the first maximum log packet sequence number that has been replayed by a node is 10, and the first maximum LSN that has been replayed is 1001. In this case, log packets with log packet sequence numbers greater than 10 and maximum LSN values greater than 1001 are all log packets that have not been replayed. Since multiple log packets that have not been replayed can be replayed in ascending order of log packet sequence numbers and LSNs, and one log packet is replayed each time, only the first unreplayed log packet that meets the conditions can be selected and loaded into the memory object.
[0067] It should be noted that, for each node, the log packages in the archived log files in the archived log file linked list of the node are scanned in sequence, and the first log package that meets the following conditions is loaded into the memory object of the node; wherein, the sequential scanning can be understood as, when the first log package that meets the following conditions is loaded into the memory object of the node and the scanning will continue after the replay, when scanning again, the second log package that originally meets the following conditions can be considered to be the first log package that meets the following conditions, and so on, until all the logs to be replayed of all nodes are completely replayed.
[0068] It should be noted that in the process of loading the log package into the rarch_ep of the node, the log dependency information at the end of the log package can be parsed to parse the corresponding N_DPS group [DPS_EP, DPS_LSN] data group. To distinguish, the log package loaded into the rarch_ep can be named rpkg. After loading the corresponding log package, [APPLY_SEQ, APPLY_LSN] can also be stored in the rarch_ep of the node. When the rarch_ep of all nodes is initialized, zero or one initial log package rpkg has been loaded in the rarch_ep of each node.
[0069] Figure 3 A schematic diagram of memory object initialization provided in the first embodiment of the present invention. Figure 3 As shown, taking node 1 and node 2 as examples, the archive log file linked list of each node may include n archive log files. For each node, by comparing the [APPLY_SEQ, APPLY_LSN] corresponding to the node with the log package sequence number of the log package in the archive log file and the largest LSN value in the log package, the log package rpkg that can be loaded into the memory object can be determined from the n archive log files in the archive log file linked list of each node. The log package rpkg loaded into the memory object can also carry the [DPS_EP, DPS_LSN] data group.
[0070] S120, scanning each of the memory objects in sequence, and in each scanning process, determining the optimal log package to be replayed according to the log dependency information corresponding to the currently scanned memory object.
[0071] S130: After scanning of each memory object is completed, replay the optimal log package to be replayed.
[0072] In this embodiment, first, the memory objects (i.e., rarch_ep) in each node in the shared storage cluster can be integrated into an array, and the array can be represented by rarch_ep_arr; wherein the array subscript can represent each node number (i.e., the node number corresponding to each memory object); the array rarch_ep_arr can be recorded in the control node of the shared storage cluster. Then, the array items in the array rarch_ep_arr can be scanned in sequence (the array item can refer to each rarch_ep in the array), wherein zero or one log package rpkg is loaded in each initialized rarch_ep. Finally, in each scanning process, it is determined whether rpkg exists on each rarch_ep; if rpkg does not exist, the next rarch_ep can be scanned; if rpkg exists, the optimal log package to be replayed can be determined according to the log dependency information corresponding to the currently scanned rarch_ep, and after each rarch_ep is scanned, the optimal log package to be replayed is replayed.
[0073] Optionally, determining the optimal log package to be replayed based on the log dependency information corresponding to the currently scanned memory object includes: sequentially scanning each data group in the log dependency information corresponding to the currently scanned memory object, and for the currently scanned data group, finding a target memory object from each memory object that has the same node number as the node number in the currently scanned data group, and the first maximum LSN that has been replayed corresponding to the target memory object; if the first maximum LSN that has been replayed corresponding to the target memory object is greater than or equal to the LSN value in the currently scanned data group, then the log corresponding to the currently scanned data group has been replayed; otherwise, the log corresponding to the currently scanned data group has been replayed. Not replayed; if there are one or more logs corresponding to the data groups in the log dependency information corresponding to the currently scanned memory object that have not been replayed, continue to scan the next memory object; if the logs corresponding to each data group in the log dependency information corresponding to the currently scanned memory object have been replayed, update the target node number, target minimum LSN and target maximum LSN according to the currently scanned memory object, where the target node number, target minimum LSN and target maximum LSN are respectively used to record the node number of the optimal log package to be replayed, the minimum LSN value in the optimal log package to be replayed and the maximum LSN value in the optimal log package to be replayed.
[0074] Among them, the log dependency information corresponding to the currently scanned rarch_ep includes multiple [DPS_EP, DPS_LSN] data groups. First, each data group in the log dependency information can be scanned in turn; for the currently scanned data group [DPS_EP, DPS_LSN], find the rarch_ep whose node number is consistent with DPS_EP (that is, the node number in the currently scanned data group) in each rarch_ep of the array rarch_ep_arr as the target rarch_ep (the target rarch_ep can be recorded as rarch_ep').
[0075] Then, obtain the APPLY_LSN (i.e., the first largest LSN that has been replayed) of the corresponding record on rarch_ep' to determine whether APPLY_LSN is greater than or equal to DPS_LSN (i.e., the LSN value in the currently scanned data group). If so, it can be said that the log corresponding to the currently scanned data group [DPS_EP, DPS_LSN] has been sent for replay; if not, it can be said that the log corresponding to the currently scanned data group [DPS_EP, DPS_LSN] has not been sent for replay. Finally, on this basis, if there are logs corresponding to one or more data groups in the log dependency information corresponding to the currently scanned rarch_ep that have not been replayed, the next rarch_ep can be scanned; if the logs corresponding to each data group in the log dependency information corresponding to the currently scanned rarch_ep have been replayed, the target node number, target minimum LSN and target maximum LSN can be updated according to the currently scanned rarch_ep, where the target node number, target minimum LSN and target maximum LSN can be used to record the node number of the optimal log package to be replayed, the minimum LSN value in the optimal log package to be replayed and the maximum LSN value in the optimal log package to be replayed, respectively.
[0076] In this embodiment, variables EP, min_LSN and max_LSN can be set to represent the target node number, the target minimum LSN and the target maximum LSN respectively. The initial values of the above variables EP, min_LSN and max_LSN can all be set to -1. EP, min_LSN and max_LSN can be updated according to the node number of the currently scanned rarch_ep and the minimum LSN value and the maximum LSN value in the log package loaded by rarch_ep.
[0077] Optionally, the target node number, target minimum LSN and target maximum LSN are updated according to the memory object currently being scanned, including: if the target node number is an initial value, the target node number is set to the node number of the memory object currently being scanned, the target minimum LSN is set to the minimum LSN in the log package loaded by the memory object currently being scanned, and the target maximum LSN is set to the maximum LSN in the log package loaded by the memory object currently being scanned; if the target node number is not an initial value, the target minimum LSN is compared with the minimum LSN in the log package loaded by the memory object currently being scanned: if the target minimum LSN is less than the minimum LSN in the log package loaded by the memory object currently being scanned, the target node number, target minimum LSN and target maximum LSN remain unchanged; if the target minimum LSN is greater than the minimum LSN in the log package loaded by the memory object currently being scanned, the target node number is set to the current node number of the memory object currently being scanned, and the target minimum LSN is set to the maximum LSN in the log package loaded by the memory object currently being scanned. The LSN is set to the minimum LSN in the log package loaded by the memory object currently scanned, and the target maximum LSN is set to the maximum LSN in the log package loaded by the memory object currently scanned; if the target minimum LSN is equal to the minimum LSN in the log package loaded by the memory object currently scanned, then compare the target maximum LSN with the maximum LSN in the log package loaded by the memory object currently scanned; if the target maximum LSN is less than or equal to the maximum LSN in the log package loaded by the memory object currently scanned, the target node number, target minimum LSN, and target maximum LSN remain unchanged; if the target maximum LSN is greater than the maximum LSN in the log package loaded by the memory object currently scanned, then set the target node number to the current node number of the memory object currently scanned, set the target minimum LSN to the minimum LSN in the log package loaded by the memory object currently scanned, and set the target maximum LSN to the maximum LSN in the log package loaded by the memory object currently scanned.
[0078] The specific process of updating EP, min_LSN and max_LSN according to the currently scanned rarch_ep is as follows:
[0079] Determine whether the variable EP is the initial value -1. If the variable EP is the initial value -1, EP can be set to the node number of the currently scanned rarch_ep, min_LSN can be set to the minimum LSN value in the log package loaded by the currently scanned rarch_ep, and max_LSN can be set to the maximum LSN in the log package loaded by the currently scanned rarch_ep.
[0080] If the variable EP is not the initial value -1, you can compare min_LSN with the minimum LSN in the log package loaded by the currently scanned rarch_ep. The comparison results can be divided into the following categories:
[0081] If min_LSN is less than the minimum LSN value in the log package loaded by the currently scanned rarch_ep, EP, min_LSN and max_LSN can remain unchanged;
[0082] If min_LSN is greater than the minimum LSN value in the log package loaded by the currently scanned rarch_ep, EP can be set to the current node number of the currently scanned rarch_ep, min_LSN can be set to the minimum LSN in the log package loaded by the currently scanned rarch_ep, and max_LSN can be set to the maximum LSN in the log package loaded by the currently scanned rarch_ep;
[0083] If min_LSN is equal to the minimum LSN of the log package loaded in the currently scanned rarch_ep, then max_LSN can be compared with the maximum LSN of the log package loaded in the currently scanned rarch_ep;
[0084] If max_LSN is less than or equal to the maximum LSN in the log package loaded by the currently scanned rarch_ep, EP, min_LSN, and max_LSN remain unchanged;
[0085] If max_LSN is greater than the maximum LSN in the log package loaded by the currently scanned rarch_ep, EP can be set to the current node number of the currently scanned rarch_ep, min_LSN can be set to the minimum LSN in the log package loaded by the currently scanned rarch_ep, and max_LSN can be set to the maximum LSN in the log package loaded by the currently scanned rarch_ep.
[0086] Optionally, after each memory object is scanned, the optimal log package to be replayed is replayed, including: after each memory object is scanned, if the target node number is not the initial value, the log package loaded in the memory object corresponding to the target node number is replayed; the first maximum log package sequence number that has been replayed and the first maximum LSN that has been replayed registered in the memory object corresponding to the target node number are updated according to the replayed log package, and the log package loaded in the memory object corresponding to the target node number is cleared; if the target node number is the initial value, it is determined whether no log package is loaded in each memory object; if so, the logs of each node have been replayed; otherwise, other logs that the log packages of one or more nodes depend on are lost, and the replay of the log package is stopped, and an error message is returned.
[0087] Among them, if EP is not the initial value -1, the log package rpkg loaded in the rarch_ep corresponding to EP can be replayed; according to the currently replayed rpkg, the first maximum log package sequence number (i.e. APPLY_SEQ) registered in the currently scanned rarch_ep that has been replayed is updated to the log package sequence number of the currently replayed rpkg, and the first maximum LSN (i.e. APPLY_LSN) registered in the currently scanned rarch_ep that has been replayed is updated to the maximum LSN value in the currently replayed rpkg (i.e. the currently corresponding max_LSN). On this basis, the log package rpkg loaded in the currently scanned rarch_ep can be cleared. If EP is the initial value -1, it can indicate that the array rarch_ep_arr scan is finished.
[0088] It should be noted that after clearing the loaded log packages in the currently scanned rarch_ep, the next log package that meets the conditions (the conditions are that the log package sequence number of the log package is greater than the first maximum log package sequence number that has been replayed, and the maximum LSN value in the log package is greater than the first maximum LSN that has been replayed) can be loaded into the memory object corresponding to the target node number, and then the operation of "scanning each memory object in turn, and in each scanning process, determining the optimal log package to be replayed according to the log dependency information corresponding to the currently scanned memory object, and after each memory object is scanned, replaying the optimal log package to be replayed" is executed until all the logs to be replayed of all nodes are completed.
[0089] If EP is the initial value -1, it can be determined whether the log package is not loaded in all memory objects. If so, it can be said that the logs to be replayed of all nodes have been replayed; if not, it can be said that other logs that the log package of one or more nodes depends on are lost (for example, the archived log file where the log is located may be manually deleted). At this time, the log package can be stopped from being replayed, and the corresponding error message is returned to the user. Among them, assuming that according to the log dependency information associated with log package A, it is found that log package A depends on another log B, then the loss of other logs that log package A depends on can be understood as the loss of log B that log package A depends on.
[0090] It should be noted that the specific method for determining whether all memory objects have not been loaded with log packages is not limited here. For example, a flag FLAG can be set and initialized to 0 for determination. Specifically, during the process of scanning all memory objects, if there is a loaded log package in the memory object, FLAG can be set to 1, otherwise FLAG remains at the initial value 0. After all memory objects have been scanned, determine whether FLAG is 0 at this time. If it is 0, it means that no log package has been loaded in all memory objects.
[0091] A log packet replay method provided in a first embodiment of the present invention initializes the memory object of each node in the cluster, wherein the memory object is loaded with a log packet, and the log packet is associated with log dependency information; on this basis, each memory object is scanned in sequence, and during each scan, the optimal log packet to be replayed is determined according to the log dependency information corresponding to the currently scanned memory object; after each memory object is scanned, the optimal log packet to be replayed is replayed. The method initializes the memory object of each node in the cluster, and each log packet can be associated with a log dependency information, on this basis, the optimal log packet to be replayed is determined according to the log dependency information, and after each memory object is scanned, the optimal log packet to be replayed is replayed, which can ensure that the log packets are replayed in sequence according to the log dependency information, thereby improving the efficiency of log packet replay.
[0092] Embodiment 2
[0093] Figure 4 This is a schematic diagram of the structure of a log packet replay device provided in Embodiment 2 of the present invention, which can be implemented by software and / or hardware. Figure 4 As shown, the device includes: a memory object initialization module 310, a determination module 320 and a replay module 330;
[0094] The memory object initialization module 310 is configured to initialize the memory object of each node in the cluster, wherein the memory object is loaded with a log package, and the log package is associated with the log dependency information;
[0095] The determination module 320 is configured to scan each of the memory objects in sequence, and in each scanning process, determine the optimal log package to be replayed according to the log dependency information corresponding to the currently scanned memory object;
[0096] The replay module 330 is configured to replay the optimal log package to be replayed after scanning of each memory object is completed.
[0097] In this embodiment, the device initializes the memory object of each node in the cluster through the memory object initialization module 310, and the memory object is loaded with a log package, and the log package is associated with the log dependency information; through the determination module 320, each of the memory objects is scanned in turn, and in each scanning process, the optimal log package to be replayed is determined according to the log dependency information corresponding to the currently scanned memory object; through the replay module 330, after each memory object is scanned, the optimal log package to be replayed is replayed. The device initializes the memory object of each node in the cluster, and each log package can be associated with a log dependency information. On this basis, the optimal log package to be replayed is determined according to the log dependency information, and after each memory object is scanned, the optimal log package to be replayed is replayed, which can ensure that the log package is replayed in turn according to the log dependency information, thereby improving the efficiency of log package replay.
[0098] Optionally, the memory object initialization module 310 specifically includes:
[0099] A log packet scanning unit is configured to sequentially scan log packets in archived log files in an archived log file linked list of each node.
[0100] The log packet loading unit is configured to load the first log packet that meets the following conditions into the memory object of the node: the log packet sequence number of the log packet is greater than the first maximum log packet sequence number that has been replayed, and the maximum LSN value in the log packet is greater than the first maximum LSN that has been replayed.
[0101] Optionally, the device further comprises:
[0102] The recording module is configured to record, for each node, the first maximum log packet sequence number and the first maximum LSN that have been replayed by the node before initializing the memory object of each node;
[0103] An archive log file scanning module is configured to scan the archive log files on the node, wherein the header information of each archive log file includes the second largest log packet sequence number and the second largest LSN of the log packets in the archive log file;
[0104] The adding module is configured to add the archive log file to the archive log file linked list of the node if the second maximum log package sequence number is greater than the first maximum log package sequence number and the second maximum LSN is greater than the first maximum LSN.
[0105] Optionally, the device further comprises:
[0106] The information determination module is configured to determine the log dependency information of each log package according to the node number of the modified data page recorded in the log package and the maximum LSN value corresponding to the corresponding node.
[0107] Optionally, the determination module 320 specifically includes:
[0108] A data group scanning unit is configured to sequentially scan each data group in the log dependency information corresponding to the currently scanned memory object, and for the currently scanned data group, find a target memory object having the same node number as that in the currently scanned data group from each of the memory objects, and a first maximum LSN that has been repeated corresponding to the target memory object;
[0109] The replay determination unit is configured to: if the first maximum LSN corresponding to the target memory object that has been replayed is greater than or equal to the LSN value in the data group currently being scanned, then the log corresponding to the data group currently being scanned has been replayed; otherwise, the log corresponding to the data group currently being scanned has not been replayed;
[0110] The scanning unit is configured to continue scanning the next memory object if there are one or more logs corresponding to the data groups that have not been repeated in the log dependency information corresponding to the currently scanned memory object;
[0111] The updating unit is configured to update the target node number, the target minimum LSN and the target maximum LSN according to the currently scanned memory object if the logs corresponding to each data group in the log dependency information corresponding to the currently scanned memory object have been replayed, wherein the target node number, the target minimum LSN and the target maximum LSN are respectively used to record the node number of the optimal log package to be replayed, the minimum LSN value in the optimal log package to be replayed and the maximum LSN value in the optimal log package to be replayed.
[0112] Optional, update unit, specific settings are:
[0113] If the target node number is an initial value, the target node number is set to the node number of the currently scanned memory object, the target minimum LSN is set to the minimum LSN in the log package loaded by the currently scanned memory object, and the target maximum LSN is set to the maximum LSN in the log package loaded by the currently scanned memory object;
[0114] If the target node number is not the initial value, compare the target minimum LSN with the minimum LSN in the log package loaded by the currently scanned memory object:
[0115] If the target minimum LSN is smaller than the minimum LSN in the log package loaded by the currently scanned memory object, the target node number, target minimum LSN, and target maximum LSN remain unchanged;
[0116] If the target minimum LSN is greater than the minimum LSN in the log package loaded by the currently scanned memory object, the target node number is set to the current node number where the currently scanned memory object is located, the target minimum LSN is set to the minimum LSN in the log package loaded by the currently scanned memory object, and the target maximum LSN is set to the maximum LSN in the log package loaded by the currently scanned memory object;
[0117] If the target minimum LSN is equal to the minimum LSN in the log package loaded by the currently scanned memory object, then compare the target maximum LSN with the maximum LSN in the log package loaded by the currently scanned memory object;
[0118] If the target maximum LSN is less than or equal to the maximum LSN in the log package loaded by the currently scanned memory object, the target node number, target minimum LSN, and target maximum LSN remain unchanged;
[0119] If the target maximum LSN is greater than the maximum LSN in the log package loaded by the currently scanned memory object, the target node number is set to the current node number where the currently scanned memory object is located, the target minimum LSN is set to the minimum LSN in the log package loaded by the currently scanned memory object, and the target maximum LSN is set to the maximum LSN in the log package loaded by the currently scanned memory object.
[0120] Optionally, the replay module 330 is specifically configured as follows:
[0121] After scanning of each memory object is completed, if the target node number is not the initial value, replaying the log package loaded in the memory object corresponding to the target node number;
[0122] According to the replayed log package, the replayed first maximum log package sequence number and the replayed first maximum LSN registered in the memory object corresponding to the target node number are updated, and the loaded log package in the memory object corresponding to the target node number is cleared;
[0123] If the target node number is an initial value, determining whether no log package is loaded in each of the memory objects;
[0124] If so, the logs of each of the nodes have been replayed; otherwise, other logs that the log packages of one or more nodes depend on are lost, and the log package replay is stopped and an error message is returned.
[0125] The above-mentioned log packet replay device can execute the log packet replay method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0126] Embodiment 3
[0127] Figure 5This is a schematic diagram of the structure of an electronic device provided by Embodiment 3 of the present invention. Figure 5 As shown, the electronic device provided by the fourth embodiment of the present invention includes: a storage device 42 and one or more processors 41; the processor 41 in the electronic device may be one or more, Figure 5 A processor 41 is taken as an example; the storage device 42 is used to store one or more programs; the one or more programs are executed by the one or more processors 41, so that the one or more processors 41 implement the log packet replay method as described in any one of the embodiments of the present invention.
[0128] The electronic device may further include: a communication device 43 , an input device 44 and an output device 45 .
[0129] The processor 41, storage device 42, communication device 43, input device 44 and output device 45 in the electronic device may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.
[0130] The storage device 42 in the electronic device is a computer-readable storage medium, which can be used to store one or more programs, and the program can be a software program, a computer executable program, and a module, such as the program instructions / modules corresponding to the log packet replay method provided in the first embodiment of the present invention (for example, the attached Figure 4 The modules in the log packet replay device shown include: a memory object initialization module 310 and a replay module 330). The processor 41 executes various functional applications and data processing of the electronic device by running the software programs, instructions and modules stored in the storage device 42, that is, the log packet replay method in the above method embodiment is implemented.
[0131] The storage device 42 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the storage device 42 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the storage device 42 may further include a memory remotely arranged relative to the processor 41, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0132] The communication device 43 may include a receiver and a transmitter. The communication device 43 is configured to perform information transmission and reception communication according to the control of the processor 41.
[0133] The input device 44 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. The output device 45 may include a display device such as a display screen.
[0134] Furthermore, when one or more programs included in the above-mentioned electronic device are executed by the one or more processors 41, the program performs the following operations: initialize the memory object of each node in the cluster, the memory object is loaded with a log package, and the log package is associated with the log dependency information; scan each of the memory objects in turn, and in each scanning process, determine the optimal log package to be replayed according to the log dependency information corresponding to the memory object currently scanned; after each of the memory objects is scanned, replay the optimal log package to be replayed.
[0135] Embodiment 4
[0136] Embodiment 4 of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is used to execute a log packet replay method, the method comprising: initializing a memory object of each node in a cluster, wherein the memory object is loaded with a log packet, wherein the log packet is associated with log dependency information; scanning each of the memory objects in turn, and during each scanning process, determining an optimal log packet to be replayed based on the log dependency information corresponding to the currently scanned memory object; after each of the memory objects is scanned, replaying the optimal log packet to be replayed.
[0137] Optionally, when the program is executed by a processor, it can also be used to execute the log packet replay method provided by any embodiment of the present invention.
[0138] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device.
[0139] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to: electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0140] The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, or radio frequency (RF), etc., or any suitable combination of the above.
[0141] Computer program code for performing the operation of the present invention may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0142] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. A log packet replay method, It is characterized in that include: Initialize a memory object of each node in the cluster, wherein the memory object is loaded with a log package, and the log package is associated with the log dependency information; Scanning each of the memory objects in sequence, and in each scanning process, determining the optimal log package to be replayed according to the log dependency information corresponding to the currently scanned memory object; After scanning of each of the memory objects is completed, replaying the optimal log package to be replayed; The method further comprises: Determine the log dependency information of each log package based on the node number of the modified data page recorded in the log package and the maximum LSN value corresponding to the corresponding node; Determine the optimal log package to be replayed based on the log dependency information corresponding to the currently scanned memory object, including: Scan each data group in the log dependency information corresponding to the currently scanned memory object in sequence, and for the currently scanned data group, find a target memory object with the same node number as that in the currently scanned data group from each of the memory objects, and the first maximum LSN that has been repeated corresponding to the target memory object; If the first maximum LSN corresponding to the target memory object that has been replayed is greater than or equal to the LSN value in the currently scanned data group, then the log corresponding to the currently scanned data group has been replayed, otherwise, the log corresponding to the currently scanned data group has not been replayed; If the log dependency information corresponding to the currently scanned memory object contains one or more logs corresponding to the data groups that have not been repeated, continue to scan the next memory object; If the logs corresponding to each data group in the log dependency information corresponding to the currently scanned memory object have been replayed, the target node number, target minimum LSN and target maximum LSN are updated according to the currently scanned memory object, wherein the target node number, the target minimum LSN and the target maximum LSN are respectively used to record the node number of the optimal log package to be replayed, the minimum LSN value in the optimal log package to be replayed and the maximum LSN value in the optimal log package to be replayed.
2. The method according to claim 1, It is characterized in that Initialize the memory objects of each node in the cluster, including: For each node, scan the log packets in the archive log files in the archive log file list of the node in turn; The first log packet that meets the following conditions is loaded into the memory object of the node: the log packet sequence number of the log packet is greater than the first maximum log packet sequence number that has been replayed, and the maximum LSN value in the log packet is greater than the first maximum LSN that has been replayed.
3. The method according to claim 2, It is characterized in that Before initializing the memory object of each node, also include: For each node, record the first largest log packet sequence number and the first largest LSN that the node has replayed; Scan the archive log files on the node, where the header information of each archive log file includes the second largest log packet sequence number and the second largest LSN of the log packets in the archive log file; If the second maximum log package sequence number is greater than the first maximum log package sequence number, and the second maximum LSN is greater than the first maximum LSN, the archive log file is added to the archive log file linked list of the node.
4. The method according to claim 1, It is characterized in that Update the target node number, target minimum LSN, and target maximum LSN according to the currently scanned memory object, including: If the target node number is an initial value, the target node number is set to the node number of the currently scanned memory object, the target minimum LSN is set to the minimum LSN in the log package loaded by the currently scanned memory object, and the target maximum LSN is set to the maximum LSN in the log package loaded by the currently scanned memory object; If the target node number is not the initial value, compare the target minimum LSN with the minimum LSN in the log package loaded by the currently scanned memory object: If the target minimum LSN is smaller than the minimum LSN in the log package loaded by the currently scanned memory object, the target node number, target minimum LSN, and target maximum LSN remain unchanged; If the target minimum LSN is greater than the minimum LSN in the log package loaded by the currently scanned memory object, the target node number is set to the current node number where the currently scanned memory object is located, the target minimum LSN is set to the minimum LSN in the log package loaded by the currently scanned memory object, and the target maximum LSN is set to the maximum LSN in the log package loaded by the currently scanned memory object; If the target minimum LSN is equal to the minimum LSN in the log package loaded by the currently scanned memory object, then compare the target maximum LSN with the maximum LSN in the log package loaded by the currently scanned memory object; If the target maximum LSN is less than or equal to the maximum LSN in the log package loaded by the currently scanned memory object, the target node number, target minimum LSN, and target maximum LSN remain unchanged; If the target maximum LSN is greater than the maximum LSN in the log package loaded by the currently scanned memory object, the target node number is set to the current node number where the currently scanned memory object is located, the target minimum LSN is set to the minimum LSN in the log package loaded by the currently scanned memory object, and the target maximum LSN is set to the maximum LSN in the log package loaded by the currently scanned memory object.
5. The method according to claim 1, It is characterized in that After scanning of each of the memory objects is completed, the optimal log package to be replayed is replayed, including: After scanning of each memory object is completed, if the target node number is not the initial value, replaying the log package loaded in the memory object corresponding to the target node number; According to the replayed log package, the replayed first maximum log package sequence number and the replayed first maximum LSN registered in the memory object corresponding to the target node number are updated, and the loaded log package in the memory object corresponding to the target node number is cleared; If the target node number is an initial value, determining whether no log package is loaded in each of the memory objects; If so, the logs of each of the nodes have been replayed; otherwise, other logs that the log packages of one or more nodes depend on are lost, and the log package replay is stopped and an error message is returned.
6. A log packet replay device, It is characterized in that include: A memory object initialization module, configured to initialize a memory object of each node in the cluster, wherein the memory object is loaded with a log package, and the log package is associated with log dependency information; A determination module is configured to scan each of the memory objects in sequence, and during each scanning process, determine the optimal log package to be replayed according to the log dependency information corresponding to the currently scanned memory object; A replay module, configured to replay the optimal log package to be replayed after scanning of each memory object is completed; The device also includes: An information determination module, configured to determine log dependency information of each log packet according to the node number of the modified data page recorded in the log packet and the maximum LSN value corresponding to the corresponding node; Determine the modules, including: A data group scanning unit is configured to sequentially scan each data group in the log dependency information corresponding to the currently scanned memory object, and for the currently scanned data group, find a target memory object having the same node number as that in the currently scanned data group from each of the memory objects, and a first maximum LSN that has been repeated corresponding to the target memory object; The replay determination unit is configured to: if the first maximum LSN corresponding to the target memory object that has been replayed is greater than or equal to the LSN value in the data group currently being scanned, then the log corresponding to the data group currently being scanned has been replayed; otherwise, the log corresponding to the data group currently being scanned has not been replayed; The scanning unit is configured to continue scanning the next memory object if there are one or more logs corresponding to the data groups that have not been repeated in the log dependency information corresponding to the currently scanned memory object; The updating unit is configured to update the target node number, the target minimum LSN and the target maximum LSN according to the currently scanned memory object if the logs corresponding to each data group in the log dependency information corresponding to the currently scanned memory object have been replayed, wherein the target node number, the target minimum LSN and the target maximum LSN are respectively used to record the node number of the optimal log package to be replayed, the minimum LSN value in the optimal log package to be replayed and the maximum LSN value in the optimal log package to be replayed.
7. An electronic device, It is characterized in that include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the log packet replay method as described in any one of claims 1-5.
8. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the log packet replay method as described in any one of claims 1 to 5 is implemented.
Citation Information
Patent Citations
Log replay method and device, server and storage medium
CN110442560A
Data recovery method, device and equipment and storage medium
CN111290881A