Database Node Restart Processing Method, Device, and Medium Based on Delayed Replay

Abstract

The embodiments of the present application disclose a method, device, and medium for restarting a database node based on delayed replay, belonging to the technical field of databases, and solving the problem of business discontinuity when a standby node of a database configured with delayed replay restarts. It includes performing restart detection and status capture on the primary node and the standby node respectively; determining the peak-valley period corresponding to the current time window, and adjusting the parameter configuration file of the standby node based on the peak-valley period; after the adjustment is completed, in the corresponding local real-time synchronization area, through streaming replication, the latest WAL log pulled from the primary node is synchronously replicated in real time; in the corresponding local delayed replay area, a delay queue is constructed based on the historical WAL logs that have not been applied locally, and the delay queue is dynamically sorted, and the sorted delay queue is replayed through asynchronous sharding; when it is detected that the real-time synchronization area is in a stable state of streaming replication, the parameter configuration file is restored and adjusted to complete the restart processing of the database node.

Description

Technical Field

[0001] This application relates to the field of database technology, and in particular to a method, device, and medium for restarting a database node based on delayed replay. Background Art

[0002] In modern enterprise-level database systems, to ensure high data availability and disaster recovery capabilities, the master-slave architecture has become the mainstream deployment mode. The slave node uses the streaming replication technology to synchronize the transaction logs of the master node in real time, ensuring that it can quickly take over the service in case of a master node failure and maintain business continuity.

[0003] However, when a slave node of a database with delayed replay configuration restarts, the system will face serious performance bottlenecks and service interruption risks. After restarting, the slave node needs to fully replay the locally accumulated WAL logs, which may take several hours due to factors such as large log data volume and high transaction complexity. Before the replay is completed, the slave node cannot establish a streaming replication connection with the master node, resulting in data synchronization interruption. During this period, new transactions generated by the master node cannot be synchronized to the slave node in time. If the master node fails at this time, the slave node cannot fully take over the service due to data loss, which will cause business interruption and thus result in business discontinuity. Summary of the Invention

[0004] Embodiments of this application provide a method, device, and medium for restarting a database node based on delayed replay, which are used to solve the following technical problems: When a slave node of a database with delayed replay configuration restarts, new transactions generated by the master node cannot be synchronized to the slave node in time, which will cause business interruption and thus result in business discontinuity.

[0005] Embodiments of this application adopt the following technical solutions:

[0006] Embodiments of this application provide a method for restarting a database node based on delayed replay. The method includes: when a restart signal of the slave node is obtained, performing restart detection and status capture on the master node and the slave node respectively; when the detection is passed, determining the peak-valley period corresponding to the current time window, and adjusting the parameter configuration file of the slave node based on the peak-valley period; after the adjustment is completed, based on the captured status information, in the corresponding local real-time synchronization area, performing real-time synchronization of the latest WAL logs pulled from the master node through streaming replication; and in the corresponding local delayed replay area, constructing a delay queue based on the locally unapplied historical WAL logs, and dynamically sorting the delay queue, so as to replay the sorted delay queue through asynchronous sharding; when it is detected that the real-time synchronization area is in a stable state of streaming replication, performing recovery adjustment on the parameter configuration file to complete the restart processing of the database node.

[0007] In the embodiments of the present application, a stream replication is established within seconds, the stream replication recovery time is greatly shortened, and asynchronous processing through a delay queue is used to achieve real-time synchronization between the primary and standby nodes without mutual influence. Secondly, the rapid establishment of real-time stream replication ensures that the primary node continuously sends the latest WAL logs, avoiding the risks of WAL log accumulation and interruption at the primary node. In addition, the asynchronous replay of local historical WAL logs in the embodiments of the present application reduces resource contention for real-time stream replication. The primary node does not need to retain a large number of historical WAL logs for a long time, reducing storage pressure and improving resource utilization. Even if the delay queue is not completed, the standby node can still provide read-only queries of the latest data, supporting failover and load balancing to ensure business continuity.

[0008] In one implementation manner of the present application, when a restart signal of the standby node is obtained, restart detection and status capture are respectively performed on the primary node and the standby node, specifically including: obtaining the restart signal of the standby node through a database log or a system daemon process; calling an information acquisition instruction to obtain control information corresponding to the current node, and detecting the last consistent log sequence number before restart; when the detection passes, saving the delay parameter value, the physical replication slot name, and the primary node before restart; and recording the list of local unapplied WAL log files and the corresponding log sequence number range.

[0009] In one implementation manner of the present application, detecting the last consistent log sequence number before restart specifically includes: obtaining a reference log sequence number in a preset control file; where the reference log sequence number is the position of the WAL log that has completed consistency processing corresponding to a preset historical moment; comparing the reference log sequence number with the WAL log corresponding to the standby node; determining the continuity between the prev-LSN field of the current WAL log and the end log sequence number of the previous WAL log; and performing matching verification on the WAL log through a cyclic redundancy check method; when the WAL log is continuous and the matching verification passes, it is determined that the detection of the last consistent log sequence number before restart passes.

[0010] In one implementation manner of the present application, determining the peak-valley period corresponding to the current time window and adjusting the parameter configuration file of the standby node based on the peak-valley period specifically includes: determining the peak-valley period corresponding to the current time window; when the current time window belongs to the low valley period, configuring to retain some delay parameters; when the current time window belongs to the peak period, configuring to completely disable the delay parameters, and determining the delay parameters in the parameter configuration file; setting the delay parameters to 0; after the parameter configuration file is adjusted, sending a SIGHUP signal to reload the configuration to make the delay parameters take effect.

[0011] In an implementation manner of the present application, dynamic sorting of the delay queue is performed, specifically including: determining corresponding priority data in a preset load status table based on the load status corresponding to peak and valley periods; performing a first weight assignment to the delay queue according to the priority data; performing associated data mining on historical transaction data through federated learning based on the delay queue to determine associated data in the historical transaction data, and performing a second weight assignment to the associated data according to a preset association degree; dynamically sorting the delay queue through the first weight assignment and the second weight assignment.

[0012] In an implementation manner of the present application, asynchronous sharding is used to replay the sorted delay queue, specifically including: obtaining network status parameters, and determining multiple replay policies based on the network status parameters; where the network status parameters at least include one of network bandwidth, delay parameters, and packet loss rate; inputting the multiple replay policies into a preset digital twin model to output the replay efficiency corresponding to each replay policy through the preset digital twin model; determining a reference policy among the multiple replay policies based on the replay efficiency, and replaying the sorted delay queue through the reference policy and asynchronous sharding.

[0013] In an implementation manner of the present application, after replaying the sorted delay queue through asynchronous sharding, the method further includes: calling the pg_wal_replay_resume function when it is detected that there is a blocking replay of the local WAL log; skipping the blocking replay of the local WAL log through the pg_wal_replay_resume function, and directly establishing a primary-standby connection based on the sorted delay queue and real-time streaming replication data.

[0014] In an implementation manner of the present application, when it is detected that the real-time synchronization area is in a stable state of streaming replication, the parameter configuration file is restored and adjusted to complete the restart process of the database node, specifically including: starting a background thread to periodically query the difference in log sequence numbers between the primary node and the standby node; determining that the streaming replication has been successfully established when the delay time corresponding to the real-time synchronization area is less than a preset delay time and the continuous stable duration is greater than a preset running duration; restoring the adjusted parameters in the parameter configuration file to their initial values and reloading the configuration; asynchronously replaying the local historical WAL log in the background.

[0015] An embodiment of the present application provides a database node restart processing device based on delayed replay, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: when a standby node restart signal is obtained, perform restart detection and status capture on the primary node and the standby node respectively; when the detection passes, determine the peak-valley period corresponding to the current time window, and adjust the parameter configuration file of the standby node based on the peak-valley period; after the adjustment is completed, based on the captured status information, in the corresponding local real-time synchronization area, perform real-time synchronization of the latest WAL log pulled from the primary node through stream replication; and, in the corresponding local delayed replay area, construct a delay queue based on the local unapplied historical WAL logs, and perform dynamic sorting on the delay queue to perform replay on the sorted delay queue through asynchronous sharding; when it is detected that the real-time synchronization area is in a stable state of stream replication, perform recovery adjustment on the parameter configuration file to complete the database node restart processing.

[0016] A non-volatile computer storage medium provided by an embodiment of the present application stores computer-executable instructions, and the computer-executable instructions are set to: when a standby node restart signal is obtained, perform restart detection and status capture on the primary node and the standby node respectively; when the detection passes, determine the peak-valley period corresponding to the current time window, and adjust the parameter configuration file of the standby node based on the peak-valley period; after the adjustment is completed, based on the captured status information, in the corresponding local real-time synchronization area, perform real-time synchronization of the latest WAL log pulled from the primary node through stream replication; and, in the corresponding local delayed replay area, construct a delay queue based on the local unapplied historical WAL logs, and perform dynamic sorting on the delay queue to perform replay on the sorted delay queue through asynchronous sharding; when it is detected that the real-time synchronization area is in a stable state of stream replication, perform recovery adjustment on the parameter configuration file to complete the database node restart processing.

[0017] The above at least one technical solution adopted by the embodiment of the present application can achieve the following beneficial effects: In the embodiment of the present application, stream replication is established within seconds, and the stream replication recovery time is greatly shortened. Through asynchronous processing of the delay queue, real-time synchronization between the primary node and the standby node is achieved without mutual influence. Secondly, the rapid establishment of real-time stream replication ensures that the primary node continuously sends the latest WAL logs, avoiding the risk of WAL log accumulation and interruption on the primary node. In addition, the asynchronous replay of the local historical WAL logs in the embodiment of the present application reduces the resource contention for real-time stream replication. The primary node does not need to retain a large number of historical WAL logs for a long time, reducing the storage pressure and improving the resource utilization rate. Even if the delay queue is not completed, the standby node can still provide read-only queries of the latest data, supporting failover and load balancing, and ensuring business continuity. Brief Description of the Drawings

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings described below are only some embodiments recorded in the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. In the drawings:

[0019] Figure 1 It is a flowchart of a method for restarting a database node based on delayed replay provided by an embodiment of the present application;

[0020] Figure 2 It is a schematic diagram of the process of node restart and status capture provided by an embodiment of the present application;

[0021] Figure 3 It is a schematic diagram of the process of adjusting delay parameters and streaming replication priorities provided by an embodiment of the present application;

[0022] Figure 4 It is a schematic diagram of the process of monitoring the stability of streaming replication and restoring parameters provided by an embodiment of the present application;

[0023] Figure 5 It is a schematic diagram of the structure of a device for restarting a database node based on delayed replay provided by an embodiment of the present application.

[0024] Reference Signs:

[0025] 500: Device for restarting a database node based on delayed replay, 501: Processor, 502: Memory. Detailed Embodiments

[0026] Embodiments of the present application provide a method, device, and medium for restarting a database node based on delayed replay.

[0027] In order to enable those skilled in the art to better understand the technical solutions in the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0028] The technical solutions proposed in the embodiments of the present invention will be described in detail below with reference to the drawings.

[0029] Figure 1The flowchart of a method for restarting a database node based on delayed replay provided by an embodiment of the present application is as follows: Figure 1 As shown, the method for restarting a database node based on delayed replay includes the following steps:

[0030] Step 101: When a restart signal of the standby node is obtained, perform restart detection and status capture on the primary node and the standby node respectively.

[0031] In an implementation manner of the present application, a restart signal of the standby node is obtained through database logs or a system daemon process. An information acquisition instruction is called to obtain the control information corresponding to the current node, and the last consistent log sequence number before restart is detected. When the detection is passed, the delay parameter value, physical replication slot name, and primary node before restart are saved. In addition, the list of local unapplied WAL log files and the corresponding log sequence number range are recorded.

[0032] Specifically, Figure 2 The flowchart of node restart and status capture provided by an embodiment of the present application is as follows: Figure 2 As shown, node restart and status capture include the following processes:

[0033] Step 201: Capture the restart signal of the standby node through database logs or a system daemon process.

[0034] Step 202: Call the pg_controldata command to obtain the control information of the current node, and confirm the last consistent LSN (log sequence number) before restart.

[0035] Step 203: Save the delay parameter value (recovery_min_apply_delay), physical replication slot name (primary_slot_name), and primary node before restart.

[0036] Step 204: Check the local WAL log directory, and record the list of unapplied WAL files and their corresponding LSN ranges.

[0037] For example: (0000000100000001000000A0 to 000000010000000100000500).

[0038] In an implementation manner of this application, to confirm the last consistent LSN (Log Sequence Number) before restart, including: obtaining a reference log sequence number in a preset control file, where the reference log sequence number is the position of the WAL log that has completed consistency processing corresponding to a preset historical moment. Comparing the reference log sequence number with the WAL log corresponding to the standby node. Determining the continuity between the prev-LSN field of the current WAL log and the end log sequence number of the previous WAL log, and performing matching verification on the WAL log through a cyclic redundancy check method. When the WAL logs are continuous and the matching verification passes, it is determined that the verification of the last consistent log sequence number before restart passes.

[0039] Specifically, obtain the reference LSN in pg_control. Here, pg_control is a binary file in the PostgreSQL database, which is used to store the core metadata of the database cluster. Compare it with the standby node WAL log, introduce CRC (Cyclic Redundancy Check) check to enhance reliability, read the metadata at the head of the WAL file, verify whether its prev-LSN field is continuous with the end LSN of the previous file, and check the CRC32 value. If it is continuous and the CRC matches, the log integrity is confirmed.

[0040] Furthermore, the pg_control file includes a reference LSN (Log Sequence Number). The reference LSN represents the position of the WAL log that has completed consistency processing recognized by the database system and is an important identifier for measuring the data consistency state of the database. The standby node obtains the WAL log from the primary node through streaming replication to achieve data synchronization. Compare the reference LSN obtained from pg_control with the WAL log of the standby node to check whether the WAL log on the standby node is complete and consistent with the data state of the primary node. If the WAL log of the standby node does not match the reference LSN after a certain LSN position, it indicates that there may be missing, damaged, or replication errors in the WAL log of the standby node.

[0041] To further ensure that no data errors occur during the storage and transmission of WAL logs, the CRC technology is introduced. At the receiving end, perform the same calculation on the data again, and compare the obtained CRC value with the previously stored or transmitted CRC value. If the two CRC values are the same, it means that the data has not changed during transmission or storage; otherwise, it indicates that the data may have been damaged or tampered with.

[0042] Furthermore, the header of each WAL file contains important metadata information. The prev-LSN field records the ending LSN of the previous WAL file of the current WAL file. By reading the prev-LSN field in the WAL file header and verifying whether it is continuous with the ending LSN of the previous WAL file, the logical order between WAL files can be determined whether it is correct. WAL logs are generated and recorded sequentially in chronological order, and the continuity between files is crucial for ensuring the integrity and consistency of transaction processing. If the prev-LSN field is not continuous with the ending LSN of the previous file, it means that there may be missing or incorrect in the WAL file sequence, and further inspection and repair are required to ensure the continuity and integrity of the log.

[0043] Furthermore, after verifying the continuity of the prev-LSN field, it is also necessary to check the CRC32 value of the content of the WAL file. By calculating the CRC32 value of the WAL file content and comparing it with the CRC32 value stored in the file, if the two match, it can be confirmed that the WAL file has not suffered data corruption during storage or transmission, and the content is complete and accurate. Only when the prev-LSN field is continuous and the CRC32 value matches can the integrity of the WAL file be finally confirmed. By verifying all WAL files, the integrity of the entire WAL log can be ensured, guaranteeing the stable operation and data consistency of the database system.

[0044] Step 102: In the case of passing the detection, determine the peak-valley period corresponding to the current time window, and adjust the parameter configuration file of the standby node based on the peak-valley period.

[0045] Figure 3 This is a schematic diagram of the process for adjusting delay parameters and streaming replication priorities provided by an embodiment of the present application. As Figure 3 shown, the delay parameter adjustment includes the following steps 301 content:

[0046] Step 301: Before modification, check the current time window (such as peak and valley periods), and dynamically decide whether to completely disable the delay parameter or partially retain it.

[0047] Specifically, modify the postgresql.auto.conf file of the standby node. This file is a configuration file, set recovery_min_apply_delay to 0, and send a SIGHUP signal to reload the configuration.

[0048] In an implementation manner of the present application, the peak-valley period corresponding to the current time window is determined. When the current time window belongs to the low valley period, it is configured to retain some delay parameters. When the current time window belongs to the peak period, it is configured to completely disable the delay parameters, and the delay parameters are determined in the parameter configuration file and set to 0. After the parameter configuration file is adjusted, a SIGHUP signal is sent to reload the configuration to make the delay parameters take effect.

[0049] Specifically, during the low valley period, since the system load is low and the real-time requirement for data synchronization is not so urgent, in order to reduce the processing pressure on the standby node and avoid the standby node from performing high-intensity data processing under low load, some delay parameters can be selected to be retained, and the data synchronization speed can be appropriately slowed down to allow the standby node to process data in a more relaxed state. During the peak period, in order to ensure that the data of the primary node can be quickly synchronized to the standby node, meet the strict requirements for data consistency in high-concurrency scenarios, and avoid data lag caused by delays from affecting the normal operation of the service, it is usually chosen to completely disable the delay parameters.

[0050] Furthermore, postgresql.auto.conf is an important file for the PostgreSQL database to store automatic configuration parameters, which records many key parameters affecting the operation of the database. After determining the adjustment strategy of the delay parameters, corresponding modifications need to be made in this file. The recovery_min_apply_delay parameter is used to control the minimum delay time for the standby node to apply transaction logs. Setting it to 0 means that after the standby node receives the transaction logs sent by the primary node, it will no longer perform any delay waiting and will immediately apply these logs. During the peak period, it can minimize the time interval of data synchronization to ensure that the data of the standby node quickly catches up with the changes of the primary node and realizes the real-time and consistency of data. The role of sending the SIGHUP signal is to notify the database process to reread the configuration file to make the just-modified setting of the recovery_min_apply_delay parameter take effect.

[0051] Step 103: After the adjustment is completed, based on the captured status information, in the corresponding local real-time synchronization area, the latest WAL logs pulled from the primary node are synchronously replicated in real time through streaming replication.

[0052] In an implementation manner of the present application, Figure 3 This is a schematic flowchart of the adjustment of delay parameters and streaming replication priority provided by the embodiment of the present application. As Figure 3 shown, the streaming replication in the real-time synchronization area includes the following steps 302:

[0053] Step 302: In the real-time synchronization area, directly pull the latest WAL log (LSN is greater than the LSN that has been applied locally) from the master node and perform real-time synchronization through the streaming replication process.

[0054] Step 104, and in the corresponding local delayed replay area, construct a delayed queue based on the historical WAL logs that have not been applied locally, and perform dynamic sorting on the delayed queue to replay the sorted delayed queue through asynchronous sharding.

[0055] In an implementation manner of the present application, Figure 3 FIG. is a schematic flow diagram of adjusting delay parameters and streaming replication priorities provided by an embodiment of the present application. As Figure 3 shown, the streaming replication in the delayed replay area includes the content of the following steps 303 to 304:

[0056] Step 303: In the delayed replay area, mark the old WAL logs that have not been applied locally as the "delayed queue", perform dynamic sorting based on the transaction type and data importance (such as payment transactions taking precedence), and start asynchronous sharding replay (adhering to the original delay parameters), while ensuring exclusive access to the real-time streaming replication channel resources.

[0057] In an implementation manner of the present application, based on the load status corresponding to the peak and valley periods, the corresponding priority data is determined in the preset load status table. According to the priority data, a first weight assignment is performed on the delayed queue. Based on the delayed queue, federated learning is used to perform associated data mining on the historical transaction data to identify associated data in the historical transaction data, and a second weight assignment is performed on the associated data according to the preset association degree. Through the first weight assignment and the second weight assignment, dynamic sorting is performed on the delayed queue.

[0058] Specifically, the preset load status table is a pre-set rule table that corresponds different load statuses to the corresponding priority data one by one. Through real-time monitoring, the specific load status during the current peak and valley periods is judged, and based on the monitoring results, accurate positioning is performed in the load status table to find the matching priority data. The delayed queue stores the old WAL logs that have not been applied locally, and these logs record the transaction change information of the database. The first weight assignment is to assign a weight value to each log in the delayed queue based on the previously determined priority data.

[0059] Furthermore, taking the transactions in the delay queue as the starting point, the historical transaction data of multiple data sources is included in the analysis scope. During the data processing process, each data source does not need to directly transmit and share the original data, but performs calculations locally and then securely aggregates the calculation results through encrypted communication. In this process, the hidden correlation relationships between transactions are deeply mined through the federated learning algorithm. For example, it is found that certain transactions often occur after specific transactions, or there are close data interaction relationships between the transactions of certain business lines. These mined transaction data with correlation relationships are the determined associated data.

[0060] Furthermore, the preset correlation degree is a pre-set standard for measuring the tightness of the correlation between transactions. For the associated data mined through federated learning, according to the preset correlation degree rules, weight values are assigned to these data again, that is, the second weight assignment. Transactions with a high correlation degree will be assigned a higher weight, while transactions with a low correlation degree will have a lower weight. The results of the first weight assignment and the second weight assignment are comprehensively considered to calculate the final weight value of each transaction. According to the final weight value, the delay queue is dynamically sorted, with transactions with a high weight ranked at the front of the queue and given priority for processing; transactions with a low weight are ranked behind. Through this dynamic sorting method, during the database delay replay process, transactions that are more critical to the system operation and business process can be preferentially processed, the system resources can be reasonably allocated, the database processing efficiency and data consistency can be improved, and it is ensured that the core business can operate stably and efficiently under different load states.

[0061] In an implementation manner of the present application, network status parameters are obtained, and multiple replay strategies are determined based on the network status parameters; where the network status parameters at least include one of network bandwidth, delay parameters, and packet loss rate. The multiple replay strategies are input into a preset digital twin model to output the replay efficiency corresponding to each replay strategy through the preset digital twin model. Based on the replay efficiency, a reference strategy is determined among the multiple replay strategies to replay the sorted delay queue through the reference strategy and asynchronous sharding.

[0062] Specifically, the network status parameters are obtained in real time through a monitoring mechanism, and these parameters include key indicators such as network bandwidth, delay parameters, and packet loss rate. Based on these parameters, multiple different replay strategies are generated according to pre-set rules and algorithms. For example, when the network bandwidth is low and the packet loss rate is high, a "sharding compression transmission" strategy is generated to shard and compress the WAL log data before transmission, reducing the data volume to adapt to the network conditions. Different strategies target different network environment characteristics, providing multiple options for efficient replay.

[0063] Furthermore, the digital twin model in the embodiments of the present application is a virtual mapping of the real database replay process. Through a large amount of pre-training and optimization of data, it can accurately simulate the actual operation conditions under different replay strategies. Input the generated multiple replay strategies into this preset digital twin model. During the simulation process, the model will comprehensively consider various factors such as network status, characteristics of delay queue data, and system resources, and calculate the replay efficiency corresponding to each replay strategy.

[0064] Furthermore, after obtaining the replay efficiency corresponding to each replay strategy, compare and analyze these efficiency values. The strategy with the highest replay efficiency is determined as the reference strategy. After determining the reference strategy, combined with the asynchronous sharding technology, perform a replay operation on the already sorted delay queue. Asynchronous sharding can divide the WAL log data in the delay queue into multiple segments and process these segments in parallel, further improving the replay efficiency. The combination of the reference strategy and asynchronous sharding enables the database to select the optimal way to replay the delay queue according to the network environment and data characteristics, ensuring that the data replay task can be efficiently and stably completed under different network conditions, and guaranteeing the normal operation of the database and data consistency.

[0065] Step 304: Call the pg_wal_replay_resume function to skip the blocking replay of the local WAL log and directly establish a primary-standby connection based on the dynamically sorted delay queue status and real-time streaming replication data.

[0066] In an implementation manner of the present application, when it is detected that there is a blocking replay of the local WAL log, call the pg_wal_replay_resume function, which is an internal function for resuming the WAL log replay. Through the pg_wal_replay_resume function, skip the blocking replay of the local WAL log and directly establish a primary-standby connection based on the sorted delay queue and real-time streaming replication data.

[0067] Step 105: When it is detected that the real-time synchronization area is in a stable state of streaming replication, perform a recovery adjustment on the parameter configuration file to complete the restart process of the database node.

[0068] In an implementation manner of the present application, start a background thread to periodically query the difference in log sequence numbers between the primary node and the standby node. When the delay time corresponding to the real-time synchronization area is less than the preset delay time and the continuous stable duration is greater than the preset running duration, it is determined that the streaming replication has been successfully established. Restore the adjusted parameters in the parameter configuration file to their initial values and reload the configuration. Asynchronously replay the local historical WAL log in the background.

[0069] Figure 4A schematic flow diagram of stream replication stability monitoring and parameter recovery provided by an embodiment of this application is as follows Figure 4 As shown, stream replication stability monitoring and parameter recovery include the following steps:

[0070] Step 401: Start a background thread to periodically query the LSN difference between the primary and standby nodes.

[0071] Step 402: When the latency in the real-time synchronization area is lower than the threshold (such as 5 seconds) and remains stable for 5 minutes, it is determined that the stream replication has been successfully established.

[0072] Step 403: Restore recovery_min_apply_delay to its initial value (such as 12h) and reload the configuration.

[0073] Step 404: Asynchronously replay the local old WAL logs in the background to ensure data eventual consistency without affecting real-time stream replication.

[0074] Figure 5 A schematic structural diagram of a database node restart processing device based on delayed replay provided by an embodiment of this application. As Figure 5 shown, the database node restart processing device 500 based on delayed replay includes: at least one processor 501; and a memory 502 communicatively connected to the at least one processor 501; wherein, the memory 502 stores instructions executable by the at least one processor 501, and when the instructions are executed by the at least one processor 501, the at least one processor 50 can: perform restart detection and status capture on the primary node and the standby node respectively when a standby node restart signal is obtained; when the detection passes, determine the peak-valley period corresponding to the current time window, and adjust the parameter configuration file of the standby node based on the peak-valley period; after the adjustment is completed, based on the captured status information, in the corresponding real-time synchronization area locally, perform real-time synchronization of the latest WAL logs pulled from the primary node through stream replication; and in the corresponding delayed replay area locally, construct a delayed queue based on the local unapplied historical WAL logs and perform dynamic sorting on the delayed queue, so as to asynchronously shard and replay the sorted delayed queue; when it is detected that the real-time synchronization area is in a stable state of stream replication, perform recovery adjustment on the parameter configuration file to complete the database node restart processing.

[0075] A non-volatile computer storage medium provided by an embodiment of the present application stores computer-executable instructions, which are set as follows: when a restart signal of a standby node is obtained, perform restart detection and status capture on the primary node and the standby node respectively; when the detection passes, determine the peak-valley period corresponding to the current time window, and adjust the parameter configuration file of the standby node based on the peak-valley period; after the adjustment is completed, based on the captured status information, in the corresponding local real-time synchronization area, perform real-time synchronization of the latest WAL log pulled from the primary node through streaming replication; and, in the corresponding local delayed replay area, construct a delayed queue based on the historical WAL logs not applied locally, and perform dynamic sorting on the delayed queue, so as to replay the sorted delayed queue through asynchronous sharding; when it is detected that the real-time synchronization area is in a stable state of streaming replication, perform restoration adjustment on the parameter configuration file to complete the restart processing of the database node.

[0076] The embodiments in the present application are all described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the device, equipment, and non-volatile computer storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments.

[0077] The above are only the embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the embodiments of the present application can have various changes and modifications. These modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for restarting a database node based on delayed replay, characterized in that, The method includes: When a restart signal of the standby node is obtained, perform restart detection and status capture on the primary node and the standby node respectively; When the detection is passed, determine the peak and valley periods corresponding to the current time window, and adjust the parameter configuration file of the standby node based on the peak and valley periods; After the adjustment is completed, based on the captured status information, in the corresponding local real-time synchronization area, perform real-time synchronization of the latest WAL log pulled from the primary node through streaming replication; And, in the corresponding local delayed replay area, construct a delayed queue based on the historical WAL logs that have not been applied locally, and perform dynamic sorting on the delayed queue, so as to replay the sorted delayed queue through asynchronous sharding; When it is detected that the real-time synchronization area is in a stable state of streaming replication, perform recovery adjustment on the parameter configuration file to complete the restart processing of the database node.

2. The method for restarting a database node based on delayed replay according to claim 1, wherein The performing restart detection and status capture on the primary node and the standby node respectively when a restart signal of the standby node is obtained specifically includes: Obtain the restart signal of the standby node through the database log or the system daemon process; Call the information acquisition instruction to obtain the control information corresponding to the current node, and detect the last consistent log sequence number before restart; When the detection is passed, save the delayed parameter value, the physical replication slot name, and the primary node before restart; And record the list of WAL log files that have not been applied locally and the corresponding log sequence number range.

3. The method for restarting a database node based on delayed replay according to claim 2, wherein The detecting the last consistent log sequence number before restart specifically includes: In the preset control file, obtain the reference log sequence number; wherein, the reference log sequence number is the position of the WAL log that has completed the consistency processing at the preset historical moment; Compare the reference log sequence number with the WAL log corresponding to the standby node; Determine the continuity between the prev-LSN field of the current WAL log and the end log sequence number of the previous WAL log; And perform matching verification on the WAL log through the cyclic redundancy check method; When the WAL log is continuous and the matching verification is passed, it is determined that the detection of the last consistent log sequence number before restart is passed.

4. The method for restarting a database node based on delayed replay according to claim 1, wherein The determining the peak and valley periods corresponding to the current time window and adjusting the parameter configuration file of the standby node based on the peak and valley periods specifically includes: Determine the peak and valley periods corresponding to the current time window; When the current time window belongs to the low valley period, configure to retain some delayed parameters; When the current time window belongs to the peak period, configure to completely disable the delayed parameters, and determine the delayed parameters in the parameter configuration file; Set the delayed parameters to 0; After the adjustment of the parameter configuration file is completed, send a SIGHUP signal to reload the configuration to make the delayed parameters take effect.

5. The method for restarting a database node based on delayed replay according to claim 1, wherein The performing dynamic sorting on the delayed queue specifically includes: Based on the load status corresponding to the peak and valley periods, determine the corresponding priority data in the preset load status table; According to the priority data, perform the first weight assignment on the delayed queue; Based on the delay queue, perform associated data mining on historical transaction data through federated learning to identify associated data in the historical transaction data, and perform a second weight assignment on the associated data according to a preset association degree; Perform dynamic sorting on the delay queue through the first weight assignment and the second weight assignment.

6. The method for restarting a database node based on delayed replay according to claim 1, wherein The asynchronous sharding for replaying the sorted delay queue specifically includes: Obtain network state parameters, and determine multiple replay policies based on the network state parameters; wherein, the network state parameters at least include one of network bandwidth, delay parameters, and packet loss rate; Input the multiple replay policies into a preset digital twin model to output the replay efficiency corresponding to each replay policy through the preset digital twin model; Determine a reference policy among the multiple replay policies based on the replay efficiency, and replay the sorted delay queue through the reference policy and asynchronous sharding.

7. The method for restarting a database node based on delayed replay according to claim 1, wherein After replaying the sorted delay queue through asynchronous sharding, the method further includes: When detecting a blocked replay of the local WAL log, call the pg_wal_replay_resume function; Through the pg_wal_replay_resume function, skip the blocked replay of the local WAL log and directly establish a primary-standby connection based on the sorted delay queue and real-time streaming replication data.

8. The method for restarting a database node based on delayed replay according to claim 1, wherein When detecting that the real-time synchronization area is in a stable state of streaming replication, perform recovery adjustment on the parameter configuration file to complete the restart process of the database node, specifically including: Start a background thread to periodically query the difference in log sequence numbers between the primary node and the standby node; When the delay time corresponding to the real-time synchronization area is less than a preset delay time and the continuous stable duration is greater than a preset running duration, determine that the streaming replication has been successfully established; Restore the adjusted parameters in the parameter configuration file to their initial values and reload the configuration; Asynchronously replay the local historical WAL log in the background.

9. A database node restart processing device based on delayed replay, characterized in that The device includes a memory for storing computer program instructions and a processor for executing the program instructions. When the computer program instructions are executed by the processor, the device is triggered to execute the method according to any one of claims 1-8.

10. A non-volatile computer storage medium stores computer-executable instructions, characterized in that, The computer-executable instructions can execute the method according to any one of claims 1-8.

Images