Remote data copying method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention provides a remote data replication method and device, a storage medium and electronic equipment, and relates to the field of computers.The remote data replication method comprises the steps that in response to a remote replication request, replication time information and replication speed information corresponding to target data are obtained, the remote replication request is used for requesting to remotely replicate target data from the master cluster to at least one slave cluster, and the remote replication request carries replication time information and replication speed information; and in an execution time range indicated by the copy time information, according to a copy speed range indicated by the copy speed information, remotely copying the target data from the master cluster to the at least one slave cluster.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and more specifically, to a method and apparatus for remotely copying data, a storage medium, and an electronic device. Background Art

[0002] In distributed file storage systems, traditional remote data replication methods often use fixed times and methods to replicate data. This may cause network bandwidth and system resources to be over-occupied at specific times, especially during business peaks, thereby affecting the performance of normal business operations and user experience, resulting in low efficiency of remote data replication.

[0003] Therefore, there is a technical problem in the related art that the efficiency of remote data replication is low. Summary of the Invention

[0004] The embodiments of the present application provide a method and apparatus for remote data replication, a storage medium, and an electronic device to at least solve the technical problem of low efficiency of remote data replication in related technologies.

[0005] According to one embodiment of the present application, a method for remote replication of data is provided, comprising: in response to a remote replication request, obtaining replication time information and replication speed information corresponding to target data, wherein the remote replication request is used to request remote replication of the target data from a primary cluster to at least one slave cluster, and the remote replication request carries the replication time information and replication speed information; within an execution time range indicated by the replication time information, the target data is remotely replicated from the primary cluster to at least one slave cluster in accordance with a replication speed range indicated by the replication speed information.

[0006] According to another embodiment of the present application, a remote replication device for data is provided, including: an acquisition unit, configured to acquire replication time information and replication speed information corresponding to target data in response to a remote replication request, wherein the remote replication request is used to request remote replication of the target data from a master cluster to at least one slave cluster, and the remote replication request carries the replication time information and replication speed information; and a replication unit, configured to remotely replicate the target data from the master cluster to at least one slave cluster within an execution time range indicated by the replication time information and in accordance with a replication speed range indicated by the replication speed information.

[0007] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is configured to execute the steps of any of the above method embodiments when running.

[0008] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0009] Through the embodiments provided by the present application, dynamic management and optimization of remote data replication is achieved by obtaining and utilizing the replication time and replication speed information of the target data when responding to remote replication requests. Within the execution time range, the replication rate is intelligently adjusted based on the current replication speed information to ensure that data synchronization is completed efficiently without affecting business operations. For example, during business downturns or time periods with good network conditions, the data replication speed can be increased to accelerate the data synchronization process; while during business peaks or network congestion, the replication speed is reduced to avoid interference with normal business operations. It can be understood that this embodiment effectively solves the problems of uneven resource allocation and inefficiency caused by fixed time and methods in traditional remote replication methods, and achieves optimal resource utilization and high efficiency of data replication through flexible replication strategies, thereby achieving the technical effect of improving the efficiency of remote data replication and solving the technical problem of low efficiency of remote data replication. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a hardware structure block diagram of a remote data replication method according to an embodiment of the present application;

[0011] Figure 2 is a flow chart of a method for remote data replication according to an embodiment of the present application;

[0012] Figure 3 is a schematic diagram of a method for remotely replicating data according to an embodiment of the present application;

[0013] Figure 4 This is a structural block diagram of a data remote replication device according to an embodiment of the present application. DETAILED DESCRIPTION

[0014] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0015] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0016] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal or similar computing device. Taking running on a computer terminal as an example, Figure 1 This is a hardware structure diagram of a computer terminal for a remote data replication method according to an embodiment of the present application. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA and other processing devices) and a memory 104 for storing data. The computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. For example, the computer terminal may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0017] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for determining the mapping relationship in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0018] The transmission device 106 is used to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by a communication provider of a computer terminal. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0019] As an optional solution, the remote data replication method includes the following steps:

[0020] S202, in response to a remote replication request, obtaining replication time information and replication speed information corresponding to target data, wherein the remote replication request is used to request remote replication of the target data from the primary cluster to at least one secondary cluster, and the remote replication request carries the replication time information and the replication speed information;

[0021] S204 : Remotely copy the target data from the master cluster to at least one slave cluster within the execution time range indicated by the copy time information and according to the copy speed range indicated by the copy speed information.

[0022] Optionally, in this embodiment, the above-mentioned data remote replication method can be applied to, but is not limited to, disaster recovery and data synchronization scenarios of large-scale distributed file storage clusters. With the explosive growth of data volume, enterprises and cloud service providers have placed higher requirements on data continuity and security, especially in the face of natural disasters, hardware failures, or human errors, so as to be able to quickly restore data and ensure business continuity. This embodiment effectively balances the efficiency of data replication and the impact on business operations by intelligently adjusting the time and speed of remote replication. It is particularly suitable for cloud computing data centers, where data is of high importance and the utilization efficiency of system resources and network bandwidth is strictly required.

[0023] Optionally, in this embodiment, the remote replication request is a request automatically initiated by a user or the system, and is intended to replicate data from the primary cluster to the secondary cluster to achieve redundant storage and disaster recovery backup of the data.

[0024] Optionally, in this embodiment, replication time information refers to the start time and duration of data replication. This can be configured through a timing policy to provide a time window for remote replication operations, thereby avoiding large amounts of data transmission during peak business hours that could impact system performance. Replication speed information refers to the rate at which data is replicated remotely. This can be constant or dynamically adjusted based on system load, network conditions, and business needs, ensuring optimal data replication without hindering business operations or wasting resources.

[0025] Optionally, in this embodiment, the master cluster is the cluster that stores source data, is usually active, and is responsible for daily data reading and writing and business operations. The slave cluster is the cluster that receives remotely copied data from the master cluster and is used for data backup and disaster recovery.

[0026] Optionally, in this embodiment, when the system receives a remote replication request, it first parses the parameters in the request, including the identifier of the target data, replication time information, and replication speed information. For example, suppose a remote replication request is to remotely replicate the financial data directory on the master cluster to two slave clusters, and requires replication to occur between 2:00 AM and 4:00 AM daily at a rate not exceeding 10 MB / s.

[0027] Optionally, in this embodiment, the system further determines the time window and speed limit for remote replication. In the aforementioned example, the time window is set to 2:00 AM to 4:00 AM, which is a low-peak business period, and the replication speed is limited to 10 MB / s to ensure that network resources are not overused.

[0028] Optionally, in this embodiment, within a specified time range, the system will dynamically adjust the replication speed based on current network conditions and system load, aiming to stay as close to, but not exceeding, the specified replication speed range. For example, between 2:00 AM and 4:00 AM, if the system detects a low network load, it can gradually increase the replication speed until it approaches 10 MB / s. However, if it detects a busy network, it will automatically reduce the replication speed to prevent impacting other business operations.

[0029] To illustrate further, consider a data organization whose primary data center is located in the city center. Due to natural disaster risks and business continuity requirements, it decides to establish a secondary data center in a different location and back up critical transaction data through remote replication technology. Using the above method, transaction data can be automatically replicated from the primary data center to the secondary data center during low-peak hours at night, with the replication speed automatically adjusted based on the current network and system resource conditions. This not only fully utilizes idle network bandwidth at night, but also avoids the impact on transaction system performance during peak daytime hours, ensuring that data replication tasks are completed smoothly without disrupting normal business operations, improving data security, and optimizing the efficiency of network resources.

[0030] It can be understood that this embodiment realizes an intelligent and efficient remote replication mechanism by introducing replication time information and replication speed information. Replication time information provides a time window for the execution of remote replication tasks, which is usually a low-peak period for business, and can minimize interference with the production environment. Replication speed information is used to limit the speed of data transmission to ensure that even within the time window, data replication will not excessively occupy valuable network resources or system processing capabilities, causing adverse effects on other businesses. By dynamically adjusting the replication strategy to cope with changes in the external environment, such as fluctuations in network conditions and changes in system load, data replication is ensured to be both efficient and secure, providing strong support for data synchronization and disaster recovery backup of large-scale distributed file storage clusters.

[0031] Through the embodiments provided by the present application, dynamic management and optimization of remote data replication is achieved by obtaining and utilizing the replication time and replication speed information of the target data when responding to remote replication requests. Within the execution time range, the replication rate is intelligently adjusted based on the current replication speed information to ensure that data synchronization is completed efficiently without affecting business operations. For example, during business downturns or time periods with good network conditions, the data replication speed can be increased to accelerate the data synchronization process; while during business peaks or network congestion, the replication speed is reduced to avoid interference with normal business operations. It can be understood that this embodiment effectively solves the problems of uneven resource allocation and inefficiency caused by fixed time and methods in traditional remote replication methods, and achieves optimal resource utilization and high efficiency of data replication through flexible replication strategies, thereby achieving the technical effect of improving the efficiency of remote data replication.

[0032] As an optional solution, remotely replicating the target data from the master cluster to at least one slave cluster within the execution time range indicated by the replication time information and according to the replication speed range indicated by the replication speed information includes:

[0033] Remotely copy the target data from the master cluster to at least one slave cluster at a first copy speed within the execution time range, wherein the first copy speed is a maximum copy speed of the copy speed range;

[0034] During the remote replication of target data at a first replication speed, if a blocking event is detected, the first replication speed is reduced to a second replication speed, and the target data is remotely replicated from the master cluster to at least one slave cluster at the second replication speed, wherein the second replication speed is the minimum replication speed of the replication speed range.

[0035] Optionally, in this embodiment, the first replication speed is the maximum replication speed initially set during the remote replication process based on the replication speed information. This is the highest rate used to accelerate data transmission when sufficient system resources and network conditions permit. The second replication speed is the minimum replication speed to which the system automatically reduces when a congestion event is detected. Congestion events may include network congestion, storage device overload, insufficient system resources, etc. In such cases, reducing the replication speed can prevent further deterioration of the system condition and ensure that other business operations are not affected.

[0036] Optionally, in this embodiment, a blocking event refers to any event encountered during the remote replication process that may limit the data transmission rate, including but not limited to excessive network bandwidth usage, excessive system load, storage device failure or performance degradation, etc. Such events will trigger the system to automatically reduce the replication speed.

[0037] Optionally, in this embodiment, when the execution time range indicated by the replication time information is reached, the system will begin the remote replication process at the first replication speed, i.e., data will be transmitted at the maximum replication speed. For example, assuming the replication time window for financial report data is set from 10 PM to 2 AM each night, and the first replication speed set by the replication speed information is 10 MB / s, then during this time window, the system will transmit data at a speed of 10 MB / s whenever possible.

[0038] The system's built-in monitoring mechanism continuously monitors network and system resource conditions. Once any blocking event is detected during remote replication at the first replication speed, such as when the network bandwidth reaches its upper limit or the system CPU usage reaches its threshold, the system will automatically reduce the first replication speed to the second replication speed, which is the minimum replication speed. For example, if in the above scenario, at 1 a.m. it is detected that the network bandwidth is occupied by other business operations to 90%, in order to avoid further impact on the business, the system will reduce the replication speed to 2MB / s (assuming this is the minimum speed within the replication speed range). This adjustment ensures that data replication continues without further increasing the burden on the system.

[0039] It can be understood that this embodiment emphasizes how to intelligently adjust the replication speed to cope with sudden blocking events when executing remote replication tasks. This method forms a speed adjustment range by setting the maximum replication speed (first replication speed) and the minimum replication speed (second replication speed). The system dynamically adjusts the replication speed based on the real-time monitored network and system load conditions to maintain a stable operating state of the system while ensuring the efficient execution of remote replication tasks. In this way, even in a more complex operating environment, such as network fluctuations, tight system resources, etc., it can be ensured that data replication will not be interrupted or have a serious impact on existing businesses, thereby realizing the automation and intelligence of remote data replication, and significantly improving the disaster recovery and backup capabilities and overall stability of the distributed file storage system.

[0040] Through the embodiments provided in this application, an intelligent remote replication speed control solution is used, which can not only accelerate the data synchronization process according to the preset replication time window and speed range, but also automatically slow down when encountering a blocking event, avoiding negative impacts on the system and business, ensuring the smooth progress of remote replication tasks and the normal operation of the system, thereby achieving the technical effect of improving the efficiency of remote replication of data.

[0041] As an optional solution, after remotely replicating the target data from the master cluster to the at least one slave cluster at the second replication speed, the method further includes:

[0042] During the remote replication of target data at the second replication speed, when the blocking event disappears and no other blocking events occur within the first time period after the blocking event disappears, the second replication speed is increased to a third replication speed, and the target data is remotely replicated from the master cluster to at least one slave cluster at the third replication speed, wherein the third replication speed is obtained by increasing the preset speed every N seconds on the basis of the second replication speed, and the third replication speed is less than or equal to the first replication speed, and N is a positive integer.

[0043] Optionally, in this embodiment, after the blocking event disappears, the system gradually increases the new copy speed. The third copy speed is increased based on the second copy speed according to a preset rule until it reaches the originally set first copy speed.

[0044] Optionally, in this embodiment, the first time period is an observation period after the blocking event disappears, during which the system evaluates whether the replication speed can be safely increased. It is usually a set time window, such as 5 minutes, to ensure that the network and system are stable and can support higher data transmission rates.

[0045] Optionally, in this embodiment, N seconds is the period for adjusting the copy speed. That is, the system checks every N seconds to determine whether to increase the copy speed. N is a positive integer representing a time interval, such as 5 seconds or 10 seconds. The preset speed is a fixed value or percentage by which the copy speed increases during each adjustment, such as increasing the maximum copy speed by 5% every N seconds.

[0046] Optionally, in this embodiment, when the system detects that a congestion event has disappeared and no further congestion events occur within a subsequent first time period, the system begins to gradually increase the second copy speed until it returns to the first copy speed. For example, assuming that the copy speed is reduced from 10 MB / s to 2 MB / s due to network congestion at 1:00 AM, the congestion event disappears at 1:15 AM, and no new congestion events occur within the next 5-minute observation period, the system will begin to gradually increase the copy speed from 2 MB / s by increasing the preset speed every 5 seconds (e.g., increasing the copy speed by 1%).

[0047] The system gradually increases the copy speed until it reaches or approaches the primary copy speed. This process ensures a smooth transition of system resources and network bandwidth, avoiding the potential for new system stress caused by sudden increases. In the example above, if the speed is increased by 1% of 2MB / s (0.02MB / s) every 5 seconds, the copy speed will gradually recover from 2MB / s to 10MB / s over time, achieving a smooth adjustment of the data transfer rate.

[0048] It can be understood that this embodiment further improves the dynamic speed adjustment mechanism during remote replication, especially the processing flow after the blocking event disappears. By setting the first time period as the observation period, the system can evaluate whether the network and system resources have returned to stability, and then safely increase the replication speed to avoid the risk of a new round of blocking caused by a sudden increase in speed. The preset speed and N-second parameter settings allow system administrators to flexibly adjust the recovery speed of the replication speed according to actual conditions and needs. This method not only speeds up the completion of data replication tasks, but also ensures the overall stability of the system and the effective use of resources. Especially in scenarios where large amounts of data are processed and long-distance transmission is required, it can significantly improve the efficiency and quality of remote replication.

[0049] For example, in a large enterprise's cross-regional data synchronization scenario, if the network connection between the primary and secondary data centers experiences brief congestion in the early morning, forcing replication to slow down, the slave method automatically and gradually restores data replication speed to maximum efficiency after the congestion is resolved, without requiring manual intervention. This not only reduces operational costs but also ensures the continuity and efficiency of data synchronization. For large distributed file systems, it is a key means of improving the reliability and performance of remote replication.

[0050] Through the embodiments provided in this application, by introducing a speed-increasing mechanism after the blocking event disappears, it is possible to quickly and safely restore the efficient transmission of remote replication while ensuring system stability, thereby providing more powerful and flexible data management capabilities in practical applications, thereby achieving the technical effect of improving the efficiency of remote replication of data.

[0051] As an optional solution, after remotely replicating the target data from the master cluster to at least one slave cluster at the third replication speed, the method further includes:

[0052] During the remote replication of target data at the third replication speed, if a blocking event is detected, the third replication speed is reduced to a fourth replication speed until the blocking event disappears, and the target data is remotely replicated from the master cluster to at least one slave cluster at the fourth replication speed, wherein the fourth replication speed is obtained by reducing the preset speed every N seconds on the basis of the third replication speed, and the fourth replication speed is greater than or equal to the second replication speed.

[0053] As an optional solution, the fourth replication speed is a replication speed that is reduced from the third replication speed to a lower level when the system detects that the blocking event occurs again, to ensure that the replication operation does not aggravate the existing resource bottleneck.

[0054] Optionally, in this embodiment, during remote data replication at the third replication speed, if the system detects a congestion event again, such as system resources being heavily occupied by urgent tasks, it will reduce the current replication speed (the third replication speed) to a new, lower speed (the fourth replication speed) to alleviate pressure on the system and network. For example, if the current replication speed is 8MB / s and a surge in CPU and network load is detected, the system will automatically reduce the maximum replication speed by 1% every 5 seconds, or by 0.08MB / s every 5 seconds, until a replication speed is found that maintains data replication while ensuring stable system operation.

[0055] Once the blocking event disappears and the system has returned to normal after an observation period, the system will gradually increase the copy speed from the fourth copy speed back to at least the second copy speed to ensure smooth data copy operations. For example, if the blocking event disappears and the system is in good condition during the observation period, the system will automatically gradually restore the copy speed to at least 2MB / s (the lowest speed before the blocking event), and further increase it to higher speeds as conditions permit.

[0056] As can be appreciated, this embodiment further enhances the intelligent and dynamic nature of remote replication speed control, specifically focusing on how to quickly and accurately adjust the replication speed in the event of a blocking event to prevent further degradation of system performance. By establishing a fourth replication speed, the system can immediately take action upon detecting any event that could potentially block remote replication operations, reducing data transmission speed and avoiding resource contention. Furthermore, once the blocking event disappears, the system will appropriately restore the replication speed to at least the second replication speed, ensuring the timeliness and efficiency of data replication tasks.

[0057] It should be noted that this embodiment is suitable for applications with extremely high requirements for data availability and system stability, such as large-scale e-commerce platforms. For example, in a remote replication scenario for transaction data, if a sudden system load spike occurs during the data replication process, the system can automatically adjust the replication speed to prevent the data replication operation from impacting current transaction processing performance. Once the system load returns to normal, the data replication speed will automatically increase, ensuring data integrity and timeliness while reducing potential disruption to business operations.

[0058] Through the embodiments provided in this application, dynamic and adaptive replication speed control is used to not only reflect the intelligent management of the remote replication process, but also emphasize how to achieve optimal resource utilization and efficient system operation by dynamically adjusting the replication speed in a complex and changing working environment. This is of great significance for improving the overall performance and user experience of the distributed storage system.

[0059] As an optional solution, before increasing the second copying speed to the third copying speed, the method further includes:

[0060] Obtaining a service bandwidth and a number of input and output operations of at least one slave cluster in a second time period;

[0061] When the service bandwidth is less than a preset bandwidth threshold and the number of input and output operations is less than a preset number, determining that at least one slave cluster meets the expected service condition;

[0062] In a case where at least one slave cluster meets the expected service condition, determining the first preset threshold as the preset threshold;

[0063] In a case where at least one slave cluster does not meet the expected service condition, the second preset threshold is determined as the preset threshold, wherein the first preset threshold is greater than the second preset threshold.

[0064] Optionally, in this embodiment, the second time period is a set period for monitoring business activities of the slave cluster during the remote replication process, which is different from the first time period and is mainly used to ensure that the workload of the slave cluster is not too high when receiving data.

[0065] Optionally, in this embodiment, the service bandwidth refers to the actual network transmission rate used for service operations from the cluster within a specified time. The number of input and output operations, referred to as the number of I / O operations, refers to the total number of read and write operations on the cluster within a specified time.

[0066] Optionally, in this embodiment, the expected service condition is a preset slave cluster service activity standard, including a maximum allowed service bandwidth and a maximum number of I / O operations, which is used to determine whether the system is in a low-load state during remote replication execution.

[0067] Optionally, in this embodiment, the first and second preset thresholds are used to compare standard values ​​for service bandwidth and the number of I / O operations. The first preset threshold is used when expected service conditions are met, and the second preset threshold is used when they are not. Typically, the first preset threshold is set at a higher level to allow more data transmission without affecting service, while the second preset threshold is set at a lower level to limit data transmission and ensure smooth service operations.

[0068] Optionally, in this embodiment, before considering increasing the second replication speed to the third replication speed, the system first collects the service bandwidth and number of input and output operations of the slave cluster during the second time period to assess its current workload and network conditions. For example, the system monitors the average service bandwidth and number of I / O operations of the slave cluster from 1:30 AM to 1:45 AM.

[0069] The system compares the collected service bandwidth and number of I / O operations with preset thresholds to determine whether the slave cluster meets expected service conditions. If the service bandwidth is below the first preset threshold and the number of I / O operations is also below the first preset threshold, the system considers the slave cluster to be under low load and suitable for increasing the replication speed. Conversely, if the service bandwidth approaches or exceeds the second preset threshold, or the number of I / O operations approaches or exceeds the second preset threshold, the system considers the slave cluster to be under high load and it is not appropriate to increase the replication speed, and may even need to reduce the replication speed.

[0070] If the secondary cluster meets expected service conditions during the second time period (i.e., both its service bandwidth and the number of I / O operations are below the first preset threshold), the system will increase the second replication speed to the third replication speed as planned to accelerate data transfer. Otherwise, if the service bandwidth or the number of I / O operations on the secondary cluster exceeds the second preset threshold, the system will adopt a more conservative strategy and may even reduce the current replication speed to prevent negative impacts on ongoing service operations.

[0071] For example, in a cloud service provider scenario, when a primary data center remotely replicates large amounts of data to a secondary data center, the slave method can monitor the secondary data center's network bandwidth and I / O operations in real time and automatically adjust the replication speed. If the secondary data center's business bandwidth and I / O operations are low during the early morning hours, meeting expected business conditions, the system will begin to increase the replication speed according to the established plan to speed up data transmission. Conversely, if the secondary data center experiences increased business activity during a certain period, the system will immediately detect and reduce the replication speed, ensuring that even when the secondary cluster is under additional workload, remote replication operations will not significantly impact its business operations.

[0072] Through the embodiments provided in this application, attention is paid to the real-time business status of the slave cluster to ensure that the normal business operations of the slave cluster will not be interfered with when the replication speed is increased. By introducing the concept of monitoring the business bandwidth and the number of I / O operations and the expected business conditions, the system can dynamically adjust the replication speed according to the actual working status of the slave cluster. When the slave cluster is in a low-load state, by increasing the replication speed, network resources can be fully utilized and the data synchronization process can be accelerated; when the slave cluster is in a high-load state, the system will take measures to reduce the replication speed or maintain it at a lower level to protect the smooth progress of business operations and avoid business congestion caused by data replication.

[0073] As an optional solution, in the process of increasing the second copying speed to the third copying speed, the method further includes:

[0074] Obtaining a first number of data replicas in a data recovery state and a second number of data replicas in a data backfill state from at least one cluster;

[0075] When the first quantity and the second quantity meet the expected quantity condition, the preset threshold is adjusted.

[0076] Optionally, in this embodiment, the data recovery state refers to the state in which the data replica is performing data reconstruction or recovery operations in the slave cluster. This typically occurs after partial data loss or corruption, and the system is repairing and restoring the complete data set. The data backfill state refers to the process in which the data replica is performing data consistency verification and updating in the slave cluster. This is part of the remote replication operation and ensures that the data between the master and slave clusters is completely consistent.

[0077] Optionally, in this embodiment, the first number and the second number represent the number of data copies in the slave cluster that are in a data recovery state and a data backfill state, respectively. These two indicators reflect the scale of the data repair and synchronization operations currently being performed by the slave cluster. The expected number condition is used to indicate the upper limit or range of the number of data copies in the data recovery state and the data backfill state preset by the system, and is used to determine whether the slave cluster is ready to accept data streams with higher replication speeds. If the first number and the second number exceed the expected number condition, it indicates that the slave cluster is processing a large number of data recovery and backfill operations and may not be able to handle additional data streams; conversely, if it is lower than the expected number condition, it indicates that the slave cluster is relatively idle and can take on higher data transmission tasks.

[0078] Optionally, in this embodiment, when the system decides to increase the replication speed from the second replication speed to the third replication speed, it will first obtain in real time the number of data copies (first number and second number) in the data recovery state and data backfill state in the slave cluster to evaluate the current workload of the slave cluster.

[0079] The system compares the first and second numbers obtained in real time with a preset expected number condition. For example, the expected number condition might be that the sum of the first and second numbers must not exceed 1000 replicas. If the current sum of the first and second numbers is lower than this value, it indicates that the slave cluster's current processing power is sufficient to support a higher data transfer rate. Conversely, if the sum exceeds the preset upper limit, the system determines that the slave cluster is under heavy load and is not suitable for increasing the replication speed.

[0080] If the first and second quantities meet the expected quantity conditions, meaning the slave cluster is in an acceptable data processing state, the system will adjust the preset thresholds based on actual conditions, thereby increasing the replication speed to the third speed. Adjustments to the preset thresholds can include increasing the preset service bandwidth or I / O operation thresholds to allow for more data transfer while still ensuring system stability. For example, if the preset thresholds are initially set at an average service bandwidth of 100 Mbps and an I / O operation count of 10,000 times / second, and the slave cluster data replication status is assessed as good, the preset thresholds can be adjusted to 120 Mbps and 12,000 times / second, thereby increasing the replication speed.

[0081] For example, in a storage cluster, suppose some hard drives fail and the primary cluster is sending a large amount of data to the secondary cluster to restore data integrity. At this point, a large number of data copies are being restored in the secondary cluster. Without proper monitoring and adjustment, blindly increasing the replication speed could lead to excessive resource consumption in the secondary cluster, impacting data recovery efficiency and other business operations. However, in this scenario, the secondary method ensures that the data recovery operations in the secondary cluster meet the expected quantity before increasing the replication speed. This avoids unnecessary resource contention, ensures the orderly execution of data recovery and remote replication tasks, and improves the disaster recovery response capabilities and data replication efficiency of the entire system.

[0082] Through the embodiments provided by the present application, by real-time monitoring of the recovery and backfill status of data copies in the slave cluster, the system can more accurately determine the optimal time and speed for data transmission. This method ensures that when the data copy processing volume is controllable, the copy speed can be safely increased and the data synchronization progress can be accelerated; when the data copy processing volume exceeds expectations, the copy speed is maintained or reduced to prevent overloading of the slave cluster, thereby protecting the stability and security of the remote copy operation. For large-scale distributed file storage clusters, this method can effectively avoid a large number of remote copy operations during peak periods of data recovery and backfilling, reduce potential system pressure and network delays, and ensure high data availability and disaster recovery backup capabilities.

[0083] As an optional solution, before obtaining the copy time information and copy speed information corresponding to the target data in response to the remote copy request, the method further includes:

[0084] Creating a remote replication request, wherein the remote replication request is used to request remote replication of target data from the master cluster to a first slave cluster and a second slave cluster, wherein the at least one slave cluster includes the first slave cluster and the second slave cluster;

[0085] Configure first replication time information and first replication speed information corresponding to target data for the remote replication request, where the first replication time information is used to indicate a first execution time range for remote replication of the target data from the primary cluster to the first slave cluster, and the first replication speed information is used to indicate a first replication speed range for remote replication of the target data from the primary cluster to the first slave cluster;

[0086] Configuring second replication time information and second replication speed information corresponding to the target data for the remote replication request, the second replication time information is used to indicate a second execution time range for remote replication of the target data from the primary cluster to the second slave cluster, and the second replication speed information is used to indicate a second replication speed range for remote replication of the target data from the primary cluster to the second slave cluster;

[0087] Among them, the copy time information includes the first copy time information and the second copy time information, the copy speed information includes the first copy speed information and the second copy speed information, the execution time range includes the first execution time range and the second execution time range, and the copy speed range includes the first copy speed range and the second copy speed range.

[0088] Optionally, in this embodiment, the first slave cluster and the second slave cluster refer to the first and second target slave clusters in the remote replication operation, respectively. In actual applications, there may be multiple slave clusters, each with its own replication time and speed settings.

[0089] Optionally, in this embodiment, the first replication time information / second replication time information is used to indicate the time window for transmitting the target data to the first slave cluster and the second slave cluster, respectively, ensuring that data replication occurs within an appropriate time period to avoid impacting normal business operations. The first replication speed information / second replication speed information is used to indicate the data transmission rate range when the target data is transmitted to the first slave cluster and the second slave cluster, respectively. By controlling the replication speed, system resource allocation can be optimized to avoid network congestion or storage resource overload.

[0090] Optionally, in this embodiment, before the system initiates a remote replication operation, it first creates a remote replication request containing information about the target data, the primary cluster, and at least one secondary cluster. For example, a company may need to replicate financial report data to two geographically distinct secondary clusters to enhance data availability and security. In this example, the target data is the financial report data, and the two secondary clusters are the first secondary cluster located in Beijing and the second secondary cluster located in Shanghai.

[0091] For each slave cluster, you need to configure replication time and speed information separately. For example, for the first slave cluster in the above scenario, assuming its replication time is from 10:00 PM to 2:00 AM every night and the replication speed range is 2 MB / s to 10 MB / s, then this is the first replication time and speed information.

[0092] Similarly, you also need to configure the corresponding replication time information and replication speed information for the second slave cluster. Assuming that the replication time of the second slave cluster is 01:00 to 05:00 the next day and the replication speed range is 3MB / s to 8MB / s, then this is the second replication time information and the second replication speed information.

[0093] After all necessary information is configured, the system will transfer the target data to the first slave cluster within the first execution time range according to the first replication speed range, and transfer the target data to the second slave cluster within the second execution time range according to the second replication speed range, ensuring that data can be synchronized quickly without adversely affecting the normal business operations of each slave cluster.

[0094] For example, in the aforementioned enterprise scenario, remote replication of financial report data ensures that data replication occurs during nighttime low-peak hours and at a moderate transmission speed, using pre-set primary replication time and speed information. This speeds up data synchronization while avoiding impacting business operations during peak daytime hours. Similarly, configuring secondary replication time and speed information allows the system to tailor an optimal replication strategy for each slave cluster, taking into account the varying network environments or resource constraints of the secondary slave cluster at specific times.

[0095] Through the embodiments provided in the present application, by configuring specific replication time information and replication speed information for different slave clusters, the system can flexibly arrange the timing and speed of data transmission based on factors such as the location of the slave cluster, network conditions, and resource usage, thereby ensuring rapid data synchronization while avoiding excessive burden on slave cluster resources and achieving refined management of remote replication operations.

[0096] As an alternative solution, the aforementioned remote data replication method is applied to a scenario based on adaptive optimization of remote replication of massive distributed files. In this scenario, remote disaster recovery solutions emerge to ensure the continuity, recoverability, and high availability of data storage. Remote replication technology is one of the key technologies in remote disaster recovery solutions. Its core concept is to synchronously replicate data to multiple locations, thereby minimizing data loss caused by natural disasters or human damage.

[0097] In massive distributed file systems, remote replication and scheduling strategies are crucial for ensuring high data availability and disaster recovery. However, with the rapid growth of data volumes, traditional remote replication and scheduling strategies may not meet performance and efficiency requirements.

[0098] Remote replication supports setting up separate scheduled policies. When a pair (a link between a local cluster and a remote cluster, specifying the remote replication directory and rate, and responsible for directory management) performs polling, it determines whether a synchronization policy has been triggered. If so, a snapshot is generated and automatically deleted after synchronization is complete. Snapshots created by other methods (including snapshots created by a directory's scheduled snapshot policy and manually created by the user) are not automatically deleted by remote replication even after synchronization is complete.

[0099] Data migration priorities are categorized into business-first, adaptive, and data reconstruction-first based on migration speed, with the impact on the business ranked from smallest to largest. However, existing implementations of adaptive priority simply set the data reconstruction speed to a value between business-first and reconstruction-first, and fail to automatically adjust based on business pressure.

[0100] In view of the above problems, this embodiment proposes an adaptive optimization solution based on remote replication of massive distributed files.

[0101] It should be noted that, before implementing the remote replication timing policy, this embodiment needs to create a pair first, such as Figure 3 As shown in the figure, if the master-slave cluster connection is normal, the master-slave cluster file system is normal, and a remote connection has been established, the management software requests icfsfs-mirror to create a pair. After parsing and verifying the parameters, icfsfs-mirror requests the remote cluster to create a pair. The remote cluster verifies the parameters, creates the pair, writes the data to disk, and returns it to icfsfs-mirror. icfsfs-mirror then requests the local cluster to create a pair. The local cluster verifies the parameters, creates the pair, writes the data to disk, and returns it to icfsfs-mirror.

[0102] It should be noted that after the pair is created, it is in a paused state, the resource protection status is unprotected, and the slave resource data status is incomplete. Remote replication cannot be performed at this time. You must add slave resource protection and perform synchronization before replication can be performed. The creation operation is synchronized to all MDSs in the cluster. Synchronization prevents deletion of the synchronization directory, changes to the synchronization directory path, and deletion of the last successfully synchronized snapshot in the master directory and subsequent unsynchronized snapshots.

[0103] In massive distributed file systems, remote replication and scheduling policies are crucial for ensuring high data availability and disaster recovery. However, with the rapid growth of data volumes, traditional remote replication and scheduling policies may not meet performance and efficiency requirements. Therefore, an optimized solution is needed to improve the efficiency of remote replication and scheduling policies for massive distributed files.

[0104] Optionally, in this embodiment, a method for setting and modifying a timing policy is as follows:

[0105] To add or modify a scheduled policy, select a pair in the remote replication management interface. You can select "Interval Policy," "Daily Policy," "Weekly Policy," or "Monthly Policy" for a scheduled policy. You can have a maximum of one policy per policy. Interval policies use custom "hours and minutes" as their units. Daily policies can use "0-23 hours" and custom "minutes." Weekly policies can use "Monday to Sunday" and custom "hours and minutes." Monthly policies can use "Day 1 to 31" and custom "hours and minutes."

[0106] A single pair can support up to four policies at the same time, with a maximum of one policy per policy.

[0107] For example, the command icfs daemon client.mirrortimedsnap set "parameters". Parameter description: pair_name: pair name (required), interval policy (interval): hour-minute (2-5 means 2 hours and 5 minutes), daily policy (daily): hour-minute (2-5 means 2 hours and 5 minutes every day, multiple policies can be set separated by spaces), weekly policy (weekly): week-hour-minute (2-2-5 means 2 hours and 5 minutes on the 2nd of every week), monthly policy (monthly): week-hour-minute (2-2-5 means 2 hours and 5 minutes on the 2nd of every month).

[0108] For example, the command icfs daemon client.mirrortimedsnap set "pair_name: pair1, interval: 2-5, monthly: 2-2-53-2-5" sets the scheduled synchronization interval for pair1 to 2 hours and 5 minutes, with the monthly policy being 2:05 on the 2nd and 2:05 on the 3rd.

[0109] Optionally, in this embodiment, the main process of adaptive optimization of the remote replication timing policy is as follows:

[0110] The maximum adaptive priority speed is calculated using the adaptive priority single OSD reconstruction speed configuration and cluster / storage pool parameters. The minimum adaptive priority reconstruction speed is calculated using the service priority single OSD reconstruction speed configuration and cluster / storage pool parameters. When the remote replication migration priority is set to adaptive, the full adaptive priority speed is used by default.

[0111] Optionally, when a block op (blocking event) occurs for the first time in a data pool during migration, the migration speed of the data pool is immediately reduced to a business-priority migration speed. The number of block ops that occur during the migration of the storage pool is marked.

[0112] Optionally, if the blocking op persists, maintain the business-first migration speed. If the blocking op disappears, after 5 minutes of stabilization, increase the migration speed by a certain recovery speed every 5 seconds (a configurable percentage of the maximum speed). If no blocking op occurs during this increase, continue increasing to the full speed.

[0113] Optionally, if a block op occurs, the migration speed is immediately reduced by a certain migration speed (configurable percentage of the maximum speed) every 5 seconds until the block op disappears.

[0114] Optionally, after the data pool block op disappears and remains stable for 5 minutes, the migration speed is increased by a certain percentage (configurable percentage of the maximum speed) every 5 seconds until the adaptive full speed is reached. If the block op reappears after increasing the migration speed, the migration speed is reduced after 5 cycles until the block op disappears and the speed is temporarily not increased.

[0115] Optionally, after the migration speed is maintained for more than 60 minutes, the number of block op occurrences is reset to zero.

[0116] Optionally, after the block op count is cleared, the system will continue to attempt to increase the migration speed to the adaptive full speed. If a block op occurs after the block op count is cleared, the system will re-enter the aforementioned step of immediately reducing the migration speed of the data pool to the business-priority migration speed.

[0117] Through the embodiments provided by the present application, through a rich remote replication filtering strategy, intelligent file management is achieved, thereby meeting the diverse needs of users for improved flexibility, optimized resource utilization, enhanced fault recovery capabilities, enhanced security and file processing performance, so as to achieve the purpose of meeting different application scenarios, optimizing network bandwidth and system resource utilization, etc. After the remote replication migration is started, it will have an impact on storage performance. The current optimization point is: do not block the business under adaptive priority, and increase the migration speed as much as possible without blocking the business. The remote replication migration speed is set under adaptive priority to ensure that the impact on performance is that there is no long-term block op. It does not affect the normal reading and writing of existing businesses. Adaptive data migration optimization adds the function of adjusting the data migration speed according to business pressure under the data migration adaptive priority on the basis of the existing data migration speed control function. Under adaptive priority, MON increases or decreases the data migration speed by monitoring whether there is a block op in the cluster.

[0118] Optionally, in this embodiment, the optimization of data migration speed includes the following contents.

[0119] Control recovery speed based on cluster size: Instead of having OSDs (Object Storage Devices) directly read the cluster recovery speed configuration, MONs (Monitors) now read the individual OSD recovery speed configuration, calculate the cluster recovery speed, and send it to the OSDs. Add individual OSD data recovery speed configuration for each priority level, including recovery bandwidth and number of operations (OPs).

[0120] Traverse all PGs (Placement Groups) in the timer, count the number of primary OSDs of PGs in the recovering state (data recovery state) and the backfilling state (data backfilling state), and count the total number of objects that need to be reconstructed in these PGs.

[0121] Calculate the data reconstruction speed. The calculation method is to obtain the reconstruction speed configuration of a single OSD at the current data reconstruction priority, multiply it by the number of primary OSDs in the reconstruction state, and obtain the total data reconstruction speed of the cluster.

[0122] The calculated total data reconstruction speed and the statistically obtained total object data that needs to be reconstructed are sent to the OSD.

[0123] Optionally, in this embodiment, full-speed reconstruction is performed when there is no business or low business volume:

[0124] Increase the single OSD recovery speed configuration under low io, increase the bandwidth, IO threshold, and duration configuration for Iow io judgment, and read by MON.

[0125] The current service bandwidth and IO are obtained from the timer. When both are less than the threshold and exceed the configured duration, it is determined to be in the low io state. This step is divided into determining the low io state of the cluster and the low io state of the storage pool.

[0126] If the current state is low io, the data reconstruction speed is calculated using the newly added low io data reconstruction speed configuration instead of the reconstruction speed of a certain data reconstruction priority.

[0127] Optionally, in this embodiment, the recovery speed is automatically adjusted during single disk reconstruction:

[0128] Added the data reconstruction speed calculation coefficient configuration for single-disk reconstruction, which is read by MON.

[0129] The timer counts the number of OSDs with missing objects, including the primary and secondary OSDs of the PG.

[0130] To calculate the single-disk data reconstruction speed, multiply the single-OSD data reconstruction speed at the current data reconstruction priority or low IO by the configured coefficient, and then multiply by the number of OSDs with missing data. If the calculated data reconstruction speed is lower than the speed calculated based on the cluster size, the single-disk data reconstruction speed is sent to the OSD.

[0131] It's understandable that optimizing the timing strategy for distributed remote file replication makes data transmission more efficient. The system dynamically adjusts replication time and frequency based on factors such as network conditions, file size, and file importance, avoiding large amounts of data transfer during periods of network congestion, thereby improving data transmission efficiency and stability.

[0132] By adaptively adjusting replication strategies, we can cope with various complex network environments and system load fluctuations. Whether in situations with limited network bandwidth and high system load, or in situations with smooth network and idle system, we can maintain high data replication efficiency and stability, thereby improving the stability and reliability of the entire system.

[0133] By automatically adjusting replication strategies based on system status and needs, the need for manual intervention is reduced. This not only reduces maintenance costs but also improves system maintainability and scalability.

[0134] Alternatively, in this embodiment, assume that in a large-scale scientific research project, multiple laboratories are distributed around the world and need to share and synchronize a large amount of experimental data. Using the above method, more efficient and secure remote replication can be achieved in the following ways:

[0135] Intelligent scheduling optimization: Through trained deep learning models, changes in network bandwidth and laboratory workload are predicted to achieve intelligent avoidance of data replication and laboratory work peak periods, ensuring rapid synchronization of scientific research data without affecting daily scientific research activities in the laboratory.

[0136] Event-driven replication strategy: When a laboratory encounters an unexpected power outage or hardware failure, the solution automatically identifies this emergency event and initiates an emergency replication strategy, prioritizing the replication of critical scientific research data to other secure slave clusters to prevent data loss and accelerate the recovery process.

[0137] Cross-cloud replication management: Data involved in scientific research projects may be stored on multiple cloud platforms. Through intelligent management platform scheduling, efficient data replication between different cloud service providers can be achieved. At the same time, the optimal storage location is automatically selected based on the price and performance of the cloud service, reducing costs and improving research efficiency.

[0138] Optionally, in this embodiment, the current adaptive optimization scheme mainly relies on preset rules and thresholds. In the future, a deep learning model can be introduced to train and learn historical system load and network bandwidth change trends to predict future system status, achieve more accurate dynamic adjustment of replication speed, and improve overall efficiency and resource utilization.

[0139] -In addition to considering the single system load and network bandwidth, the algorithm can also incorporate more objective functions, such as energy consumption, latency, cost, etc. Through multi-objective optimization technology, the total utility under different strategies is comprehensively evaluated to find the optimal balance point.

[0140] Optionally, you can develop replication strategies based on specific events. For example, when a system-level alarm is detected (such as an earthquake warning, fire alarm, etc.), it will automatically switch to emergency replication mode to prioritize the protection of critical data and enhance disaster response capabilities.

[0141] Optionally, users are allowed to customize replication priorities based on the importance or sensitivity of files. For certain critical files, replication can be forced even during business peak hours to ensure data security and compliance.

[0142] Optionally, it supports incremental replication, which only transfers data that has changed since the last successful replication. At the same time, it combines version control mechanisms to ensure that any historical version can be tracked and restored during the data synchronization process, which is crucial for data auditing and compliance.

[0143] When performing replication across different geographical regions, develop more robust fault-tolerant mechanisms, such as multi-path data transmission, automatic reconnection after network interruption, and automatic retry of failed replication tasks, to improve the success rate and stability of remote replication under harsh network conditions.

[0144] The embodiments provided by this application allow users to set independent timing strategies for different remote replication tasks instead of using a unified replication time. This can greatly improve the flexibility of replication operations and meet the needs of different application scenarios. By customizing the replication time according to business needs, large amounts of data transmission can be avoided during network peak hours or system busy hours, thereby optimizing the utilization of network bandwidth and system resources. Setting a separate timing strategy can ensure that in the event of a failure or data loss, the replication operation of critical data can be carried out according to the scheduled time, thereby speeding up the speed of fault recovery. By finely controlling the timing of the replication operation, potential security risks can be reduced, such as preventing data transmission during sensitive periods or pausing the replication operation during system maintenance. Different replication speeds and frequencies can be set for different data volumes and network conditions to optimize the performance of the replication operation.

[0145] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0146] This embodiment also provides a remote data replication device for implementing the aforementioned embodiments and preferred implementations. Details already described will not be repeated. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. While the devices described in the following embodiments are preferably implemented using software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0147] Figure 4 is a structural block diagram of a remote data replication device according to an embodiment of the present application. Figure 4 As shown, the device includes:

[0148] An acquiring unit 402 is configured to acquire replication time information and replication speed information corresponding to target data in response to a remote replication request, wherein the remote replication request is used to request remote replication of the target data from the primary cluster to at least one secondary cluster, and the remote replication request carries the replication time information and the replication speed information;

[0149] The replication unit 404 is configured to remotely replicate the target data from the master cluster to at least one slave cluster within the execution time range indicated by the replication time information and according to the replication speed range indicated by the replication speed information.

[0150] As an optional solution, the copying unit 404 includes:

[0151] a replication module, configured to remotely replicate target data from a master cluster to at least one slave cluster at a first replication speed within an execution time range, wherein the first replication speed is a maximum replication speed of a replication speed range;

[0152] The first adjustment module is used to reduce the first replication speed to a second replication speed when a blocking event is detected during the remote replication of target data at a first replication speed, and remotely replicate the target data from the master cluster to at least one slave cluster at the second replication speed, wherein the second replication speed is the minimum replication speed of the replication speed range.

[0153] As an optional solution, the device further includes:

[0154] A second adjustment module is used to increase the second replication speed to a third replication speed after remotely replicating the target data from the master cluster to at least one slave cluster at the second replication speed. The target data is remotely replicated from the master cluster to at least one slave cluster at the third replication speed when a blocking event disappears and no other blocking events occur within a first time period after the blocking event disappears. The third replication speed is obtained by increasing a preset speed every N seconds on the basis of the second replication speed. The third replication speed is less than or equal to the first replication speed, and N is a positive integer.

[0155] As an optional solution, the device further includes:

[0156] A third adjustment module is used to, after remotely copying the target data from the master cluster to at least one slave cluster at a third replication speed, reduce the third replication speed to a fourth replication speed when a blocking event is detected during the remote replication of the target data at the third replication speed until the blocking event disappears, and remotely copy the target data from the master cluster to at least one slave cluster at the fourth replication speed, wherein the fourth replication speed is obtained by reducing the preset speed every N seconds on the basis of the third replication speed, and the fourth replication speed is greater than or equal to the second replication speed.

[0157] As an optional solution, the device further includes:

[0158] A first acquisition module is configured to acquire a service bandwidth and a number of input and output operations of at least one slave cluster in a second time period before increasing the second replication speed to a third replication speed;

[0159] a first determining module, configured to, before increasing the second replication speed to a third replication speed, determine that at least one slave cluster meets an expected service condition when the service bandwidth is less than a preset bandwidth threshold and the number of input and output operations is less than a preset number;

[0160] a second determining module, configured to determine the first preset threshold as the preset threshold before increasing the second replication speed to a third replication speed, if at least one slave cluster meets the expected service condition;

[0161] The third determination module is configured to determine, before increasing the second replication speed to the third replication speed, if at least one slave cluster does not meet the expected business conditions, the second preset threshold as the preset threshold, wherein the first preset threshold is greater than the second preset threshold.

[0162] As an optional solution, the device further includes:

[0163] a second acquisition module, configured to acquire, in a process of increasing the second replication speed to a third replication speed, a first number of data replicas in a data recovery state and a second number of data replicas in a data backfill state from at least one of the clusters;

[0164] The fourth adjustment module is configured to adjust the preset threshold value when the first quantity and the second quantity meet the expected quantity condition during the process of increasing the second copy speed to the third copy speed.

[0165] As an optional solution, the device further includes:

[0166] a creation module, configured to create a remote replication request before obtaining replication time information and replication speed information corresponding to target data in response to the remote replication request, wherein the remote replication request is used to request remote replication of the target data from the master cluster to a first slave cluster and a second slave cluster, wherein the at least one slave cluster includes the first slave cluster and the second slave cluster;

[0167] a first configuration module configured to configure first replication time information and first replication speed information corresponding to the target data for the remote replication request before obtaining replication time information and replication speed information corresponding to the target data in response to the remote replication request, wherein the first replication time information is used to indicate a first execution time range for remote replication of the target data from the master cluster to the first slave cluster, and the first replication speed information is used to indicate a first replication speed range for remote replication of the target data from the master cluster to the first slave cluster;

[0168] a second configuration module configured to configure, for the remote replication request, second replication time information and second replication speed information corresponding to the target data before obtaining replication time information and replication speed information corresponding to the target data in response to the remote replication request, the second replication time information being used to indicate a second execution time range for remote replication of the target data from the primary cluster to the second slave cluster, and the second replication speed information being used to indicate a second replication speed range for remote replication of the target data from the primary cluster to the second slave cluster;

[0169] Among them, the copy time information includes the first copy time information and the second copy time information, the copy speed information includes the first copy speed information and the second copy speed information, the execution time range includes the first execution time range and the second execution time range, and the copy speed range includes the first copy speed range and the second copy speed range.

[0170] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.

[0171] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0172] It should be noted that the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0173] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any of the above method embodiments when run.

[0174] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0175] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0176] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0177] An embodiment of the present application further provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores the computer program product, and when the computer program is executed by a processor, the steps of the method in each embodiment of the present application are implemented.

[0178] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.

[0179] Obviously, those skilled in the art should understand that the modules or steps of the present application described above can be implemented using a general-purpose computing device, they can be concentrated on a single computing device, or distributed across a network composed of multiple computing devices, they can be implemented using program code executable by the computing device, and thus, they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be performed in a different order than herein, or they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0180] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various modifications and variations of the present application are possible. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A remote data replication method, characterized in that: include: In response to a remote replication request, obtaining replication time information and replication speed information corresponding to target data, wherein the remote replication request is used to request remote replication of the target data from the primary cluster to at least one secondary cluster, and the remote replication request carries the replication time information and the replication speed information; The target data is remotely replicated from the master cluster to the at least one slave cluster within the execution time range indicated by the replication time information and according to the replication speed range indicated by the replication speed information.

2. The method according to claim 1, characterized in that The remotely copying the target data from the master cluster to the at least one slave cluster within the execution time range indicated by the copy time information and according to the copy speed range indicated by the copy speed information includes: remotely copying the target data from the master cluster to the at least one slave cluster at a first copy speed within the execution time range, wherein the first copy speed is a maximum copy speed of the copy speed range; During the remote replication of the target data at the first replication speed, if a blocking event is detected, the first replication speed is reduced to a second replication speed, and the target data is remotely replicated from the master cluster to the at least one slave cluster at the second replication speed, wherein the second replication speed is the minimum replication speed of the replication speed range.

3. The method according to claim 2, characterized in that After remotely replicating the target data from the master cluster to the at least one slave cluster at the second replication speed, the method further includes: During the remote replication of the target data at the second replication speed, if the blocking event disappears and no other blocking events occur within a first time period after the blocking event disappears, the second replication speed is increased to a third replication speed, and the target data is remotely replicated from the master cluster to the at least one slave cluster at the third replication speed, wherein the third replication speed is a replication speed obtained by increasing a preset speed every N seconds on the basis of the second replication speed, and the third replication speed is less than or equal to the first replication speed, and N is a positive integer.

4. The method according to claim 3, characterized in that After remotely replicating the target data from the master cluster to the at least one slave cluster at the third replication speed, the method further includes: During the remote replication of the target data at the third replication speed, when the blocking event is detected, the third replication speed is reduced to a fourth replication speed until the blocking event disappears, and the target data is remotely replicated from the master cluster to the at least one slave cluster at the fourth replication speed, wherein the fourth replication speed is obtained by reducing the preset speed every N seconds on the basis of the third replication speed, and the fourth replication speed is greater than or equal to the second replication speed.

5. The method according to claim 3, characterized in that Before increasing the second copying speed to a third copying speed, the method further includes: Obtaining a service bandwidth and a number of input and output operations of the at least one slave cluster in a second time period; When the service bandwidth is less than a preset bandwidth threshold and the number of input and output operations is less than a preset number, determining that the at least one slave cluster meets the expected service condition; In a case where the at least one slave cluster meets the expected service condition, determining a first preset threshold as the preset threshold; In a case where the at least one slave cluster does not meet the expected service condition, a second preset threshold is determined as the preset threshold, wherein the first preset threshold is greater than the second preset threshold.

6. The method according to claim 3, characterized in that In the process of increasing the second copying speed to a third copying speed, the method further includes: Obtaining a first number of data replicas in a data recovery state and a second number of data replicas in a data backfill state from the at least one slave cluster; When the first quantity and the second quantity meet the expected quantity condition, the preset threshold is adjusted.

7. The method according to any one of claims 1 to 6, characterized in that Before obtaining the copy time information and copy speed information corresponding to the target data in response to the remote copy request, the method further includes: Creating the remote replication request, wherein the remote replication request is used to request remote replication of the target data from the master cluster to a first slave cluster and a second slave cluster, wherein the at least one slave cluster includes the first slave cluster and the second slave cluster; Configuring first replication time information and first replication speed information corresponding to the target data for the remote replication request, wherein the first replication time information is used to indicate a first execution time range for remotely replicating the target data from the primary cluster to the first slave cluster, and the first replication speed information is used to indicate a first replication speed range for remotely replicating the target data from the primary cluster to the first slave cluster; configuring second replication time information and second replication speed information corresponding to the target data for the remote replication request, wherein the second replication time information is used to indicate a second execution time range for remotely replicating the target data from the primary cluster to the second slave cluster, and the second replication speed information is used to indicate a second replication speed range for remotely replicating the target data from the primary cluster to the second slave cluster; Among them, the copy time information includes the first copy time information and the second copy time information, the copy speed information includes the first copy speed information and the second copy speed information, the execution time range includes the first execution time range and the second execution time range, and the copy speed range includes the first copy speed range and the second copy speed range.

8. A remote data replication device, characterized in that: include: an acquiring unit, configured to acquire, in response to a remote replication request, replication time information and replication speed information corresponding to target data, wherein the remote replication request is used to request remote replication of the target data from the primary cluster to at least one secondary cluster, and the remote replication request carries the replication time information and the replication speed information; The replication unit is configured to remotely replicate target data from the master cluster to at least one slave cluster within an execution time range indicated by the replication time information and according to a replication speed range indicated by the replication speed information.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method according to any one of claims 1 to 7 when executed by a processor.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images