Method, electronic device and computer program product for data synchronization

Abstract

The embodiment of the invention relates to a method for data synchronization, electronic equipment and a computer program product. The method includes determining an environmental state of a synchronization environment by monitoring environmental metrics for the synchronization environment. The method further includes selecting a synchronization operation for target data to be synchronized from a synchronization decision database based on the determined environmental state. The method further includes initiating a synchronization operation between the first storage device and the second storage device of the synchronization environment. The method also includes updating the synchronization decision database in response to the synchronization operation being performed. In this way, appropriate synchronization operation can be dynamically selected for the synchronization environment under the influence of multiple factors according to state monitoring for the synchronization environment. Besides, after the synchronization operation is initiated between the home terminal and the opposite terminal, synchronization decision knowledge is enriched by utilizing synchronization feedback, and subsequent selection is promoted to adapt to the change of a synchronization environment, so that the synchronization reliability is improved, and the efficiency is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to the field of storage, and more particularly to methods, electronic devices, and computer program products for data synchronization. Background Art

[0002] Data synchronization is an important technology for sharing data between different locations and systems while maintaining data consistency and integrity. With the development of storage technology and network communication, the application scenarios and business requirements of data synchronization are constantly expanding and increasing. It is necessary to adopt effective data synchronization strategies to ensure that the data to be synchronized can be reliably copied from the local end to the remote end, and at the same time, it is expected that the synchronization process meets strict timeliness requirements and security requirements.

[0003] Today, data synchronization has been widely applied to various application scenarios and business requirements. For protection purposes, replication and backup technologies are used to synchronize data from, for example, a production site to a remote site to prevent data destruction or loss due to natural disasters or accidents (such as power outages). For storage purposes, compression and deduplication technologies are used to effectively utilize space. For application purposes, data is transmitted between edge nodes and a central cloud. Summary of the Invention

[0004] Embodiments of the present disclosure provide a solution for data synchronization. Through this intelligent synchronization solution, an appropriate synchronization operation can be selected for a changing synchronization environment, and synchronization feedback can be iteratively utilized to enrich synchronization decision-making knowledge, thereby improving the reliability of synchronization and enhancing efficiency.

[0005] In a first aspect of the present disclosure, a method for data synchronization is provided. The method includes determining an environmental state of a synchronization environment by monitoring environmental metrics for the synchronization environment. The method further includes selecting, based on the determined environmental state, a synchronization operation for target data to be synchronized from a synchronization decision database. The method further includes initiating a synchronization operation between a first storage device and a second storage device in the synchronization environment. The method further includes updating the synchronization decision database in response to the synchronization operation being executed.

[0006] In another aspect of the present disclosure, an electronic device for data synchronization is provided. The electronic device includes a processor and a memory coupled to the processor. Instructions are stored on the memory, and when executed by the processor, these instructions cause the electronic device to perform actions, including determining the environmental state of the synchronization environment by monitoring environmental metrics for the synchronization environment. The actions further include selecting a synchronization operation for target data to be synchronized from a synchronization decision database based on the determined environmental state. The actions further include initiating a synchronization operation between a first storage device and a second storage device in the synchronization environment. The actions further include updating the synchronization decision database in response to the synchronization operation being executed.

[0007] In yet another aspect of the present disclosure, a computer program product is provided. The computer program product is tangibly stored on a non-transitory computer-readable storage medium and includes computer-executable instructions that, when executed by a processor of a computer, cause the computer to perform a method or process according to an embodiment of the present disclosure.

[0008] The solution for data synchronization according to an embodiment of the present disclosure can dynamically select an appropriate synchronization operation for a synchronization environment affected by multiple factors based on the state monitoring of the synchronization environment. At the same time, after initiating a synchronization operation between the local end and the peer end, the synchronization feedback is utilized to enrich the synchronization decision knowledge, promoting subsequent selection to adapt to changes in the synchronization environment, thereby improving the reliability of synchronization and enhancing the efficiency.

[0009] Note that the Summary of the Invention section is provided to introduce a series of concepts in a simplified form, which will be further described in the Detailed Description below. The Summary of the Invention section is not intended to identify the key features or essential features of the present disclosure, nor is it intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] By describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings, the above and other objects, features, and advantages of the present disclosure will become more clearly understood. In the drawings:

[0011] Figure 1 A schematic diagram illustrating an example environment in which a method and / or process according to an embodiment of the present disclosure may be implemented;

[0012] Figure 2 A flowchart illustrating a method for data synchronization according to an embodiment of the present disclosure;

[0013] Figure 3 A schematic diagram illustrating a data synchronization process according to an embodiment of the present disclosure;

[0014] Figure 4A schematic diagram showing an example of environmental metrics for a synchronous environment 310 according to an embodiment of the present disclosure;

[0015] Figure 5 A schematic diagram showing an example configuration of a synchronizer according to an embodiment of the present disclosure;

[0016] Figure 6 A diagram showing a data synchronization workflow according to an embodiment of the present disclosure; and

[0017] Figure 7 It is a schematic block diagram of an example device that can be used to implement an embodiment of the present disclosure.

[0018] In all the figures, the same or similar reference numerals generally denote the same or similar elements. Specific embodiments

[0019] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0020] In the description of the embodiments of the present disclosure, the term "comprising" and its variations should be understood as open-ended inclusion, that is, "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects, unless clearly indicated otherwise.

[0021] As described above, with the increasingly expanding and increasing application scenarios and business requirements of data synchronization, a solution is desired that can ensure the reliable synchronization of data to be synchronized between one end and the other end (for example, the production end of the data and a remote end different from the production end (such as the consumption end, cloud, etc.)), while also meeting strict timeliness requirements and security requirements. Such a solution may involve appropriate replication and backup techniques, as well as compression and deduplication techniques, etc. to meet actual usage needs.

[0022] The choice of compression and deduplication strategies is related to the performance and efficiency of data synchronization. A related synchronization scheme aims to decompress the data to be synchronized at the local end and send it to the peer end, and the peer end compresses it after receiving and writes it to the disk. The situation of deduplication is similar. This is a relatively complex implementation, which increases the difficulty of synchronization due to the compression and decompression operations at each end, and at the same time, transmitting the decompressed data wastes bandwidth resources. Another scheme directly synchronizes the compressed and / or deduplicated data from the local end to the peer end, but its synchronization result may not be satisfactory. For example, the reliability may deteriorate.

[0023] To address at least some of the above and other potential problems, embodiments of the present disclosure propose a solution for data synchronization. The solution includes determining the environmental state of the synchronization environment by monitoring environmental metrics for the synchronization environment. The solution also includes selecting a synchronization operation for the target data to be synchronized from a synchronization decision database based on the determined environmental state. The solution also includes initiating a synchronization operation between a first storage device and a second storage device in the synchronization environment. The solution also includes updating the synchronization decision database in response to the synchronization operation being executed. In this way, an appropriate synchronization operation can be dynamically selected for the synchronization environment affected by multiple factors according to the status monitoring of the synchronization environment. At the same time, after initiating the synchronization operation between the local end and the peer end, the synchronization feedback is used to enrich the synchronization decision knowledge, promote the subsequent selection to adapt to the changes in the synchronization environment, thereby improving the reliability of synchronization and enhancing the efficiency.

[0024] The following refers to Figures 1 to 7 to illustrate the basic principles and several exemplary implementations of the present disclosure. It should be understood that these exemplary embodiments are given only to enable those skilled in the art to better understand and then implement the embodiments of the present disclosure, and do not limit the scope of the present disclosure in any way.

[0025] Figure 1 FIG. shows a schematic diagram of an exemplary environment 100 in which a method and / or process according to an embodiment of the present disclosure can be implemented. The exemplary environment 100 exemplarily shows an environment in which the target data to be synchronized is synchronized. As Figure 1 shown, the exemplary environment 100 may include a client device 110, a first storage device 120, and a second storage device 130. It should be understood that only a limited number of devices and systems are shown here for the purpose of easy understanding and illustration, but the embodiments of the present disclosure are not limited thereto, and may also include other components. For example, the exemplary environment 100 may further include one or more monitoring units (not shown), which may be configured to monitor the exemplary environment 100, such as monitoring the network communication status in the exemplary environment 100, etc.

[0026] According to an embodiment of the present disclosure, the client device 110 may initiate a synchronization operation for data synchronization between the first storage device 120 and the second storage device 130 (e.g., a synchronization operation from the first storage device 120 to the second storage device 130, or a synchronization operation from the second storage device 130 to the first storage device 120). In other words, the data synchronization between the first storage device 120 and the second storage device 130 may be bidirectional. The client device 110 may include the above-mentioned monitoring unit for monitoring the environment in which the synchronization operation is performed on the data to be synchronized, such as but not limited to whether the initiated synchronization operation ends, the network communication status between the client device 110 and the first storage device 120, the network communication status between the client device 110 and the second storage device 130, the network communication status between the first storage device 120 and the second storage device 130, the characteristics of the data to be synchronized, the configuration of the synchronization process, and other environmental metrics that may affect data synchronization.

[0027] The client device 110 may have computing capabilities corresponding to the data synchronization according to an embodiment of the present disclosure for performing corresponding operations or steps of the method thereon. In some embodiments, the client device 110 may be coupled with the first storage device 120 to form a single system. Similarly, the client device 110 may also be coupled with the second storage device 130 or a combination of the first storage device 120 and the second storage device 130 to form a corresponding storage system.

[0028] The client device 110 may include, but is not limited to, a personal computer, a laptop computer, a server computer, a mobile device (such as a smart phone, a tablet computer, etc.), a wearable electronic device, a multimedia player, a personal digital assistant (PDA), a smart home device, a consumer electronic product, or a distributed computing environment including any one or more of the above devices. It should be understood that the client device 110 is Figure 1 schematically shown as one client device for the purpose of easy illustration and understanding, but in the example environment 100, more client devices may be arranged according to actual needs.

[0029] According to an embodiment of the present disclosure, the first storage device 120 and the second storage device 130 may be communicatively coupled (e.g., via a line or a network) to implement the data synchronization according to an embodiment of the present disclosure. The first storage device 120 and the second storage device 130 may include a cluster of storage devices such as disk arrays. As Figure 1As shown, storage device 120 includes a plurality of disks 120-1, 120-2, 120-3, etc., and storage device 130 includes a plurality of disks 130-1, 130-2, 130-3, etc. Storage devices 120 and 130 may be geographically separated from each other, and the requested storage space may be provided by means of management software of a distributed storage system (e.g., installed on client device 110).

[0030] It should be understood that Figure 1 the storage devices 120 and 130 illustrated in and the disks included therein are merely exemplary, and the scope of the present disclosure is not limited in this regard. For example, the exemplary environment 100 may include more or fewer storage devices, and each storage device may include more or fewer disks. By way of example and not limitation, examples of disks may include hard disk drives (HDDs), solid state drives (SSDs), solid state hybrid drives (SSHDs), etc., and are coupled together, for example, via a line or a network, etc. It should be understood that an appropriate type of disk may be employed according to specific usage requirements.

[0031] The above has been described in connection with Figure 1 the exemplary environment 100 in which the methods and / or processes according to embodiments of the present disclosure may be implemented. The flowchart of method 200 for data synchronization according to embodiments of the present disclosure will be described below in connection with Figure 2 By means of this method 200, it is possible to capture changes in the monitored synchronization environment under the influence of multiple factors and thus select an appropriate synchronization operation therefor, and iteratively enrich the synchronization decision-making knowledge with synchronization feedback in order to better adapt to changes in the synchronization environment.

[0032] At block 210, the environmental state of the synchronization environment is determined by monitoring environmental metrics for the synchronization environment. According to embodiments of the present disclosure, the synchronization environment (such as Figure 1 the exemplary environment 100 illustrated in) may exhibit different states during the synchronization process under the influence of multiple factors (such as latency, etc.). By monitoring the environmental metrics for the synchronization environment, it is possible to better understand the environmental state of the synchronization environment, thereby adjusting the synchronization strategy to be implemented. Hereinafter, the environmental metric monitoring and environmental state identification according to embodiments of the present disclosure will be further described in detail.

[0033] At block 220, based on the determined environmental state, a synchronization operation for the target data to be synchronized is selected from the synchronization decision database. According to an embodiment of the present disclosure, based on a deep understanding of the environmental state, the associated synchronization decision knowledge in the synchronization decision database facilitates the selection of a synchronization operation corresponding to the determined environmental state. This decision-making process is dynamic and can make corresponding adjustments according to changes in the environment. In some embodiments, the synchronization decision database may include a large amount of historical data and empirical data, which is of reference value for selecting a suitable synchronization operation. Hereinafter, the synchronization decision database according to an embodiment of the present disclosure will be described in further detail.

[0034] At block 230, a synchronization operation is initiated between a first storage device and a second storage device in the synchronization environment. According to an embodiment of the present disclosure, an appropriate synchronization operation corresponding to the determined environmental state is initiated between the first storage device and the second storage device, so that the performance of data synchronization is improved. For example, bandwidth resources are saved while ensuring data consistency and integrity.

[0035] At block 240, in response to the synchronization operation being executed, the synchronization decision database is updated. After the synchronization operation adapted to the synchronization environment is executed, the performance of data synchronization will be evaluated, for example, based on the timeliness of synchronization, or the integrity and consistency of the data after being synchronized. If the evaluated performance is good, the same or similar synchronization operation can be selected the next time a similar environmental state appears to achieve the desired synchronization performance; if the evaluated performance is not good, the synchronization operation or such different operations will be avoided the next time a similar environmental state appears. Next, the determined synchronization state and the synchronization performance evaluated for the selected synchronization operation will be added to the synchronization decision database as supplementary synchronization decision knowledge for subsequent synchronization decisions. In this way, the synchronization decision knowledge is iteratively enriched using synchronization feedback. The selection of synchronization operations will become more and more intelligent and more and more adaptable to various complex environmental changes.

[0036] The method 200 for data synchronization according to an embodiment of the present disclosure can dynamically select an appropriate synchronization operation for a synchronization environment affected by multiple factors based on the state monitoring of the synchronization environment. At the same time, after initiating a synchronization operation between the local end and the peer end, the synchronization feedback is used to enrich the synchronization decision knowledge, which promotes the subsequent selection to adapt to the changes in the synchronization environment, thereby improving the reliability of synchronization and the efficiency.

[0037] Figure 3 is a schematic diagram illustrating a data synchronization process 300 according to an embodiment of the present disclosure. As Figure 3 shown, the synchronization environment 310, the synchronizer 320, and the synchronization decision database 330 can interact with each other. As Figure 3The synchronizer 320 shown in [Figure 0], and the subprocesses for the data synchronization process 300 indicated by the respective arrows, may be implemented based on software and may run on a device with computing capabilities, such as the client device 110. The synchronization database 330 may be deployed to be accessible by the synchronizer 320. Additionally, the environmental state 301, synchronization feedback 302, and synchronization operations 303 may be cached for use after being transmitted and received.

[0038] As Figure 3 shown in [Figure 0], the synchronizer 320 may be configured to determine the environmental state 301 of the synchronization environment 310. According to an embodiment of the present disclosure, the synchronizer 320 may capture the measured values of the monitored environmental metrics for the synchronization environment 310 at each of at least one moment. The measured values of these metrics change due to changes in the environment, so it is necessary to capture the measured values of the environmental metrics at a frequency set depending on the monitoring accuracy to understand the environmental state 301 of the synchronization environment 310.

[0039] According to an embodiment of the present disclosure, the environmental metrics for the synchronization environment 310 include one or more of the following: array configuration, system state, network state, transmitted data size, data size before reduction, compression ratio, and data block size, where the array configuration includes the number of volumes, the number of asynchronous replication sessions, and the asynchronous replication recovery point objective (RPO), the system state includes the central processing unit (CPU) utilization, and the network state includes write latency and input / output operations per second (IOPS). Hereinafter, an exemplary description of the environmental metrics for the synchronization environment 310 will be provided in conjunction with Figure 4 an example 400 of the environmental metrics for the synchronization environment 310 according to an embodiment of the present disclosure.

[0040] Figure 4 A schematic diagram illustrating an example 400 of the environmental metrics for the synchronization environment 310 according to an embodiment of the present disclosure is shown. It should be understood that the measured values of the monitored environmental metrics are presented in the form of structured data here, but this is exemplary and not restrictive, and there may be other forms for collecting and organizing them. Additionally, Figure 4 only a limited number of environmental metrics for the synchronization environment 310 are shown in [Figure 0], which is only for the purpose of facilitating understanding and easy illustration, and more or fewer environmental metrics may be selected according to the monitoring accuracy.

[0041] As Figure 4As shown, the first to third columns of Example 400 respectively show the number of replication source volumes, write latency, and input / output operations per second monitored at multiple times. The multiple times described herein may be spaced at the same time interval (e.g., the time interval corresponding to the RPO), that is, the environmental metrics for the synchronous environment 310 are monitored at a predetermined frequency. The fourth to sixth columns of Example 400 respectively show "TRIF-288", the compression ratio, and the synchronization duration monitored at multiple times, where "TRIF-288" indicates the compression and deduplication strategy of the synchronization process, which is also referred to as the compression transfer switch hereinafter. The seventh to ninth columns of Example 400 respectively show the transferred data size, unreduced_data_size (the size of the data before reduction), and the synchronization rate, where unreduced_data_size is also the size of the data to be synchronized before being compressed. In addition, the tenth to twelfth columns of Example 400 respectively show the input / output size (i.e., the size of the data block), io_workload_cpu_utilization(source), and io_workload_cpu_utilization(destination), where io_workload_cpu_utilization(source) and io_workload_cpu_utilization(destination) respectively indicate the CPU utilization of the local end and the peer end.

[0042] Referring back Figure 3 , according to an embodiment of the present disclosure, the synchronizer 320 may assign weights to each of at least one time, and the sum of the assigned weights is equal to 1. In some application scenarios, the importance of different times is different, and more attention is preferably given to more important times, where the sum of the weight values assigned to each time is 1. In addition, the synchronizer 320 may determine the environmental state 301 of the synchronous environment 310 for a time period including the at least one time by weighted averaging the metric values of the environmental metrics captured at each time based on the assigned weights. In this way, the weighted average of the metric values of the environmental metrics monitored at multiple times can be used to characterize the environmental state 301 of the synchronous environment 310 for this time period (hereinafter, also referred to as an episode).

[0043] By way of example and not limitation, an example of using the metric values of the environmental metrics of the synchronous environment 110 at the current time t, the time t - 1 / 2 of the time period corresponding to the RPO before the time t RPO / 2 , and the time t - 1 of the time period corresponding to the RPO before the time t RPO to characterize the environmental state 301 of the synchronous environment 110 is shown in the following formula (1):

[0044] I t = ω now ·I now + ω RPO / 2 ·I RPO / 2 + ω RPO ·I RPO (1)

[0045] Where I t indicates the environmental state of the synchronization environment 110 for the period up to time t, ω now is the weight for the current moment, I now is the measured value of the environmental metric monitored at the current moment, ω RPO / 2 is the weight for time t RPO / 2 I RPO / 2 is the measured value of the environmental metric monitored at time t RPO / 2 ω RPO is the weight for time t RPO I RPO is the measured value of the environmental metric monitored at time t RPO where ω now + ω RPO / 2 + ω RPO = 1, for example, the values of ω now , ω RPO / 2 ω RPO are 0.5, 0.3, and 0.2 respectively.

[0046] According to an embodiment of the present disclosure, for example, the synchronization operation 303 between the first storage device 120 and the second storage device 130 may include a first synchronization operation and a second synchronization operation. The first synchronization operation may include decompressing the compressed target data (the data to be synchronized between the first storage device 120 and the second storage device 130) at one of the first storage device 120 and the second storage device 130, and then transmitting the decompressed target data to the other of the first storage device 120 and the second storage device 130, and in response to receiving the decompressed target data, compressing and storing the decompressed target data at the other of the first storage device 120 and the second storage device 130. In other words, the method of decompressing first and then transmitting. In addition, the second synchronization operation may include transmitting and storing the compressed target data from one of the first storage device and the second storage device to the other. In other words, the method of transmitting directly without decompression. It should be understood that, for the purpose of easy understanding, only the first synchronization operation and the second synchronization operation are given as examples, but the embodiments of the present disclosure are not limited thereto. For example, a combination of the first synchronization operation and the second synchronization operation, as well as other different operations, may also be adopted.

[0047] As Figure 3As shown in the figure, in response to the synchronizer 320 determining the environmental state 301 of the target period, the synchronizer 320 may obtain associated synchronization decision knowledge from the synchronization decision database 330 for making a decision. According to an embodiment of the present disclosure, the synchronization decision database 330 includes a plurality of synchronization reference records, and each of these synchronization reference records includes a reference environmental state of the synchronization environment 310, and a first synchronization operation score and a second synchronization operation score corresponding to the reference environmental state, where the first synchronization operation score indicates the duration required to synchronize the target data by performing the first synchronization operation in the reference environmental state, and the second synchronization operation score indicates the duration required to synchronize the target data by performing the second synchronization operation in the reference environmental state. It should be understood that the synchronization operation score indicates the performance of performing the corresponding synchronization operation in the synchronization state 301, and is not limited to the synchronization duration. For example, the integrity and reliability of the target data after synchronization may also be considered. An example of the synchronization reference record is given in Table 1 below.

[0048] Table 1

[0049]

[0050]

[0051] The synchronization reference records are presented in the form of structured data here, but this is exemplary and not restrictive. There may also be other forms for collecting and organizing them. In Table 1, a row of the table serves as an entry indicating a synchronization reference record, which includes a reference environmental state (which can be a vector or can be reduced to a single value), a first synchronization operation score, a second synchronization operation score, etc. It should be understood that only a limited number of synchronization reference records are shown here, which is only for the purpose of easy understanding. The synchronization decision database 330 may also include more or fewer synchronization reference records. In the following, the process of selecting a synchronization operation 303 for the target data to be synchronized from the synchronization decision database 330 according to an embodiment of the present disclosure will be further described in detail.

[0052] According to an embodiment of the present disclosure, after determining the environmental state 301 of the synchronization environment 310, the synchronizer 330 may compare the determined environmental state 301 with the reference environmental state included in each synchronization reference record (for example, the first column in Table 1) to obtain the similarity between the environmental state and the reference environmental state, and then may identify the reference environmental state having the maximum similarity with the environmental state 301 as the target reference environmental state, and may determine the first target synchronization operation score and the second target synchronization operation score corresponding to the target reference environmental state.

[0053] By way of example and not limitation, if the determined environmental state 301 is identical to the reference environmental state S in the first synchronous reference record t The most similar (with the greatest similarity), the reference environment state S t Identify it as the target reference environment state and determine the difference with the reference environment state S t The corresponding first operation score Q t and the second operation score Q' t It should be understood that, in addition to determining the reference environmental state most similar to the determined environmental state 301 , one or more reference environmental states whose similarities are within a predetermined threshold range may also be selected, and the scope of the present disclosure is not limited in this respect.

[0054] According to an embodiment of the present disclosure, a first synchronization operation can be selected in response to a first target synchronization operation score corresponding to the target reference environment state being greater than a second synchronization operation score corresponding to the target reference environment state, and a second synchronization operation can be selected in response to a first target synchronization operation score corresponding to the target reference environment state being less than or equal to a second synchronization operation score corresponding to the target reference environment state.

[0055] The synchronization operation score in each synchronization reference record indicates the performance of synchronizing the target data when executing the synchronization operation corresponding to the synchronization operation score under the corresponding reference environment state, such as the time required for synchronization or the completeness of the target data after synchronization. As described above, the first synchronization operation score indicates the length of time required to execute the first synchronization operation to synchronize the target data under the reference environment state, and the second synchronization operation score indicates the length of time required to execute the second synchronization operation to synchronize the target data under the reference environment state.

[0056] According to an embodiment of the present disclosure, in response to the first target synchronization operation score corresponding to the target reference environment state being greater than the second synchronization operation score corresponding to the target reference environment state, the first synchronization operation may be selected. That is, under the determined environment state 301, the performance of executing the first synchronization operation is better than the second synchronization operation, and a decompression and retransmission method is adopted between the storage nodes. In addition, in response to the first target synchronization operation score corresponding to the target reference environment state being less than or equal to the second synchronization operation score corresponding to the target reference environment state, the second synchronization operation is selected. That is, under the determined environment state 301, the performance of executing the first synchronization operation is better than the second synchronization operation, and a non-decompression direct transmission method is adopted between the storage nodes. In the case of equal scores, transmitting compressed data by non-decompression direct transmission can save bandwidth resources to a certain extent.

[0057] In response to the determined synchronization operation being executed, the synchronization feedback 302 can be utilized to update the decision database 330, where the synchronization feedback 302 can perform measurements reflecting synchronization performance, such as the duration of the determined synchronization operation for synchronizing the target data, or the integrity and consistency of the target data after being synchronized. According to an embodiment of the present disclosure, based on the determined environmental state 301 and the duration consumed for synchronizing the target data by performing the selected synchronization operation, an additional synchronization reference record can be formed, and the formed additional synchronization reference record can be added to the synchronization decision database. In this way, the synchronization decision knowledge can be iteratively enriched using the synchronization feedback, thereby improving the reliability of synchronization and enhancing efficiency.

[0058] Figure 5 FIG. illustrates a schematic diagram of an exemplary configuration of a synchronizer 320 according to an embodiment of the present disclosure. As Figure 5 shown, the synchronizer 320 can include a capturer 510, a decision maker 520, and an initiator 530, which are coupled to each other for data synchronization according to an embodiment of the present disclosure. These units or subsystems can be implemented based on software and can run on a device with computing capabilities (such as the client device 110). It should be understood that only a limited number of units and subsystems are shown here for the purpose of facilitating understanding and easy illustration as an exemplary breakdown of the synchronizer 320, but the embodiments of the present disclosure are not limited thereto and may also include other components.

[0059] In some embodiments, the capturer 510 can be configured to capture the measured values of environmental metrics characterizing the environmental state 301 of the synchronization environment 310 at multiple times and perform data processing (such as normalization, dimensionality reduction, etc.) on the captured data. Based on the environmental state 301 from the capturer 510, the decision maker 520 can be configured to select a synchronization operation suitable for the environmental state 301 according to the synchronization decision knowledge in the synchronization decision database 330. In addition, the initiator 530 can be configured to initiate the selected synchronization operation among various storage nodes. The operations and steps of the capturer 510, the decision maker 520, and the initiator 530 can be as described above in conjunction with Figure 3 that described.

[0060] Figure 6 FIG. illustrates a diagram of a data synchronization workflow 600 according to an embodiment of the present disclosure. At 610, the entire workflow starts. At 620, the corresponding synchronization operation score (hereinafter also referred to as the Q value) of the synchronization operation is initialized. At 630, a period starts, and a period of a predetermined time period up to time t is described here as a non-limiting example. At 640, the state S t of this period to be used for synchronization is selected, and the action A t is selected. At 650, the action A is initiatedt and move to the next state S t+1 , and calculate the reward score R t for performing action A t in state S t (which may be positive or negative). At 650, update the Q-value of state S t , which indicates the performance of performing action A t in state S t . The following equation (2) exemplarily shows the calculation process: Q new (S t , A t ) <- Q(S t , A t ) + α[R t + γQ(S t+1 , A t+1 ) - Q(S t , A t )] (2)

[0061] The Q-value can indicate the possible predicted score for taking action A t+1 in state S t+1 in the next period plus the reward score R t+1 observed based on the execution of action A t+1 . In other words, the Q-value can be corrected based on the deviation observed from the execution of the predicted action, and such deviation may be positive feedback or negative feedback. For example, at 650, the Q-value of the previous period (state S t , action A t ) is updated using the reward score R t . In equation (2), α is the learning rate, which determines the degree to which newly acquired information covers old information, and γ is the reward factor, which indicates the importance of the reward used for compensation. Next, at 670, determine whether it is the final state. If it is yes at 670, end the entire workflow, and if it is no, return to 640 to continue the iteration.

[0062] Figure 7 illustrates a schematic block diagram of an example device 700 that can be used to implement some embodiments in accordance with the present disclosure. As Figure 7As shown in the figure, device 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 702 or computer program instructions loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The CPU 701, ROM 702, and RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0063] Multiple components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, an optical disc, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0064] Each of the processes and processes described above, such as method 200, can be executed by the processing unit 701. For example, in some embodiments, method 200 can be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the CPU 701, one or more actions of the method 200 described above can be executed.

[0065] The present disclosure can be a method, an apparatus, a system, and / or a computer program product. The computer program product can include a computer-readable storage medium having thereon computer-readable program instructions for performing various aspects of the present disclosure.

[0066] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example—but not limited to—an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punched card or raised structures in grooves having instructions stored thereon, and any suitable combination of the foregoing. The computer-readable storage medium as used herein is not construed as an instantaneous signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0067] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or can be downloaded to an external computer or an external storage device through a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include a copper transmission cable, an optical fiber transmission, a wireless transmission, a router, a firewall, a switch, a gateway computer, and / or an edge server. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0068] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine - related instructions, microcode, firmware instructions, state - setting data, or source code or object code written in any combination of one or more programming languages, including object - oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer - readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand - alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer - readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field - programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer - readable program instructions to implement various aspects of the present disclosure.

[0069] Aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer - readable program instructions.

[0070] These computer - readable program instructions can be provided to a processing unit of a general - purpose computer, a special - purpose computer, or other programmable data - processing apparatus to produce a machine such that, when the instructions are executed by the processing unit of the computer or other programmable data - processing apparatus, a device is created that implements the functions / acts specified in one or more blocks of the flowchart and / or block diagram. These computer - readable program instructions can also be stored in a computer - readable storage medium, which causes a computer, a programmable data - processing apparatus, and / or other devices to operate in a particular manner, so that the computer - readable medium storing the instructions includes a manufacture that includes instructions for implementing various aspects of the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0071] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operations or steps to be performed on the computer, other programmable data processing apparatus, or other devices to produce a computer-implemented process such that the instructions executed on the computer, other programmable data processing apparatus, or other devices implement the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0072] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two consecutive blocks may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or acts, or by a combination of dedicated hardware and computer instructions.

[0073] The various embodiments of the present disclosure have been described above. The above description is exemplary and exhaustive and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles of the embodiments, the practical application, or the improvement of the technology in the market, or to enable other ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for data synchronization, comprising: Determining an environmental state of the synchronization environment by monitoring environmental metrics for the synchronization environment; Selecting a synchronization operation for target data to be synchronized from a synchronization decision database based on the determined environmental state; Initiating the synchronization operation between a first storage device and a second storage device in the synchronization environment; And Updating the synchronization decision database in response to the synchronization operation being executed.

2. The method according to claim 1, wherein determining the environmental state of the synchronization environment comprises: Capturing a metric value of the environmental metric at each of at least one moment; Assigning a weight to each of the at least one moment, the sum of the assigned weights being equal to 1; And Determining the environmental state for a time period including the at least one moment by weighted averaging the metric values captured at each moment based on the assigned weights.

3. The method according to claim 2, wherein the environmental metric comprises one or more of the following: array configuration, system state, network state, transferred data size, data size before reduction, compression ratio, and data block size; and wherein the array configuration comprises the number of volumes, the number of asynchronous replication sessions, and the asynchronous replication recovery point objective RPO; the system state comprises the central processing unit CPU utilization; and the network state comprises write latency and input / output operations per second IOPS.

4. The method according to claim 1, wherein the synchronization operation comprises a first synchronization operation and a second synchronization operation, The first synchronization operation comprises: Decompressing the compressed target data at one of the first storage device and the second storage device; Transmitting the decompressed target data to the other of the first storage device and the second storage device; And In response to receiving the decompressed target data, compressing and storing the decompressed target data at the other of the first storage device and the second storage device, and The second synchronization operation comprises: Transmitting and storing the compressed target data from one of the first storage device and the second storage device to the other.

5. The method according to claim 4, wherein the synchronization decision database comprises a plurality of synchronization reference records, each synchronization reference record in the plurality of synchronization reference records comprising a reference environmental state of the synchronization environment and a first synchronization operation score and a second synchronization operation score corresponding to the reference environmental state, The first synchronization operation score indicates the duration required to synchronize the target data by executing the first synchronization operation in the reference environmental state, and The second synchronization operation score indicates the duration required to synchronize the target data by executing the second synchronization operation in the reference environmental state.

6. The method according to claim 5, further comprising: Obtain the similarity between the determined environmental state and the reference environmental state included in each synchronization reference record by comparing the determined environmental state with the reference environmental state; Identify the reference environmental state having the maximum similarity with the environmental state as the target reference environmental state; And Determine a first target synchronization operation score and a second synchronization operation score corresponding to the target reference environmental state.

7. The method according to claim 6, wherein selecting the synchronization operation for the target data to be synchronized from the synchronization decision database includes: In response to the first target synchronization operation score corresponding to the target reference environmental state being greater than the second synchronization operation score corresponding to the target reference environmental state, select the first synchronization operation; And In response to the first target synchronization operation score corresponding to the target reference environmental state being less than or equal to the second synchronization operation score corresponding to the target reference environmental state, select the second synchronization operation.

8. The method according to claim 1, wherein updating the synchronization decision database includes: Form an additional synchronization reference record based on the determined environmental state and the duration consumed for synchronizing the target data by performing the selected synchronization operation; And Add the formed additional synchronization reference record to the synchronization decision database.

9. An electronic device, comprising: A processor; And A memory coupled to the processor and storing instructions that, when executed by the processor, cause the electronic device to perform actions, the actions including: Determine the environmental state of the synchronization environment by monitoring environmental metrics for the synchronization environment; Based on the determined environmental state, select a synchronization operation for the target data to be synchronized from the synchronization decision database; Initiate the synchronization operation between a first storage device and a second storage device in the synchronization environment; and In response to the synchronization operation being executed, update the synchronization decision database.

10. The electronic device according to claim 9, wherein determining the environmental state of the synchronization environment includes: Capture the metric value of the environmental metric at each moment in at least one moment; Assign a weight to each moment in the at least one moment, and the sum of the assigned weights is equal to 1; And Determine the environmental state for the time period including the at least one moment by weighted averaging the metric values captured at each moment based on the assigned weights.

11. The electronic device according to claim 10, wherein the environmental metric includes one or more of the following: array configuration, system state, network state, transfer data size, data size before reduction, compression ratio, and data block size; and Wherein the array configuration includes the number of volumes, the number of asynchronous replication sessions, and the asynchronous replication recovery point objective RPO; The system state includes the central processing unit CPU utilization rate; and The network state includes write latency and input / output operations per second IOPS.

12. The electronic device according to claim 9, wherein the synchronization operation includes a first synchronization operation and a second synchronization operation, The first synchronization operation includes: Decompressing the compressed target data at one of the first storage device and the second storage device; Transmitting the decompressed target data to the other of the first storage device and the second storage device; And In response to receiving the decompressed target data, compressing and storing the decompressed target data at the other of the first storage device and the second storage device, and The second synchronization operation includes: Transmitting and storing the compressed target data from one of the first storage device and the second storage device to the other.

13. The electronic device according to claim 12, wherein the synchronization decision database includes a plurality of synchronization reference records, and each synchronization reference record in the plurality of synchronization reference records includes a reference environment state of the synchronization environment, and a first synchronization operation score and a second synchronization operation score corresponding to the reference environment state, The first synchronization operation score indicates the duration required to synchronize the target data by performing the first synchronization operation in the reference environment state, and The second synchronization operation score indicates the duration required to synchronize the target data by performing the second synchronization operation in the reference environment state.

14. The electronic device according to claim 12, the action further includes: Obtaining a similarity between the environment state and the reference environment state by comparing the determined environment state with the reference environment state included in each synchronization reference record; Identifying the reference environment state having the maximum similarity with the environment state as the target reference environment state; And Determining a first target synchronization operation score and a second target synchronization operation score corresponding to the target reference environment state.

15. The electronic device according to claim 14, wherein selecting the synchronization operation for the target data to be synchronized from the synchronization decision database includes: Selecting the first synchronization operation in response to the first target synchronization operation score corresponding to the target reference environment state being greater than the second synchronization operation score corresponding to the target reference environment state; And Selecting the second synchronization operation in response to the first target synchronization operation score corresponding to the target reference environment state being less than or equal to the second synchronization operation score corresponding to the target reference environment state.

16. The electronic device according to claim 9, wherein updating the synchronization decision database includes: Forming an additional synchronization reference record based on the determined environment state and the duration consumed for synchronizing the target data by performing the selected synchronization operation; And Adding the formed additional synchronization reference record to the synchronization decision database.

17. A computer program product tangibly stored on a non-transitory computer-readable medium and comprising computer-executable instructions that, when executed by a processor of a computer, cause the computer to: Determine an environmental state of the synchronization environment by monitoring environmental metrics for the synchronization environment; Select a synchronization operation for target data to be synchronized from a synchronization decision database based on the determined environmental state; Initiate the synchronization operation between a first storage device and a second storage device in the synchronization environment; And Update the synchronization decision database in response to the synchronization operation being executed.

18. The computer program product according to claim 17, wherein determining the environmental state of the synchronization environment comprises: Capturing a metric value of the environmental metric at each of at least one moment; Assigning a weight to each of the at least one moment, the sum of the assigned weights being equal to 1; And Determining the environmental state for a time period including the at least one moment by weighted-averaging the metric values captured at each moment based on the assigned weights.

19. The computer program product according to claim 17, wherein the computer-executable instructions, when executed by the processor, further cause the computer to: Obtain a similarity between the environmental state and a reference environmental state by comparing the determined environmental state with each reference environmental state in a plurality of synchronization reference records in the synchronization decision database; Identify the reference environmental state having the greatest similarity to the environmental state as a target reference environmental state; And Determine a first target synchronization operation score and a second synchronization operation score in a synchronization reference record including the target reference environmental state.

20. The computer program product according to claim 17, wherein updating the synchronization decision database comprises: Forming an additional synchronization reference record based on the determined environmental state and the duration consumed to synchronize the target data by executing the selected synchronization operation; And Adding the formed additional synchronization reference record to the synchronization decision database.

Images