Method and device for synchronizing differential data, computer device and storage medium

Abstract

The application relates to the technical field of clouds, and discloses a method and device for synchronizing differential data, a computer device and a storage medium, which comprise the following steps: when a trigger moment of a current period is reached and it is determined that the current period is not the first period, obtaining a first data operation log and first task information from a multi-cloud management platform, and obtaining a second data operation log and second task information from a target third-party cloud platform; comparing the first data operation log and the second data operation log to determine a first data identifier set; comparing the first task information and the second task information to determine a second data identifier set; the first data identifier set and the second data identifier set constitute a target differential data identifier set; according to the target differential data identifier set, differential data is extracted from the target third-party cloud platform, and the differential data is synchronized to the multi-cloud management platform. The application can improve the efficiency of synchronizing data.

Description

Technical Field

[0001] This invention relates to the field of cloud technology, and more specifically to methods, apparatus, computer equipment, and storage media for synchronizing differential data. Background Technology

[0002] In the field of cloud technology, multi-cloud management platforms are typically used to manage different types of third-party cloud platforms. Users can modify data stored on third-party cloud platforms from within the multi-cloud management platform, and vice versa.

[0003] When a user modifies data on a third-party cloud platform, inconsistencies may arise between the data stored on that platform and the data stored on the multi-cloud management platform. Therefore, current multi-cloud management platforms typically synchronize all data stored on the third-party cloud platform to the multi-cloud management platform at a fixed time each day.

[0004] However, when dealing with large amounts of data, the data synchronization process is slow and inefficient. Summary of the Invention

[0005] In view of this, the present invention provides a method, apparatus, computer device and storage medium for synchronizing differential data, so as to solve the problem of low efficiency in synchronizing data.

[0006] In a first aspect, the present invention provides a method for synchronizing differential data, the method being applied to a multi-cloud management platform, the method comprising: When the trigger time of the current cycle is reached and it is determined that the current cycle is not the first cycle, the first data operation log and the first task information of the previous cycle corresponding to the current cycle are obtained from the multi-cloud management platform, and the second data operation log and the second task information of the previous cycle are obtained from the target third-party cloud platform. By comparing the first data operation log and the second data operation log, a first set of data identifiers that the user has completed operations on the target third-party cloud platform in the previous period is determined; By comparing the first task information and the second task information, a second set of data identifiers is determined for the user in the previous period who did not complete the operation on the target third-party cloud platform. The first data identifier set and the second data identifier set constitute the target difference data identifier set; Based on each data identifier in the target difference data identifier set, extract the difference data corresponding to the data identifier from the target third-party cloud platform, and synchronize the difference data to the multi-cloud management platform.

[0007] The method for synchronizing differential data provided by this invention has the following advantages: During each synchronization process, it is not necessary to synchronize all the data stored on the third-party cloud platform; only the differing data needs to be synchronized. Since the amount of differing data is much smaller than the amount of data stored on the third-party cloud platform, data synchronization efficiency can be greatly improved.

[0008] In one optional implementation, each data operation record in the first data operation log and the second data operation log includes a data identifier, operation type, and operation time; The step of comparing the first data operation log and the second data operation log to determine the first data identifier set of operations that the user has completed on the target third-party cloud platform in the previous period includes: For each data operation record in the second data operation log, determine whether there is a target data operation record in the second data operation log that does not match any of the data operation records in the first data operation log, wherein the target operation record satisfies the following: it does not match the data identifier, the operation type, and the operation time in any of the data operation records in the first data operation log; When there is at least one target data operation record in the second data operation log, the data identifier of each target data operation record in the at least one target data operation record is counted to form the first data identifier set.

[0009] Specifically, for the same third-party cloud platform, since there may be multiple operations on the same data in the data operation logs, both data identifiers and operation types can be matched. Since a user may have performed the same type of operation on the same data on both the third-party cloud platform and the multi-cloud management platform, the operation time can be matched to determine if the operation is the same. In summary, by matching data identifiers, operation types, and operation times, discrepancies can be accurately identified.

[0010] In one optional implementation, each task record in the first task information and the second task information includes the task identifier and the data identifier of the data corresponding to the task. The step of comparing the first task information and the second task information to determine the second data identifier set of the user's uncompleted operations on the target third-party cloud platform in the previous period includes: For each task record in the second task information, determine whether there exists a target task record in the second task information that does not match any of the task records in the first task information, wherein the task identifier in the target task record does not match the task identifier in any of the task records in the first task information; When there is at least one target task record in the second task information, the data identifier of each target task record in the at least one target task record is counted to form the second data identifier set.

[0011] Specifically, at the triggering moment, the user may be logging into the target third-party cloud platform and performing an operation that is not yet complete. If this data is synchronized through the first and second data operation logs in the next cycle, it may result in a prolonged period of data inconsistency. Therefore, comparing the first and second task information can identify the discrepancies in the data where operations are not yet completed, ensuring the timeliness of the data.

[0012] In one optional implementation, when the amount of data in the target difference data identifier set exceeds a first preset threshold, the step of extracting difference data corresponding to each data identifier from the target third-party cloud platform and synchronizing the difference data to the multi-cloud management platform, based on each data identifier in the target difference data identifier set, includes: Extract the difference data from the target third-party cloud platform based on each data identifier in the target difference data identifier set; Based on the first preset threshold, the difference data is synchronized from the target third-party cloud platform to the multi-cloud management platform in multiple batches.

[0013] Specifically, since the amount of differential data may be quite large, by setting a first preset threshold, the differential data can be synchronized in multiple steps, which can reduce the amount of data synchronized each time and improve synchronization efficiency.

[0014] In one optional implementation, each data operation record in the second data operation log further includes a data type, and the method further includes: Based on each data identifier in the target difference data identifier set, determine the target data operation record corresponding to each data identifier in the second data operation log; Based on the operation time and data type in the target data operation record, determine the average operation time for each data type; Obtain the number of consecutive periods corresponding to each data type that were not operated through the target third-party cloud platform from the multi-cloud management platform; Based on the trigger time of the current cycle, determine the time period type corresponding to the current cycle; Based on the correspondence between data types and frequency band types, determine the frequency band type to which each data type belongs; Based on the time period type, the frequency band type, the number of consecutive cycles, the average operation time, and the second preset threshold, the cycle duration of the next cycle corresponding to the current cycle is determined.

[0015] Specifically, since some data types undergo fewer modification operations, using the same synchronization cycle for all data types would lead to more processing during the identification of discrepancies, resulting in slower processing speeds and wasted resources. Therefore, using different synchronization cycles for different data types can save resources and improve the efficiency of synchronizing discrepancies. However, an excessively long synchronization cycle may result in a large amount of discrepancies to be synchronized, affecting data timeliness. Therefore, a second preset threshold can limit the synchronization cycle for discrepancies, minimizing the amount of discrepancies in each synchronization process while ensuring data timeliness.

[0016] In one optional implementation, determining the cycle length of the next cycle corresponding to the current cycle based on the time period type, the frequency band type, the number of consecutive cycles, the average operation time, and a second preset threshold includes: The target increment coefficient is determined based on the frequency band type and the time period type; The duration of the next cycle is determined based on the target increment coefficient, the number of consecutive cycles, the average operation time, and the second preset threshold.

[0017] Specifically, since different users operate on data for different time periods and at different frequencies for different data types, determining the cycle length by combining the time period type and frequency band type can balance the processing of different data synchronization across multiple cloud management platforms and avoid the problem of excessive resource consumption during a certain period.

[0018] In one optional implementation, the cycle length of the next cycle is determined based on the target increment coefficient, the number of consecutive cycles, the average operation time, and the second preset threshold. It can be represented by the following expression: T = Max(T0 + k0 * n * T0, T) th )……(1) Where T is the cycle length of the next cycle, T0 is the average operation time, k0 is the target increment coefficient, n is the number of consecutive cycles, and T th The second preset threshold.

[0019] Specifically, by selecting one of the synchronization period and the second preset threshold for each data type determined by the time period type and the frequency band type, the characteristics of the data type itself can be taken into account, while avoiding the problem of excessively long synchronization periods.

[0020] In a second aspect, the present invention provides an apparatus for synchronizing differential data, the apparatus comprising: The acquisition module is used to acquire, when the trigger time of the current cycle is reached and it is determined that the current cycle is not the first cycle, the first data operation log and the first task information of the previous cycle corresponding to the current cycle from the multi-cloud management platform, and the second data operation log and the second task information of the previous cycle from the target third-party cloud platform. The comparison module is used to compare the first data operation log and the second data operation log to determine a first data identifier set of operations that the user has completed on the target third-party cloud platform in the previous period; and to compare the first task information and the second task information to determine a second data identifier set of operations that the user has not completed on the target third-party cloud platform in the previous period; the first data identifier set and the second data identifier set constitute a target difference data identifier set. The synchronization module is used to extract the difference data corresponding to each data identifier from the target third-party cloud platform according to each data identifier in the target difference data identifier set, and synchronize the difference data to the multi-cloud management platform.

[0021] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the method for synchronizing differential data as described in the first aspect or any corresponding embodiment thereof.

[0022] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform a method for synchronizing differential data according to the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0023] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1This is a flowchart illustrating a method for synchronizing differential data according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating another method for synchronizing differential data according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating a method for determining the duration of a cycle according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the correspondence between frequency band type, time period type, and increment coefficient according to an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the data processing flow of a multi-cloud management platform according to an embodiment of the present invention; Figure 6 This is a structural block diagram of a device for synchronizing differential data according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] In the cloud technology field, multi-cloud management platforms are used to manage different types of third-party cloud platforms. The multi-cloud management platform stores all data stored by the third-party cloud platforms. Users can log in to the multi-cloud management platform and modify the data stored by the third-party cloud platforms through their interfaces, or they can directly log in to the third-party cloud platforms to modify the data stored there. In the second scenario, discrepancies may arise between the data stored on the multi-cloud management platform and the third-party cloud platforms. The multi-cloud management platform can synchronize the data stored by the third-party cloud platforms to maintain consistency between the two cloud platforms.

[0027] This invention provides a method for synchronizing differential data, thereby improving the efficiency of data synchronization.

[0028] According to an embodiment of the present invention, a method embodiment for synchronizing differential data is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0029] The following embodiments use a single third-party cloud platform as an example. In practical applications, a multi-cloud management platform may manage multiple third-party cloud platforms. For each third-party cloud platform, a process of synchronizing differential data can be performed according to the specific implementation methods described below. Accordingly, the first data operation log, the second data operation log, the first task information, and the second task information may include the identification information of the third-party cloud platform.

[0030] This embodiment provides a method for synchronizing differential data, applied to a multi-cloud management platform. The multi-cloud management platform stores a persistent database and a cache database. The persistent database can store data from various third-party cloud platforms, as well as data operation logs and task information. The cache database can store data from various third-party cloud platforms, enabling data backup and load balancing during data retrieval, thus reducing the load on the persistent database. Figure 1 This is a flowchart of a method for synchronizing differential data according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps: Step S101: When the trigger time of the current cycle is reached and it is determined that the current cycle is not the first cycle, the first data operation log and the first task information of the previous cycle corresponding to the current cycle are obtained from the multi-cloud management platform, and the second data operation log and the second task information of the previous cycle are obtained from the target third-party cloud platform.

[0031] The trigger time for the current cycle can be determined by the multi-cloud management platform based on a preset duration cycle.

[0032] Specifically, when the multi-cloud management platform detects the trigger time of the current period and determines that the current period is the first period, it can use the trigger time of the current period as the cutoff time and obtain the data operation logs before the cutoff time. When the multi-cloud management platform detects the trigger time of the current period and determines that the current period is not the first period, it can determine the target time period corresponding to the previous period based on the trigger time of the current period and the trigger time of the previous period. Furthermore, based on the target time period, it can obtain the first data operation log and the first task information within the target time period.

[0033] In addition, the multi-cloud management platform can send a first retrieval request to the target third-party cloud platform. This first retrieval request includes a target time period. Upon receiving the first retrieval request, the target third-party cloud platform can retrieve the second data operation log and second task information for the target time period from its own stored database, and then send them to the multi-cloud management platform. After receiving the second data operation log and second task information, the multi-cloud management platform can proceed with subsequent processing.

[0034] Step S102: Compare the first data operation log and the second data operation log to determine the first data identifier set of operations that the user has completed on the target third-party cloud platform in the previous period.

[0035] Specifically, the first data operation log only includes data operation logs of users logging into the multi-cloud management platform and operating on data stored on the target third-party cloud platform, while the second data operation log includes data operation logs of users logging into the multi-cloud management platform and operating on data stored on the target third-party cloud platform, as well as data operation logs of users directly logging into the target third-party cloud platform and operating on their own stored data. Therefore, the multi-cloud management platform can compare the first and second data operation logs to determine the data operation logs of users directly logging into the target third-party cloud platform and operating on their own stored data. Furthermore, a first data identifier set can be constructed based on the data identifiers in the data operation logs determined in the above process.

[0036] Step S103: Compare the first task information and the second task information to determine the second data identifier set that the user did not complete the operation on the target third-party cloud platform in the previous cycle.

[0037] Specifically, the first task information only includes task information about users logging into the multi-cloud management platform to operate on data stored on the target third-party cloud platform, while the second task information includes task information about users logging into the multi-cloud management platform to operate on data stored on the target third-party cloud platform, as well as task information about users directly logging into the target third-party cloud platform to operate on their own stored data. Therefore, the multi-cloud management platform can compare the first and second task information to determine the task information about users directly logging into the target third-party cloud platform to operate on their own stored data. Furthermore, a second set of data identifiers can be constructed based on the data identifiers in the determined task information.

[0038] Thus, the first set of data identifiers and the second set of data identifiers can constitute the target difference data identifier set.

[0039] Step S104: Based on each data identifier in the target difference data identifier set, extract the difference data corresponding to the data identifier from the target third-party cloud platform, and synchronize the difference data to the multi-cloud management platform.

[0040] Specifically, the multi-cloud management platform can send a second retrieval request to the target third-party cloud platform. This second retrieval request includes each data identifier from the target set of difference data identifiers. Upon receiving the second retrieval request, the target third-party cloud platform can retrieve the difference data corresponding to each data identifier from its own stored database, and then send each data identifier and the difference data to the multi-cloud management platform. The multi-cloud management platform can then delete the data corresponding to each data identifier from the persistent database and the cache database based on the data identifiers in the target set of difference data identifiers. Finally, upon receiving the difference data, the multi-cloud management platform can store the difference data in the persistent database and the cache database.

[0041] In some possible implementations, before storing the differential data, the multi-cloud management platform can first identify the data format of the third-party cloud platform and determine whether the data format of the third-party cloud platform is the same as that of the multi-cloud management platform. If not, the differential data can be input into the format conversion model to obtain the converted differential data before storage.

[0042] The method for synchronizing differential data provided in this embodiment does not require synchronizing the full amount of data stored on the third-party cloud platform during each synchronization process; only the differential data needs to be synchronized. Since the amount of differential data is much smaller than the amount of data stored on the third-party cloud platform, this significantly improves data synchronization efficiency.

[0043] This embodiment provides a method for synchronizing differential data, which is applied to a multi-cloud management platform. Figure 2 This is a flowchart of a method for synchronizing differential data according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: When the trigger time of the current cycle is reached and it is determined that the current cycle is not the first cycle, the first data operation log and the first task information of the previous cycle corresponding to the current cycle are obtained from the multi-cloud management platform, and the second data operation log and the second task information of the previous cycle are obtained from the target third-party cloud platform.

[0044] Please see details Figure 1 Step S101 of the illustrated embodiment will not be described again here.

[0045] Step S202: Compare the first data operation log and the second data operation log to determine the first data identifier set of operations that the user has completed on the target third-party cloud platform in the previous period.

[0046] Each data operation record in both the first and second data operation logs includes a data identifier, operation type, and operation time. The operation type can be add, delete, or modify, and the operation time can include the start and end times. The first data operation log can be shown in Table 1, and the second data operation log can be shown in Table 2.

[0047] Table 1

[0048] Table 2

[0049] Specifically, step S202 includes: Step S2021: For each data operation record in the second data operation log, determine whether there is a target data operation record in the second data operation log that does not match any data operation record in the first data operation log.

[0050] The target operation record must meet the following condition: it does not match the data identifier, operation type, or operation time of any data operation record in the first data operation log. The target data operation record may exhibit the following characteristics: First, the data identifier is different from that in each data operation record in the first data operation log.

[0051] For example, the seventh data operation record in Table 2.

[0052] Second, the data identifier is the same as that in at least one data operation record in the first data operation log, but the operation type is different from that in at least one data operation record.

[0053] For example, the sixth data operation record in Table 2.

[0054] Third, the data identifier and operation type are the same as those in at least one data operation record in the first data operation log, but the operation time does not match that in at least one data operation record, meaning that the operation time in the target operation data record does not include the operation time in the at least one data operation record.

[0055] For example, the first data operation record in Table 2 has the same data identifier and operation type as the first data operation record in Table 1, but the operation time of the first data operation record in Table 1 does not include the operation time of the first data operation record in Table 2.

[0056] Step S2022: When there is at least one target data operation record in the second data operation log, count the data identifier of each target data operation record in the at least one target data operation record to form a first data identifier set.

[0057] Specifically, when the multi-cloud management platform determines that there is at least one target data operation record in the second data operation log, it can statistically analyze the data identifiers of the target data operation record to obtain the first data identifier set.

[0058] For example, based on Tables 1 and 2, three target data operation records are identified in the second data operation log: the first, sixth, and seventh data operation records. The data identifiers corresponding to these three target data operation records are "001", "002", and "003", respectively. These three data identifiers constitute the first data identifier set.

[0059] Step S203: Compare the first task information and the second task information to determine the second data identifier set that the user did not complete the operation on the target third-party cloud platform in the previous cycle.

[0060] Each task record in the first and second task information includes a task identifier and a data identifier for the data corresponding to the task. The first task information can be shown in Table 3, and the second task information can be shown in Table 4.

[0061] Table 3

[0062] Table 4

[0063] Specifically, step S203 includes: Step S2031: For each task record in the second task information, determine whether there is a target task record in the second task information that does not match any of the task records in the first task information.

[0064] In particular, the task identifier in the target task record does not match the task identifier in each task record in the first task information.

[0065] Specifically, since different cloud platforms use different task identifiers, the multi-cloud management platform can first determine the task identifier for each task record in the second task information. Then, based on the stored correspondence between multi-cloud management platform task identifiers and target third-party cloud platform task identifiers (as shown in Table 5), it can determine the multi-cloud management platform task identifier corresponding to each task record in the second task information. Finally, based on the determined multi-cloud management platform task identifier, it can determine whether there is a task record in the first task information with the same multi-cloud management platform task identifier.

[0066] Example 1: For the first task record in Table 4, its task identifier is "AAA". Based on Table 5 and "AAA", its corresponding multi-cloud management platform task identifier can be determined to be "111". Table 3 shows that there is a task record with the task identifier "111".

[0067] Example 2: For the fifth task record in Table 4, its task identifier is "EEE". Based on Table 5 and "EEE", its corresponding multi-cloud management platform task identifier can be determined to be "555". From Table 3, it can be seen that there is no task record with the task identifier "555", so the fifth task record in Table 4 can be determined as the target task record.

[0068] Table 5

[0069] Step S2032: When there is at least one target task record in the second task information, count the data identifiers of each target task record in the at least one target task record to form a second data identifier set.

[0070] Specifically, when the multi-cloud management platform determines that there is at least one target task record in the second task information, it can statistically analyze the data identifiers of the target task records to obtain a second set of data identifiers. For example, according to Tables 3, 4, and 5, the determined second set of data identifiers contains a data identifier "069".

[0071] The first set of data identifiers and the second set of data identifiers constitute the target difference data identifier set.

[0072] In other possible implementations, Tables 3 and 5 could be a single table, where the information from the second column of Table 5 is merged into Table 3.

[0073] In other possible implementations, each data operation record in both the first and second data operation logs can include the operation result. Accordingly, in the process of determining the target difference data identifier set described above, only data operation records with a successful operation result can be considered. Each task record in both the first and second task information can include the task status. Accordingly, in the process of determining the target difference data identifier set described above, only task records with an incomplete task status can be considered.

[0074] Step S204: Based on each data identifier in the target difference data identifier set, extract the difference data corresponding to the data identifier from the target third-party cloud platform, and synchronize the difference data to the multi-cloud management platform.

[0075] Specifically, step S204 includes: Step S2041: Extract difference data from the target third-party cloud platform based on each data identifier in the target difference data identifier set.

[0076] Specifically, the multi-cloud management platform can send a third-party acquisition request to the target third-party cloud platform. This third-party acquisition request includes each data identifier in the target difference data identifier set and a first preset threshold. Upon receiving the third-party acquisition request, the target third-party cloud platform can retrieve the difference data corresponding to each data identifier from its own stored database, based on the data identifier in the third-party acquisition request.

[0077] Step S2042: Based on the first preset threshold, synchronize the difference data from the target third-party cloud platform to the multi-cloud management platform in multiple steps.

[0078] Specifically, the multi-cloud management platform can group the differential data according to the first preset threshold in the third acquisition request, obtaining multiple groups of differential data. Each time, one group of differential data and its corresponding data identifier are sent to the multi-cloud management platform. Upon receiving each group of differential data and its data identifier, the multi-cloud management platform can store the differential data in the appropriate location based on the data identifier.

[0079] Additionally, the first preset threshold can also be applied during the initial management of a third-party cloud platform. Specifically, when the multi-cloud management platform receives an instruction to manage the target third-party cloud platform, it can send a fourth acquisition request to the target third-party cloud platform. This fourth acquisition request may include the first preset threshold. After receiving the fourth acquisition request, the target third-party cloud platform can divide its stored data into multiple data transmission units, using the first preset threshold as the unit for each transmission, and send the data to the multi-cloud management platform multiple times. The multi-cloud management platform then receives the data sent by the target third-party cloud platform in multiple transmissions.

[0080] Since the target third-party cloud platform may store a large amount of data, receiving data in batches during the management process can alleviate the problem of the multi-cloud management platform receiving too much data at the same time. This can reduce the pressure on the multi-cloud management platform in terms of computing and network, making the operation of the multi-cloud management platform more stable.

[0081] The method for synchronizing differential data provided in this embodiment addresses the issue that, for the same cloud platform, multiple operations on the same data may exist in the data operation logs. Therefore, matching can be performed based on both data identifiers and operation types. Since users may perform the same type of operation on the same data on both third-party cloud platforms and multi-cloud management platforms, matching can be performed based on operation time. In summary, by matching data identifiers, operation types, and operation times, differential data can be accurately identified. Because at the triggering moment, the user may be logging into the target third-party cloud platform and performing an operation that is not yet complete, synchronizing this data again in the next cycle via the first and second data operation logs could lead to prolonged periods of data inconsistency. Therefore, comparing the first and second task information can identify differential data where operations are not yet complete, ensuring data timeliness. Furthermore, since the amount of differential data may be large, setting a first preset threshold allows for multiple synchronizations of the differential data, reducing the amount of data synchronized each time and improving synchronization efficiency.

[0082] Each data operation record in the second data operation log may also include a data type. Accordingly, this embodiment provides a method for determining the period duration, applied to a multi-cloud management platform. Figure 3 This is a flowchart of a method for determining the duration of a period according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps: Step S301: Based on each data identifier in the target difference data identifier set, determine the target data operation record corresponding to each data identifier in the second data operation log.

[0083] Specifically, the multi-cloud management platform can determine the target data operation record in the second data operation log based on each data identifier in the target difference data identifier set. In this way, the multi-cloud management platform can perform subsequent processing based on the information included in the target data operation record.

[0084] Step S302: Determine the average operation time for each data type based on the operation time and data type in the target data operation record.

[0085] The data types can be cloud hosts, basic networks, data centers, hard drives, images, etc.

[0086] Specifically, the multi-cloud management platform can first identify target data operation records with the same data type, and then average the operation time of the target data operation records with the same data type to obtain the average operation time for each data type.

[0087] Step S303: Obtain from the multi-cloud management platform the number of consecutive cycles for each data type that have not been operated through the target third-party cloud platform.

[0088] Specifically, the multi-cloud management platform can store the correspondence between data types and the number of consecutive periods. After each synchronization of differential data, the multi-cloud management platform can determine the data type corresponding to each data identifier in the target differential data identifier set. If differential data exists for a certain data type in the current period, the number of consecutive periods corresponding to that data type can be reset to zero. If differential data does not exist for a certain data type in the current period, the number of consecutive periods corresponding to that data type can be incremented by one. Therefore, in determining the period duration, the multi-cloud management platform can determine the number of consecutive periods corresponding to each data type from the correspondence between data types and the number of consecutive periods.

[0089] Step S304: Determine the time period type corresponding to the current period based on the trigger time of the current period.

[0090] Specifically, the multi-cloud management platform can record and statistically analyze users' daily operation time, obtaining multiple time period types for user operation data. These time period types can include busy periods and idle periods. For example, a busy period could be 7:00 to 19:00, and an idle period could be 19:00 to 7:00.

[0091] Step S305: Determine the frequency band type to which each data type belongs based on the correspondence between data type and frequency band type.

[0092] The frequency band type can be high frequency, medium frequency, or low frequency.

[0093] Specifically, a multi-cloud management platform can store the correspondence between data types and frequency band types. For example, cloud hosts correspond to high-frequency types, and image libraries correspond to medium-frequency types.

[0094] Step S306: Determine the cycle duration of the next cycle corresponding to the current cycle based on the time period type, frequency band type, number of consecutive cycles, average operation time, and second preset threshold.

[0095] Specifically, step S306 includes: Step S3061: Determine the target increment coefficient based on the frequency band type and time period type.

[0096] Specifically, a multi-cloud management platform can store the correspondence between frequency band type, time period type, and increment coefficient, such as... Figure 4 As shown, further, the target increment coefficient can be determined in the correspondence based on the frequency band type and time period type determined above.

[0097] Step S3062: Determine the cycle length of the next cycle based on the target increment coefficient, the number of consecutive cycles, the average operation time, and the second preset threshold.

[0098] Specifically, the multi-cloud management platform can use the following expression to determine the duration of the next cycle: T = Max(T0 + k0 * n * T0, T) th )……(1) Where T is the cycle length of the next cycle, T0 is the average operation time, k0 is the target increment coefficient (positive), n is the number of consecutive cycles, and T th This is the second preset threshold.

[0099] Specifically, if the user does not log in to the third-party cloud platform to operate on this data type within the current period, the period duration of this data type in the next period will increase by k0T0. If the user has not operated on this data type for several consecutive periods, and the determined T0 + k0*n*T0 is greater than the second preset threshold, then the second preset threshold can be used as the period duration of the next period.

[0100] The method for determining the synchronization period provided in this embodiment addresses the issue that some data types require fewer modification operations. If all data types use the same synchronization period, it would lead to more processing during the determination of difference data, resulting in slower processing speeds and wasted resources. Therefore, using different synchronization periods for different data types can save resources and improve the efficiency of synchronizing difference data. Since an excessively long synchronization period may result in a large amount of difference data to be synchronized and affect data timeliness, a second preset threshold can be used to limit the synchronization period of difference data, minimizing the amount of difference data in each synchronization process. This also ensures data timeliness.

[0101] like Figure 5As shown, the multi-cloud management platform may include a monitoring device, a processing device, a synchronization cycle device, a persistent database, and a cache database. The target third-party cloud platform may include a monitoring device. The monitoring device of the multi-cloud management platform stores first data operation logs and first task information. The monitoring device of the target third-party cloud platform stores second data operation logs and second task information. The processing device compares the first and second data operation logs, and compares the first and second task information, to determine the target difference data identifier set, the cycle length of the next cycle, and the number of consecutive cycles. The synchronization cycle device triggers the processing of the synchronization difference data for the next cycle based on the cycle length of the next cycle.

[0102] This embodiment also provides a device for synchronizing differential data, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0103] This embodiment provides a device for synchronizing differential data, such as... Figure 6 As shown, it includes: The acquisition module 601, when the trigger time of the current cycle is reached and it is determined that the current cycle is not the first cycle, acquires the first data operation log and the first task information of the previous cycle corresponding to the current cycle from the multi-cloud management platform, and acquires the second data operation log and the second task information of the previous cycle from the target third-party cloud platform. The comparison module 602 compares the first data operation log and the second data operation log to determine the first data identifier set of operations that the user has completed on the target third-party cloud platform in the previous period; it compares the first task information and the second task information to determine the second data identifier set of operations that the user has not completed on the target third-party cloud platform in the previous period; the first data identifier set and the second data identifier set constitute the target difference data identifier set. The synchronization module 603 extracts the difference data corresponding to each data identifier from the target third-party cloud platform based on each data identifier in the target difference data identifier set, and synchronizes the difference data to the multi-cloud management platform.

[0104] In one optional implementation, each data operation record in the first data operation log and the second data operation log includes a data identifier, operation type, and operation time. Comparison module 602 is used for: For each data operation record in the second data operation log, determine whether there is a target data operation record in the second data operation log that does not match any data operation record in the first data operation log. The target operation record satisfies the following: it does not match any data identifier, operation type, or operation time in any data operation record in the first data operation log. When there is at least one target data operation record in the second data operation log, the data identifier of each target data operation record in the at least one target data operation record is counted to form the first data identifier set.

[0105] In one optional implementation, each task record in the first task information and the second task information includes a task identifier and a data identifier for the data corresponding to the task. Comparison module 602 is used for: For each task record in the second task information, determine whether there is a target task record in the second task information that does not match any of the task records in the first task information, wherein the task identifier in the target task record does not match the task identifier in any of the task records in the first task information. When there is at least one target task record in the second task information, the data identifier of each target task record in the at least one target task record is counted to form the second data identifier set.

[0106] In an optional implementation, when the amount of data in the target difference data identifier set exceeds a first preset threshold, the synchronization module 603 is configured to: Extract the difference data from the target third-party cloud platform based on each data identifier in the target difference data identifier set; Based on the first preset threshold, the difference data is synchronized from the target third-party cloud platform to the multi-cloud management platform in multiple batches.

[0107] In one optional implementation, each data operation record in the second data operation log further includes a data type, and the apparatus further includes: The determination module 604 is used to determine the target data operation record corresponding to each data identifier in the second data operation log based on each data identifier in the target difference data identifier set; and to determine the average operation time of each data type based on the operation time and data type in the target data operation record. The acquisition module 601 is used to obtain the number of consecutive periods corresponding to each data type that have not been operated through the target third-party cloud platform from the multi-cloud management platform; The determination module 604 is used to determine the time period type corresponding to the current period based on the trigger time of the current period; determine the frequency band type to which each data type belongs based on the correspondence between data type and frequency band type; and determine the period duration of the next period corresponding to the current period based on the time period type, frequency band type, number of consecutive periods, average operation time and second preset threshold.

[0108] In one alternative implementation, the determining module 604 is configured to: Determine the target increment factor based on the frequency band type and time period type; The cycle length of the next cycle is determined based on the target increment coefficient, the number of consecutive cycles, the average operation time, and the second preset threshold.

[0109] In one alternative implementation, the determining module 604 is represented by the following expression: T = Max(T0 + k0 * n * T0, T) th )……(1) Where T is the cycle length of the next cycle, T0 is the average operation time, k0 is the target increment coefficient, n is the number of consecutive cycles, and T th This is the second preset threshold.

[0110] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0111] In this embodiment, the device for synchronizing differential data is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0112] This invention also provides a computer device having the above-described features. Figure 6 The device shown is for synchronizing differential data.

[0113] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 7As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 7 Take a processor 10 as an example.

[0114] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0115] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0116] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0117] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0118] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0119] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0120] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for synchronizing differential data, characterized in that, The method is applied to a multi-cloud management platform, which manages different types of third-party cloud platforms. The method includes: When the trigger time of the current cycle is reached and it is determined that the current cycle is not the first cycle, the first data operation log and the first task information of the previous cycle corresponding to the current cycle are obtained from the multi-cloud management platform, and the second data operation log and the second task information of the previous cycle are obtained from the target third-party cloud platform. Each data operation record in the first data operation log and the second data operation log includes a data identifier, operation type, operation time, and data type. The data type is any one of cloud host, basic network, data center, hard disk, and image. By comparing the first data operation log and the second data operation log, a first set of data identifiers that the user has completed operations on the target third-party cloud platform in the previous period is determined; By comparing the first task information and the second task information, a second set of data identifiers is determined for the user in the previous period who did not complete the operation on the target third-party cloud platform. The first data identifier set and the second data identifier set constitute the target difference data identifier set; Based on each data identifier in the target difference data identifier set, extract the difference data corresponding to the data identifier from the target third-party cloud platform, and synchronize the difference data to the multi-cloud management platform; Based on each data identifier in the target difference data identifier set, determine the data type corresponding to each data identifier; within the current period, after determining that there is difference data of the target data type, reset the number of consecutive periods corresponding to the target data type to zero; after determining that there is no difference data of the target data type, increment the number of consecutive periods corresponding to the target data type by one, wherein the target data type is any data type; Based on each data identifier in the target difference data identifier set, determine the target data operation record corresponding to each data identifier in the second data operation log; based on the operation time and data type in the target data operation record, determine the average operation time for each data type; obtain from the multi-cloud management platform the number of consecutive cycles corresponding to each data type that were not operated through the target third-party cloud platform; based on the trigger time of the current cycle, determine the time period type corresponding to the current cycle; based on the correspondence between data type and frequency band type, determine the frequency band type to which each data type belongs; based on the time period type, the frequency band type, the number of consecutive cycles, the average operation time, and the second preset threshold, determine the cycle duration of the next cycle corresponding to the current cycle for the data type.

2. The method according to claim 1, characterized in that, The step of comparing the first data operation log and the second data operation log to determine the first data identifier set of operations that the user has completed on the target third-party cloud platform in the previous period includes: For each data operation record in the second data operation log, determine whether there is a target data operation record in the second data operation log that does not match any of the data operation records in the first data operation log, wherein the target operation record satisfies the following: it does not match the data identifier, the operation type, and the operation time in any of the data operation records in the first data operation log; When there is at least one target data operation record in the second data operation log, the data identifier of each target data operation record in the at least one target data operation record is counted to form the first data identifier set.

3. The method according to claim 2, characterized in that, Each task record in the first task information and the second task information includes a task identifier and a data identifier for the data corresponding to the task; The step of comparing the first task information and the second task information to determine the second data identifier set of the user's uncompleted operations on the target third-party cloud platform in the previous period includes: For each task record in the second task information, determine whether there exists a target task record in the second task information that does not match any of the task records in the first task information, wherein the task identifier in the target task record does not match the task identifier in any of the task records in the first task information; When there is at least one target task record in the second task information, the data identifier of each target task record in the at least one target task record is counted to form the second data identifier set.

4. The method according to any one of claims 1 to 3, characterized in that, When the amount of data in the target difference data identifier set exceeds a first preset threshold, the step of extracting difference data corresponding to each data identifier from the target third-party cloud platform and synchronizing the difference data to the multi-cloud management platform, based on each data identifier in the target difference data identifier set, includes: Extract the difference data from the target third-party cloud platform based on each data identifier in the target difference data identifier set; Based on the first preset threshold, the difference data is synchronized from the target third-party cloud platform to the multi-cloud management platform in multiple batches.

5. The method according to claim 1 or 2, characterized in that, The step of determining the cycle duration of the next cycle corresponding to the current cycle based on the time period type, the frequency band type, the number of consecutive cycles, the average operation time, and a second preset threshold includes: The target increment coefficient is determined based on the frequency band type and the time period type; The duration of the next cycle is determined based on the target increment coefficient, the number of consecutive cycles, the average operation time, and the second preset threshold.

6. The method according to claim 5, characterized in that, The determination of the cycle length of the next cycle based on the target increment coefficient, the number of consecutive cycles, the average operation time, and the second preset threshold is expressed by the following expression: T=Max(T0+k0×n×T0,T th )……(1) Where T is the cycle length of the next cycle, T0 is the average operation time, k0 is the target increment coefficient, n is the number of consecutive cycles, and T th The second preset threshold.

7. A device for synchronizing differential data, characterized in that, The device is applied to a multi-cloud management platform, which manages different types of third-party cloud platforms. The device includes: The acquisition module is used to acquire, when the trigger time of the current cycle is reached and it is determined that the current cycle is not the first cycle, the first data operation log and the first task information of the previous cycle corresponding to the current cycle from the multi-cloud management platform, and the second data operation log and the second task information of the previous cycle from the target third-party cloud platform. Each data operation record in the first data operation log and the second data operation log includes a data identifier, operation type, operation time, and data type. The data type is any one of cloud host, basic network, data center, hard disk, and image. The comparison module is used to compare the first data operation log and the second data operation log to determine a first data identifier set of operations that the user has completed on the target third-party cloud platform in the previous period; and to compare the first task information and the second task information to determine a second data identifier set of operations that the user has not completed on the target third-party cloud platform in the previous period; the first data identifier set and the second data identifier set constitute a target difference data identifier set. The synchronization module is used to extract the difference data corresponding to each data identifier from the target third-party cloud platform based on each data identifier in the target difference data identifier set, and synchronize the difference data to the multi-cloud management platform; determine the data type corresponding to each data identifier based on each data identifier in the target difference data identifier set; within the current period, after determining that difference data of the target data type exists, reset the number of consecutive periods corresponding to the target data type to zero, and after determining that difference data of the target data type does not exist, increment the number of consecutive periods corresponding to the target data type by one, wherein the target data type can be any data type; and based on each data identifier in the target difference data identifier set, in the current period... The second data operation log identifies the target data operation record corresponding to each data identifier; based on the operation time and data type in the target data operation record, the average operation time for each data type is determined; the number of consecutive cycles corresponding to each data type that were not operated through the target third-party cloud platform is obtained from the multi-cloud management platform; based on the trigger time of the current cycle, the time period type corresponding to the current cycle is determined; based on the correspondence between data type and frequency band type, the frequency band type to which each data type belongs is determined; based on the time period type, the frequency band type, the number of consecutive cycles, the average operation time, and the second preset threshold, the cycle duration of the next cycle corresponding to the current cycle for the data type is determined.

8. A computer device, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method for synchronizing differential data as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method of synchronizing differential data as described in any one of claims 1 to 6.

Images