A method and device for cloud computing resource data incremental synchronization

Abstract

The present application relates to the field of cloud computing, in particular to a kind of cloud computing resource data incremental synchronization method and device.Mainly include: according to the last synchronization of each resource type Resource quantity and each resource type Resource total amount, calculate the resource recent change ratio of each resource type, according to the start time of last synchronization task, the resource change coefficient of each resource type and resource recent change ratio, calculate the pull data start time of each resource type;Synchronization task starts, according to resource dependency relationship in turn obtains the incremental data of each resource type between pull data start time and current time, and the incremental data of each resource type is synchronized.The present application can meet the demand of only incremental resource synchronization in the case of only part of resource variation, avoid the resource excessive occupation caused by full amount synchronization, improve the synchronization effectiveness and synchronization efficiency of computing resource in cloud platform.

Description

Technical Field

[0001] This invention relates to the field of cloud computing, and in particular to a method and apparatus for incremental synchronization of cloud computing resource data. Background Technology

[0002] In cloud computing platforms, underlying data needs to be synchronized to a local database to improve response speed. Currently, most existing computing resource data synchronization solutions in the industry periodically pull all resources from the underlying layer, compare them with the data in the local database, correct any discrepancies in the local data, and finally complete the data synchronization. This method is called full synchronization. For example, every 30 seconds, all cloud host, host machine, image, cloud disk, snapshot, network, router, virtual network card, tenant, user, and other resource data are synchronized upwards.

[0003] The full synchronization technical solution has the following drawbacks:

[0004] 1) In a real-world cloud environment, not all computing resources change frequently. Only when users issue services will a few resources change. Typically, less than 1% of resources change during the gap between two adjacent synchronization tasks. Therefore, the effectiveness of the full synchronization method is very low. 2) The full synchronization method requires the underlying computing service to periodically search for resources across the entire system. This can easily cause the system to have excessively high CPU and memory usage in certain periods, leading to external request blocking and reduced service capacity. 3) When the amount of computing resources is large, after the full synchronization task pulls all resources, the Portal will spend a long time comparing and processing the result data.

[0005] Due to the aforementioned shortcomings, existing synchronization technologies cannot directly meet the needs of certain scenarios, such as: 1) Three cloud hosts have changed in the past 5 seconds, and only the data of these three cloud hosts should be synchronized; 2) Six cloud disks have changed in the past hour, and only the data of these six cloud disks should be synchronized; similar scenarios are not limited to the above two types of resources. In the above scenarios, full synchronization cannot meet the requirements.

[0006] Therefore, how to overcome the shortcomings of existing technologies and solve the shortcomings of existing full-scale synchronization technical solutions is a problem to be solved in this technical field. Summary of the Invention

[0007] In view of the above-mentioned defects or improvement needs of existing technologies, the present invention solves the problems of low effectiveness and efficiency of existing cloud computing resource synchronization methods and high computing service resource occupancy.

[0008] The embodiments of the present invention adopt the following technical solutions:

[0009] In a first aspect, the present invention provides a method for incremental synchronization of cloud computing resource data, specifically comprising: calculating the recent change ratio of each type of resource based on the changed resource quantity and total resource quantity of each type of resource at the time of the last synchronization; calculating the data retrieval start time of each type of resource based on the start time of the last synchronization task, the resource change coefficient of each type of resource, and the recent change ratio of resources; when the synchronization task starts, sequentially acquiring the incremental data of each type of resource from the data retrieval start time to the current time according to the resource dependency relationship, and synchronizing the incremental data of each type of resource.

[0010] Preferably, the recent change ratio of resources specifically includes: after the completion of the last synchronization task, obtaining the number of changed resources and the total amount of resources of each type in the last synchronization task, and using the ratio of the number of changed resources of each type to the total amount of resources obtained in the last synchronization task as the recent change ratio of resources of that type in this synchronization.

[0011] Preferably, calculating the data retrieval start time for each type of resource further includes: calculating the elastic time window for the current synchronization of each type of resource based on the recent change ratio of the resource at the last synchronization, and then shifting the elastic time window back to the past based on the task start time of the last synchronization of the resource type, so that the obtained time point is the data retrieval start time for the current synchronization.

[0012] Preferably, the elastic time window for each type of resource during this synchronization is calculated, specifically including: after the last synchronization task is completed, the product of the fixed maximum time window, the recent change ratio of the resource type in the last synchronization task, and the resource change coefficient of the resource type is used as the value of the elastic time window for the resource type during this synchronization task.

[0013] Preferably, the resource change coefficient specifically includes: setting the value of the resource change coefficient according to the regular change characteristics of the resource; the more frequently the resource type changes, the larger the value of the resource change coefficient.

[0014] Preferably, the resource change coefficient also includes configuring the resource change coefficient in the form of a file and / or database outside the synchronization program.

[0015] Preferably, the incremental data for each type of resource between the start time of data retrieval and the current time is obtained. Specifically, this includes: encapsulating the start time of data retrieval as a time parameter, sending a data retrieval request carrying the time parameter to the underlying layer through a list query interface, and the underlying interface returning the incremental data that has changed between the start time of data retrieval and the current time.

[0016] Preferably, the incremental data of each type of resource is synchronized, specifically including: according to the order of resource dependencies between different types of resources, the incremental data of each type of resource is pulled from the bottom layer to the local library in sequence, wherein the order of the resource dependencies is to synchronize the most basic resources first and the edge resources last.

[0017] Preferably, synchronizing incremental data for each type of resource further includes: adding each type of resource that needs to perform synchronization tasks to a first-in-first-out queue according to the order of their dependencies, and synchronizing the incremental data of each type of resource sequentially according to the queue order.

[0018] On the other hand, the present invention provides an apparatus for incremental synchronization of cloud computing resource data, specifically comprising at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, the memory stores instructions that can be executed by the at least one processor, and the instructions, after being executed by the processor, are used to complete the method for incremental synchronization of cloud computing resource data in the first aspect.

[0019] Compared with existing technologies, the beneficial effects of this invention are as follows: A resource change coefficient is defined based on the regular change characteristics of resources; a recent resource change ratio is calculated based on recent resource changes; an elastic time window for data retrieval is dynamically calculated based on the resource change coefficient and the recent resource change ratio; and the data retrieval start time is calculated based on the elastic time window. Only incremental data after the data retrieval start time is synchronized. This method can meet the requirement of synchronizing only incremental resources when only some resources change, avoiding excessive resource consumption caused by full synchronization, and improving the effectiveness and efficiency of computing resource synchronization in the cloud platform. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0021] Figure 1 A flowchart of a method for incremental synchronization of cloud computing resource data provided in an embodiment of the present invention;

[0022] Figure 2 This is a schematic diagram of a computing resource data synchronization model in a cloud platform according to an embodiment of the present invention;

[0023] Figure 3 This is a schematic diagram of the time axis model for the time recording of the synchronization task in an embodiment of the present invention;

[0024] Figure 4This is a schematic diagram of the elastic time window model for synchronization tasks in an embodiment of the present invention;

[0025] Figure 5 This is a schematic diagram of the time axis model for querying resources in a synchronization task according to an embodiment of the present invention;

[0026] Figure 6 A flowchart of another method for incremental synchronization of cloud computing resource data provided in an embodiment of the present invention;

[0027] Figure 7 This is a schematic diagram illustrating the dependencies between computing resources in the cloud platform in an embodiment of the present invention;

[0028] Figure 8 This is a schematic diagram of the resource synchronization queue in an embodiment of the present invention;

[0029] Figure 9 This is a schematic diagram illustrating the synchronization effect in an embodiment of the present invention;

[0030] Figure 10 A flowchart of another method for incremental synchronization of cloud computing resource data provided in an embodiment of the present invention;

[0031] Figure 11 This is a resource change coefficient table in a specific implementation scenario of this invention;

[0032] Figure 12 This is a synchronization task start schedule in a specific implementation scenario of this invention;

[0033] Figure 13 This is a flexible time window table in a specific implementation scenario of this invention;

[0034] Figure 14 A schematic diagram of a device for incremental synchronization of cloud computing resource data provided in an embodiment of the present invention;

[0035] The reference numerals in the attached figures are as follows:

[0036] 11: Processor; 12: Memory. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0038] This invention is an architecture of a specific functional system. Therefore, the specific embodiments mainly describe the functional logic relationship of each structural module, and do not limit the specific software and hardware implementation methods.

[0039] Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0040] Example 1:

[0041] In a cloud computing platform, users manage computing resources through an upper-layer web application (Portal), such as issuing or retrieving cloud hosts from a webpage. When the amount of computing resources in the environment is large, the Portal's direct querying of data through underlying interfaces is likely to cause performance issues, especially in scenarios involving querying related resources. For example, displaying a list of cloud hosts requires querying multiple types of resource data, such as tenants, users, images, specifications, host machines, networks, and virtual network cards. High request-response latency directly impacts user experience.

[0042] Due to performance issues with the aforementioned query methods, the upper-layer Portal stores computing resources in a local database as a secondary storage. This allows most query operations to be completed locally, improving request response efficiency and significantly enhancing the user experience. Based on this secondary resource storage model, to ensure data consistency between the upper and lower layers, the Portal periodically retrieves the underlying resources to the local database via scheduled tasks.

[0043] In this embodiment, in order to improve the efficiency of synchronization in cloud computing, this embodiment provides a method for incremental synchronization of cloud computing resource data, which includes a basic condition construction stage and an incremental synchronization task execution stage.

[0044] like Figure 1 As shown, the specific steps of the cloud computing resource data incremental synchronization method provided in this embodiment of the invention are as follows:

[0045] Step 101: Based on the changed resource quantity and total resource quantity of each type of resource at the time of the last synchronization, calculate the recent change ratio of resources for each type of resource. Based on the start time of the last synchronization task, the resource change coefficient of each type of resource, and the recent change ratio of resources, calculate the data retrieval start time for each type of resource.

[0046] This step is the foundational conditions building phase, which involves defining resource change coefficients, recording the task start time, the number of changed resources and the total amount of resources, calculating the recent resource change ratio, calculating the elastic time window, and preparing for calculating the data retrieval start time.

[0047] To facilitate deployment on existing cloud computing platforms, the incremental synchronization method provided in this embodiment maintains consistency with existing synchronization technologies within the overall process framework, such as... Figure 2As shown, the synchronization task is executed periodically, and a synchronization task is performed once every specified synchronization period time is reached.

[0048] To achieve incremental resource synchronization, it is necessary to model the specific synchronization logic and obtain the time parameters used for incremental synchronization. Based on the typical change characteristics of computing resources, resource change coefficients are defined for each type of resource to represent the frequency of changes caused by the resource's own attributes. Simultaneously, a recent change ratio is defined based on the recent changes of each type of resource to represent the dynamic change frequency caused by recent resource usage. During each synchronization, the data retrieval start time for each type of resource needs to be calculated based on the resource change coefficient and recent change ratio at the time of the last synchronization, indicating the potential start time of changes for that resource.

[0049] Step 102: When the synchronization task starts, the incremental data of each type of resource from the start time of data retrieval to the current time is obtained in sequence according to the resource dependency relationship, and the incremental data of each type of resource is synchronized.

[0050] This step is the incremental synchronization task execution phase. Resources are pulled according to time parameters, resource dependencies are defined, and incremental synchronization is performed on various types of resources in sequence.

[0051] At the start of each synchronization task, dependencies between resources are defined based on the relationships between various computing resource attributes. Changes are then searched for resources after the data retrieval start time to obtain newly added or updated data (incremental data) since the last update. Only incremental data is retrieved and synchronized; unchanged data is not synchronized. This avoids the high CPU and other computing resource utilization and low synchronization efficiency issues caused by full data synchronization.

[0052] After acquiring incremental data sequentially based on dependencies, a timed periodic synchronization task is executed. According to resource dependencies, various resources are synchronized sequentially in a queue. When acquiring a target resource, the corresponding start time is carried, and the effectively changed resources are pulled from the underlying layer, thus completing one incremental resource data synchronization operation. Specifically, dependencies can be determined based on specific cloud computing business logic and network architecture. For example, a cloud host needs to use the host machine's hardware resources and the data in the image for computation; therefore, the cloud host depends on the host machine and the image.

[0053] After steps 101-102 provided in this embodiment, the time interval for pulling synchronization data can be dynamically adjusted according to the characteristics of resource changes. While ensuring the integrity of the synchronization data, compared with the existing full synchronization method, the amount of resources that need to be synchronized is reduced and the synchronization efficiency of resources is improved.

[0054] In step 101, the resource change coefficient value needs to be set according to the typical resource change characteristics in the cloud platform. The more frequently a resource type changes, the larger the resource change coefficient value should be; conversely, the less frequently a resource type changes, the smaller the resource change coefficient value should be. For example, in a specific cloud platform, the resource change coefficient ranges from 0.1 to 1.0. Cloud host and cloud disk resources change frequently, so their resource change coefficient value is set to a larger value, such as 1.0; tenant and user resources do not change frequently, so their resource change coefficient value is set to a smaller value, such as 0.1. Furthermore, to facilitate flexible adaptation to the actual environment, the resource change coefficient can be configured externally to the synchronization program in the form of a file or database. This allows system administrators to flexibly adjust the configuration according to the characteristics of the actual environment to adapt to their actual business needs without modifying the code.

[0055] On the other hand, in step 101, during a specific task execution process, such as Figure 3 As shown, when synchronizing each type of resource, the start time (startTm) is recorded before the synchronization task is executed, and the number of changed resources (changedCount) and the total resource count (totalCount) are recorded after synchronization is completed. These two values ​​can be obtained through the data returned by the underlying interface. In typical cloud platform application scenarios, the execution frequency and cycle of synchronization tasks are not fixed. A certain period of time elapses between the completion of the previous task and the start of the next task; for example, the time interval between the end time of the previous task and the start time of the next task might be 30 seconds. Due to business changes, even for the same type of resource, the number of changed resources and the total resource count may differ in different tasks. After the previous synchronization task is completed, the number of changed resources and the total resource count for each type of resource in the previous synchronization task are obtained. The ratio of the number of changed resources to the total resource count for that type of resource obtained in the previous synchronization task is used as the recent change ratio for that type of resource in the current synchronization. This ratio reflects the recent changes in the resource. Specifically, the recent change ratio is calculated as: the number of changed resources obtained in the previous synchronization task ÷ the total resource count. For example, if there are a total of 1000 cloud servers in the environment, and 20 servers that have changed are pulled this time, then the recent change ratio of cloud server resources is 20 ÷ 1000 = 0.02.

[0056] Furthermore, such as Figure 4As shown, this embodiment proposes the concept of a flexible time window (tmSpan). The flexible time window for each type of resource during the current synchronization is calculated based on the recent change ratio of the resource at the time of the last synchronization. The flexible time window represents a time condition for the synchronization task to pull resource data. This time condition is a time interval dynamically calculated using several factors. These factors include: a fixed maximum time window, a resource change coefficient, and the recent change ratio of the resource calculated based on the data from the last synchronization. Specifically, the product of the fixed maximum time window, the recent change ratio of the resource type in the last synchronization task, and the resource change coefficient of the resource type is used as the value of the flexible time window for the resource type during the current synchronization task. The flexible time window is calculated as "fixed maximum time window" × "resource change coefficient" × "recent change ratio of the resource". The fixed maximum time window can be determined based on actual conditions such as the amount of resources being synchronized. In a specific scenario, after testing, the optimal fixed maximum time window is set to 1 day, or 1440 minutes. According to the above calculation method, for the same type of resource, that is, when the resource change coefficient remains unchanged, the larger the recent change ratio of the resource, the larger the elastic time window, and vice versa. For different types of resources, that is, when the resource change coefficients are different, assuming that the recent change ratio of the resources is the same, the larger the resource change coefficient, the larger the elastic time window, and vice versa. Figure 4 In the diagram, the sum of the dashed and solid boxes for each data type represents the maximum elastic time window of 1440 minutes. The rightmost point of the solid box indicates the start time of the last update. The solid boxes represent the actual calculated elastic time window for each data type: 1126 minutes for cloud hosts, 890 minutes for network cards, and 580 minutes for tenants. Using elastic time windows allows for dynamic adjustments to the resource acquisition timeframe based on recent resource changes, acquiring only resources that have recently undergone changes, ensuring complete resource acquisition while minimizing the amount of synchronized resources.

[0057] In step 102, as Figure 5As shown, the data fetch start time is encapsulated as a time parameter. A data retrieval request carrying this time parameter is sent to the underlying interface via a list query interface. This tells the underlying interface to only return resources that have changed since the last task start time (startTM on the left in the diagram), shifted back one elastic time window (tmSpan in the diagram) to the current time (startTm on the right in the diagram). The underlying interface returns the incremental data that has changed between the data fetch start time and the current time. Specifically, based on the last synchronized task start time of this type of resource, the elastic time window is shifted back to obtain the current synchronized data fetch start time. Specifically, the data fetch start time is calculated as: the last synchronized start time of this resource - the elastic time window calculated this time.

[0058] Based on steps 101-102 and the specific method description above, taking the resource synchronization of a cloud host as an example, such as... Figure 6 As shown, incremental data retrieval can be accomplished through the following process.

[0059] Step 201: Set the resource change ratio of the cloud server to 1.0.

[0060] Step 202: Record the start time of the last cloud server synchronization, startTm. (N-1) It was 2022-08-10 12:10:10.

[0061] Step 203: After the last synchronization of cloud hosts was completed, record the number of changed resources (changedCount) as 20 and the total number of resources (totalCount) as 1000.

[0062] Step 204: Calculate the nearChangedRatio of the cloud server's resources as 20 ÷ 1000 = 0.02.

[0063] Step 205: Set the fixed maximum time window fixedMaxTmSpan to 1440 minutes.

[0064] Step 206: Calculate the elastic time window floatTmSpan of the cloud server as 1440×1.0×0.02=30 (rounded up).

[0065] Step 207: Calculate the start time of data fetch from the cloud server as 2022-08-10 11:40:10.

[0066] Step 208: When fetching incremental data through the underlying cloud server query interface, the following example request can be executed to send the path of the data list to be fetched and the fetch time parameter to the cloud server. The fetch path is represented in the form of a hyperlink:

[0067] http: / / ip:port / nova / server-list? fetchFrom=2022-08-10 11:40:10.

[0068] In cloud platforms, resources have certain dependencies; for example, a cloud host depends on a network interface card (NIC), and a NIC depends on the network. Figure 7 As shown, the dependencies between various resources can be defined based on the business logic of cloud computing. The diagram presents a typical cloud computing use case. The dependencies between devices in the diagram are represented by arrows after being analyzed according to the business logic of the actual scenario. Each resource depends on the resource pointed to by the arrow. After obtaining the incremental resources for each type of resource based on the dynamic data retrieval start time, the incremental data for each type of resource is sequentially retrieved from the underlying layer to the local database according to the order of resource dependencies between different types of resources.

[0069] Furthermore, a queue can be used to manage the order of resource synchronization. Each type of resource requiring synchronization is added to a first-in-first-out queue according to its dependency relationship, and incremental data for each type of resource is synchronized sequentially according to the queue order. For example... Figure 8 As shown, the order of resource dependencies is to synchronize the most basic resources first, and then synchronize the edge resources last. All resource types depend on the lowest-level tenant as the basic resource, and the cloud hosts that depend on other resources are edge resources. Figure 8 Each resource type enters from the "Resource Enqueue" position on the left and depends on the resource type on its right, which is closer to the resource queue exit, "Resource Dequeue Execution Synchronization". The resource type dependency order listed in the figure is as follows: the preceding item depends on the following item: cloud host - network card - subnet - tenant. When performing resource synchronization tasks, the defined dependency order must be followed for synchronization. The synchronization order in the figure is as follows: tenant, subnet, network card, cloud host.

[0070] The incremental data synchronization method for cloud computing resources provided in this embodiment can achieve the following beneficial effects compared with existing synchronization technologies:

[0071] 1) The full synchronization of cloud computing resources is changed to incremental synchronization based on the dynamic ratio of recent resource changes. Each task only synchronizes the incremental data that has actually changed on the basis of existing data, which improves the effectiveness of the task.

[0072] 2) By setting a flexible time window, the time period for fetching resources is made closest to the actual time period of data change, which greatly alleviates the pressure on the underlying service to retrieve resources, reduces the time for the Portal to assemble the result data, and improves the efficiency of resource synchronization, such as... Figure 9 As shown, this improves the overall system performance;

[0073] 3) During the execution of the synchronization task, various resources are synchronized sequentially in the form of a queue according to the dependencies between resources, which ensures the correctness and integrity of the upper-layer resource data.

[0074] Example 2:

[0075] Based on the incremental synchronization method for cloud computing resource data provided in Example 1, this example also provides a specific instance of incremental synchronization of cloud computing resource data in a particular implementation scenario. Taking the incremental synchronization of 10 typical cloud computing resources—cloud hosts, host machines, images, cloud disks, volume snapshots, networks, routers, virtual network cards, tenants, and users—as an example, this example provides a detailed explanation of the incremental synchronization method for cloud computing resource data provided in Example 1 in a real-world scenario. In other implementation scenarios, the method in this scenario can be referenced, and the specific implementation details can be adjusted according to specific needs and business logic, following the method provided in Example 1.

[0076] like Figure 10 As shown, the incremental synchronization method provided in Example 1 can be specifically implemented through the following steps.

[0077] Step 301: Set the resource change coefficient.

[0078] Based on the typical change characteristics of computing resources in the cloud platform, a resource change coefficient is assigned to each resource. In the specific implementation scenario of this embodiment, the resource change coefficient ranges from 0.1 to 1.0. A larger value indicates that the resource changes more frequently, and vice versa. Cloud hosts and cloud disks are primary business resources and are the objects most frequently operated by users, thus changing the most frequently; therefore, their change coefficient is set to 1.0. Host machines, images, volume snapshots, networks, routers, and virtual network cards are supporting or derivative business resources and do not change frequently; therefore, their change coefficient is set to 0.5. Tenants and users are basic management resources and generally do not change for a long time; therefore, their change coefficient is set to 0.1. The specific configuration of the change coefficient for each type of resource is as follows: Figure 11 As shown.

[0079] Step 302: Read the start time, incremental data, and total resources of the previous task record.

[0080] During the execution of each resource synchronization task, the start time is recorded before synchronizing a certain type of resource, and the number of changed resources acquired and the total number of that type of resource in the system are recorded after synchronization of that type of resource is completed. For example: Figure 3 As shown, before synchronizing cloud server resources, the current time is recorded as 2022-08-10 12:10:10. After synchronizing cloud server resources, 20 data entries are recorded. There are currently 1000 cloud servers in the system. After the historical synchronization tasks are completed, the following can be obtained: Figure 12 The record table shown. The current synchronization task will be executed based on this historical data.

[0081] Step 303: Calculate the recent change ratio of resources.

[0082] The recent change ratio of each resource type is calculated based on the changed resource quantity and total resource amount recorded in the previous synchronization task. Recent change ratio = Changed resource quantity ÷ Total resource amount. For example, if the changed number of cloud server resources recorded in the previous synchronization task was 20 and the total resource amount was 1000, then the recent change ratio of the cloud server resources is 0.02. Based on this rule, the recent change ratio of all resources can be calculated.

[0083] Step 304: Calculate the elastic time window and encapsulate the request time parameters.

[0084] Synchronous tasks retrieve data for a specific resource type incrementally based on a time parameter including the start time of the data retrieval. If no query parameters are provided when a synchronous task retrieves data for a specific resource type, the underlying interface will return all resource data of that type. If a specific start time parameter is provided, such as... Figure 5 As shown, this allows for the retrieval of only incremental resource data that has changed. To ensure that the retrieved incremental resources include only those that have recently undergone changes and that no data is missed, the time parameter is derived by shifting the corresponding elastic time window back from the start time of the previous task. For example, if the start time of the last synchronization of the cloud host was 2022-08-10 12:10:10 and the elastic time window was 29 minutes, then the data retrieval start time parameter would be 2022-08-10 11:41:10. This elastic time window has been proven effective in practice. The elastic time window value is the product of the fixed maximum time window, the resource change coefficient, and the recent change ratio of the resource. To ensure the effectiveness of the value, the fixed maximum time window is set to 1 day (i.e., 1440 minutes). In this embodiment, the elastic time window and data retrieval start time for each resource type are as follows: Figure 13 As shown.

[0085] Step 305: Define resource dependencies.

[0086] Dependencies exist between computing resource attributes in a cloud platform. For example, cloud hosts depend on host machines and images, volume snapshots depend on cloud disks, and users depend on tenants. These dependencies between cloud computing resources are explicitly defined here.

[0087] Step 306: Plan the resource synchronization sequence.

[0088] To ensure the correctness and integrity of synchronized resource data, based on the resource dependencies agreed upon in step 305, the following resource synchronization sequence can be planned: Tenant → User → Image → Hypervisor → Instance → Volume → Volume Snapshot → Router → Network → Virtual NIC (Port).

[0089] Step 307: Synchronize each type of resource separately in the form of a queue.

[0090] Following the sequence planned in step 306, synchronization tasks are performed sequentially on various resources using a queue. For example... Figure 8 As shown, according to the order in step 306, the tenant is synchronized first, and the virtual network card is synchronized last.

[0091] As can be seen from the above process, the incremental synchronization method for cloud computing resources provided in Example 1 models the specific logic of the synchronization task and completes two stages: the basic condition construction stage and the incremental synchronization task execution stage. It can flexibly adapt to actual application scenarios, dynamically adjust the resource retrieval time according to resource changes, and obtain the most accurate changed resources.

[0092] Example 3:

[0093] Based on the incremental synchronization method for cloud computing resource data provided in Embodiments 1 and 2 above, the present invention also provides an apparatus for implementing the incremental synchronization of cloud computing resource data using the above method, such as... Figure 14 The diagram shown is a schematic representation of the device architecture according to an embodiment of the present invention. The device for incremental synchronization of cloud computing resource data in this embodiment includes one or more processors 11 and a memory 12. Figure 14 Take a processor 11 as an example.

[0094] Processor 11 and memory 12 can be connected via a bus or other means. Figure 14 Taking the example of a connection between China and Israel via a bus.

[0095] The memory 12 serves as a non-volatile computer-readable storage medium for incremental synchronization of cloud computing resource data. It can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the incremental synchronization methods for cloud computing resource data in Embodiments 1 and 2. The processor 11 executes various functional applications and data processing of the device for incremental synchronization of cloud computing resource data by running the non-volatile software programs, instructions, and modules stored in the memory 12, thereby implementing the incremental synchronization methods for cloud computing resource data in Embodiments 1 and 2.

[0096] Memory 12 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 12 may optionally include memory remotely located relative to processor 11, which can be connected to processor 11 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0097] The program instructions / modules are stored in memory 12. When executed by one or more processors 11, they perform the cloud computing resource data incremental synchronization method described in Embodiments 1 and 2 above, for example, performing the above-described... Figure 1 , Figure 6 and Figure 10 The steps shown.

[0098] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0099] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for incremental synchronization of cloud computing resource data, characterized in that, Specifically, it includes: Based on the changed resource quantity and total resource quantity of each type of resource at the time of the last synchronization, calculate the recent change ratio of resources for each type of resource. Based on the start time of the last synchronization task, the resource change coefficient of each type of resource, and the recent change ratio of resources, calculate the data retrieval start time for each type of resource. When the synchronization task starts, incremental data for each type of resource is retrieved sequentially from the start time of data retrieval to the current time according to resource dependencies, and the incremental data for each type of resource is synchronized. The calculation of the data retrieval start time for each type of resource also includes: The elastic time window for the current synchronization of each type of resource is calculated based on the recent change ratio of the resource at the time of the last synchronization. Based on the task start time of the last synchronization of the resource type, the elastic time window is pushed back to the past, and the resulting time point is the data retrieval start time of the current synchronization.

2. The method for incremental synchronization of cloud computing resource data according to claim 1, characterized in that, The aforementioned recent change ratio of resources specifically includes: After the previous synchronization task is completed, obtain the changed resource quantity and total resource quantity of each type of resource in the previous synchronization task. Use the ratio of the changed resource quantity to the total resource quantity of each type of resource obtained in the previous synchronization task as the recent change ratio of that type of resource in this synchronization.

3. The method for incremental synchronization of cloud computing resource data according to claim 1, characterized in that, The calculation of the elastic time window for each type of resource during this synchronization specifically includes: After the previous synchronization task is completed, the product of the fixed maximum time window, the recent change ratio of the resource type in the previous synchronization task, and the resource change coefficient of the resource type will be used as the value of the elastic time window for the current synchronization task.

4. The method for incremental synchronization of cloud computing resource data according to claim 3, characterized in that, The resource change coefficient specifically includes: The resource change coefficient is set according to the resource's regular change characteristics. The more frequently a resource type changes, the larger the resource change coefficient should be.

5. The method for incremental synchronization of cloud computing resource data according to claim 3, characterized in that, The resource change coefficient also includes: Configure resource change factors outside the synchronization program in the form of files and / or databases.

6. The method for incremental synchronization of cloud computing resource data according to claim 1, characterized in that, The acquisition of incremental data for each type of resource from the start time of data retrieval to the current time specifically includes: The start time of data retrieval is encapsulated as a time parameter, and a data retrieval request carrying the time parameter is sent to the underlying interface through the list query interface. The underlying interface returns the incremental data that has changed between the start time of data retrieval and the current time.

7. The method for incremental synchronization of cloud computing resource data according to claim 1, characterized in that, The synchronization of incremental data for each type of resource specifically includes: In accordance with the order of resource dependencies between different types of resources, incremental data of each type of resource is sequentially pulled from the underlying layer to the local database. The order of resource dependencies is to synchronize the most basic resources first, and then synchronize the edge resources last.

8. The method for incremental synchronization of cloud computing resource data according to claim 1, characterized in that, The synchronization of incremental data for each type of resource also includes: Each type of resource that needs to perform a synchronization task is added to a first-in-first-out queue according to the order of their dependencies, and the incremental data of each type of resource is synchronized sequentially according to the queue order.

9. A device for incremental synchronization of cloud computing resource data, characterized in that: The method includes at least one processor and a memory, which are connected via a data bus. The memory stores instructions that can be executed by the at least one processor. After being executed by the processor, the instructions are used to complete the method for incremental synchronization of cloud computing resource data as described in any one of claims 1-8.

Images