Resource scheduling method, synchronization system, device, storage medium and program product

Abstract

The embodiment of the invention provides a resource scheduling method, a synchronization system, equipment, a storage medium and a program product. The system comprises a central node and a plurality of synchronization nodes, a scheduling component is deployed on the central node, and a bouncing component and a synchronization instance are deployed on each synchronization node; the method comprises the following steps: a scheduling component obtains a future resource demand quantity of a first synchronization node; when the first synchronization node cannot meet the future resource demand of the first synchronization node, the scheduling component determines a to-be-migrated instance from the synchronization instances deployed on the first synchronization node and migrates the to-be-migrated instance to a second synchronization node; the target bouncing component determines a resource capacity expansion demand of the first synchronization instance according to the current resource usage amount of the first synchronization instance; and the target bouncing component performs resource expansion on the first synchronization instance by using the idle resource of the first synchronization node according to the resource expansion demand. According to the technical scheme provided by the embodiment of the invention, the response speed of instance resource capacity expansion can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a resource scheduling method, synchronization system, device, storage medium, and program product. Background Technology

[0002] A data synchronization system is a technical solution for transferring data between different databases. It works by pulling and parsing the logs from the source database in real time, and then writing the parsed content to the target database in real time. The performance of this data synchronization system is not isolated but coupled with the real-time operational status of both the source and target databases.

[0003] To ensure real-time writes, operations and maintenance personnel often allocate excessive resources to the data synchronization system, resulting in resource waste. Even so, in extreme cases, the resources required by the data synchronization system may exceed the resources allocated by the operations and maintenance personnel, causing latency in the data synchronization link. Furthermore, the load of the source database is often related to the service systems providing services to users (e.g., transaction systems, order systems), exhibiting peak and trough characteristics. While allocating resources to the data synchronization system according to peak times ensures real-time writes during peak periods, excessive resources during trough periods also lead to waste. Summary of the Invention

[0004] In view of the above problems, this application is made to provide a resource scheduling method, synchronization system, device, storage medium and program product that solves or at least partially solves the above problems.

[0005] A first aspect of this application provides a resource scheduling method for a data synchronization system, characterized in that the data synchronization system includes a central node and multiple synchronization nodes, the central node is equipped with a scheduling component, and each synchronization node is equipped with a boosting component and a synchronization instance for performing data synchronization tasks.

[0006] The scheduling component obtains the future resource requirements of the first synchronization node among the plurality of synchronization nodes; the future resource requirements of the first synchronization node are predicted based on the historical resource usage of the synchronization instances deployed on the first synchronization node;

[0007] When the resources of the first synchronization node cannot meet the future resource requirements of the first synchronization node, the scheduling component determines the instance to be migrated from the synchronization instances deployed on the first synchronization node and migrates the instance to be migrated to the second synchronization node among the plurality of synchronization nodes; the resources of the second synchronization node meet the future resource requirements of the second synchronization node and the future resource requirements of the instance to be migrated.

[0008] The target escalation component determines the resource expansion requirements of the first synchronization instance based on the current resource usage of the first synchronization instance. The target escalation component refers to the escalation component deployed on the first synchronization node, and the first synchronization instance refers to the synchronization instance currently deployed on the first synchronization node.

[0009] The target boosting component expands the resources of the first synchronization instance by utilizing the idle resources of the first synchronization node according to the resource expansion requirements.

[0010] Optionally, the data synchronization system may also include a control node.

[0011] A second aspect of this application provides a data synchronization system, comprising a central node and multiple synchronization nodes. The central node is equipped with a scheduling component, and each synchronization node is equipped with a spurious component and a synchronization instance for performing data synchronization tasks.

[0012] The scheduling component is configured to: obtain the future resource requirements of a first synchronization node among the plurality of synchronization nodes; the future resource requirements of the first synchronization node are predicted based on the historical resource usage of the synchronization instances deployed on the first synchronization node; when the resources of the first synchronization node cannot meet the future resource requirements of the first synchronization node, determine an instance to be migrated from the synchronization instances deployed on the first synchronization node and migrate the instance to be migrated to a second synchronization node among the plurality of synchronization nodes; the resources of the second synchronization node meet the future resource requirements of the second synchronization node and the future resource requirements of the instance to be migrated;

[0013] The target expansion component is used to: determine the resource expansion requirements of the first synchronization instance based on the current resource usage of the first synchronization instance, wherein the target expansion component refers to the expansion component deployed on the first synchronization node, and the first synchronization instance refers to the synchronization instance currently deployed on the first synchronization node; and expand the resources of the first synchronization instance using the idle resources of the first synchronization node according to the resource expansion requirements.

[0014] A third aspect of this application provides an electronic device. The electronic device includes: a memory and a processor, wherein,

[0015] The memory is used to store programs;

[0016] The processor, coupled to the memory, is configured to execute the program stored in the memory to implement the method described in any of the preceding embodiments.

[0017] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a computer, can implement the method described in any of the preceding claims.

[0018] A fifth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the preceding claims.

[0019] In the technical solution provided in this application embodiment, each synchronization node is equipped with a pop-up component, which is responsible for expanding the resources of the synchronization instances on that node. A scheduling component is deployed on the central node. Based on the future resource requirements of each synchronization node, the scheduling component pre-executes cross-node migrations of instances to ensure that most synchronization nodes can utilize their own resources to complete the resource expansion of synchronization instances within a certain period, thereby reducing the need for cross-node resource expansion. When the pop-up component can utilize the resources of its own node to complete the resource expansion of synchronization instances, the response speed and real-time performance of resource expansion can be effectively improved. Furthermore, when a pop-up component on a synchronization node malfunctions, it only affects the expansion tasks on that synchronization node and does not affect the expansion tasks on other synchronization nodes, resulting in a relatively small impact. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the structure of a data synchronization system provided in an embodiment of this application;

[0022] Figure 2 A flowchart illustrating a resource scheduling method provided in an embodiment of this application;

[0023] Figure 3 A schematic diagram of the local jump-up process of a serverless data synchronization system from a standalone perspective, provided in an embodiment of this application;

[0024] Figure 4 A schematic diagram of the remote escalation process of a serverless data synchronization system from a cluster perspective, provided in an embodiment of this application;

[0025] Figure 5 A schematic diagram of the timing prediction process of a serverless data synchronization system from a cluster perspective, provided in an embodiment of this application;

[0026] Figure 6 A schematic diagram of the cluster resource management process of a serverless data synchronization system from a cluster perspective, provided as an embodiment of this application;

[0027] Figure 7 A schematic diagram of the metering and billing process of a serverless data synchronization system from a cluster perspective, provided in an embodiment of this application;

[0028] Figure 8 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0030] Furthermore, some processes described in the specification, claims, and accompanying drawings of this application include multiple operations that appear in a specific order. These operations may be performed out of order or in parallel. Operation numbers such as 201, 202, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be performed sequentially or in parallel. It should be noted that the terms "first," "second," etc., used herein are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.

[0031] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0032] First, the terminology used in the embodiments of this application will be explained. It should be understood that this explanation is for the purpose of making the embodiments of this application clearer and does not necessarily constitute a limitation on the embodiments of this application.

[0033] Serverless Architecture: This is a cloud computing deployment model in which users do not need to worry about the specifications, load, high availability, and other usage details of the server running the program. They only need to pay for the service according to their usage.

[0034] Example: In serverless architecture, the concept of "instance" primarily refers to a running instance of a Function as a Service (FaaS). Specifically, when a user deploys a function on a serverless platform, an instance of that function is created when it needs to be executed, and this instance handles the corresponding task. To avoid resource waste, idle instances are destroyed. In some embodiments of this application, "instance" refers to a container group (Pod).

[0035] The resource scheduling method provided in this application is applied to a data synchronization system. For example, the data synchronization system may be a serverless architecture. Figure 1 As shown, the data synchronization system 100 may include a central node 11 and multiple synchronization nodes 12 that are communicatively connected to the central node 11. The central node 11 is equipped with a scheduling component 110, and each synchronization node 12 is equipped with a boosting component 120 and a synchronization instance 121 for performing data synchronization tasks. For example, the central node 11 and the multiple synchronization nodes 12 may constitute a data synchronization cluster 1.

[0036] Optionally, the scheduling component 110 deployed on the central node 11 is used to perform cross-node migration of instances in advance based on the future resource requirements of each synchronization node, so as to ensure that most synchronization nodes can use their idle resources to complete the resource expansion of synchronization instances within a certain period of time (e.g., the next day), thereby reducing the situation of cross-node resource expansion of synchronization instances, which helps to improve the response speed and real-time performance of resource expansion.

[0037] Optionally, the future resource requirements of a synchronization node are predicted based on the historical resource usage of one or more synchronization instances deployed on that synchronization node. The specific prediction method will be described in the following embodiments.

[0038] Optionally, the escalation component 120 deployed on each synchronization node 12 is responsible for scaling up and down the resources of the synchronization instances on that node (i.e., the synchronization node where the escalation component resides). Scaling up refers to increasing the resource allocation for a synchronization instance, while scaling down refers to decreasing the resource allocation for a synchronization instance. Resource allocation refers to the amount of resources allocated to a synchronization instance. The resource allocation for each synchronization instance defines the upper limit of the resources that the synchronization instance can use. In this application example, resources refer to computing resources and / or storage resources. Computing resources can be memory resources, central processing unit (CPU) resources, etc.

[0039] Optionally, each synchronization node 12 may deploy one or more synchronization instances 121, with different synchronization instances responsible for different data synchronization tasks. For example, different synchronization instances may be responsible for data synchronization tasks for different users. Here, a data synchronization task refers to synchronizing data from the source database to the destination database.

[0040] Optional, such as Figure 2 As shown, the above method may include the following steps:

[0041] 201. The scheduling component obtains the future resource requirements of the first synchronization node among the plurality of synchronization nodes.

[0042] The first synchronization node refers to any one of these multiple synchronization nodes.

[0043] The future resource requirements of the first synchronization node are predicted based on the historical resource usage of the synchronization instances deployed on the first synchronization node.

[0044] For example, there are N synchronization instances deployed on the first synchronization node, where N is an integer greater than or equal to 1.

[0045] Optionally, the future resource requirements of each of the N synchronization instances are predicted based on their historical resource usage. In one optional implementation, the historical resource usage of a synchronization instance includes: resource usage at multiple historical time points. The resource usage at these historical time points is sorted chronologically to obtain a time series. Based on the time series, a time series prediction algorithm is used to predict the future resource requirements of the synchronization instance. The future resource requirements of a synchronization instance can be understood as the amount of resource allocation needed by the synchronization instance in the future. The future resource requirements of a synchronization instance may include the resource requirements of the synchronization instance at future time points, i.e., the amount of resource allocation needed by the synchronization instance at future time points. The future resource requirements of a synchronization instance may include the resource requirements of the synchronization instance at Q future time points, where Q is an integer greater than or equal to 1.

[0046] Time series forecasting algorithms are methods for predicting and analyzing future trends based on historical statistical data over time series. Their basic principles include acknowledging the continuity of development and using past time series data for statistical analysis to infer future trends; simultaneously, considering the randomness caused by accidental factors, they use historical data for statistical analysis and appropriate processing to make trend predictions.

[0047] Time series forecasting algorithms can be divided into traditional methods and deep learning methods. Traditional methods include, but are not limited to, moving averages, exponential smoothing, and autoregressive integrated moving average models (ARIMA). Deep learning methods utilize neural network models for advanced pattern recognition and automatic feature extraction. Commonly used models include recurrent neural networks (RNNs), long short-term memory networks (LSTMs), gated recurrent units (GRUs), and convolutional neural networks (CNNs).

[0048] Optionally, the future resource requirement of the first synchronization node refers to the total future resource requirement of N synchronization instances. The total future resource requirement of N synchronization instances is the sum of the future resource requirements of all N synchronization instances. The future resource requirement of the first synchronization node can also be understood as the total amount of resources required by N synchronization instances in the future.

[0049] Optionally, the prediction process for the future resource demand of the first synchronization node can be implemented by the central node, for example, by the scheduling component or other components on the central node.

[0050] 202. When the resources of the first synchronization node cannot meet the future resource requirements of the first synchronization node, the scheduling component determines the instance to be migrated from the synchronization instances deployed on the first synchronization node and migrates the instance to be migrated to the second synchronization node among the plurality of synchronization nodes.

[0051] For example, the future resource requirements of the first synchronization node include the resource requirements of the first synchronization node at Q future time points. The resource requirement of the first synchronization node at the q-th future time point is the sum of the resource requirements of the N synchronization instances at the q-th future time point, where q is an integer and its value ranges from 1 to Q. The Q future time points are located within a preset future time period, such as one day in the future.

[0052] Optionally, if the resource demand of the first synchronization node at any of the Q future time points exceeds the total resources available for allocation by the first synchronization node, it is determined that the resources of the first synchronization node cannot meet the future resource demand of the first synchronization node. If the resource demand of the first synchronization node at any of the Q future time points does not exceed the total resources available for allocation by the first synchronization node, it is determined that the resources of the first synchronization node meet the future resource demand of the first synchronization node.

[0053] The total resources allocated to the first synchronization node refer to the total amount of resources pre-configured on the first synchronization node for allocation to synchronization instances. In practical applications, the first synchronization node includes not only synchronization instances but also target pop-up components, which also require certain resources to operate. Therefore, the total resources allocated to the first synchronization node are generally less than the total resources of the first synchronization node itself.

[0054] When the first synchronization node has T resource demands among Q future time points that are greater than the total resources available for allocation by the first synchronization node, the earliest of these T future time points corresponding to these T resource demands is determined as the time point when remote boosting is required. Here, T is an integer greater than or equal to 1.

[0055] When the resources of the first synchronization node cannot meet the future resource requirements of the first synchronization node, the scheduling component can determine the instance to be migrated from the above N synchronization instances.

[0056] For example, a synchronization instance can be randomly selected from N synchronization instances to be determined as the instance to be migrated.

[0057] It should be noted that after randomly selecting a synchronization instance and completing the migration, if the resources of the first synchronization node are still insufficient to meet the total future resource requirements of the remaining synchronization instances, new instances to be migrated can be identified from the remaining synchronization instances and the migration can be performed until the resources of the first synchronization node meet the total future resource requirements of the remaining synchronization instances.

[0058] For example, the synchronization instance with the highest future resource demand among N synchronization instances can be identified as the instance to be migrated. In this way, after migrating one instance to be migrated, the resources of the first synchronization node may be able to meet the total future resource demand of the remaining synchronization instances, which can reduce the number of instances to be migrated and thus reduce the impact on the data synchronization task.

[0059] The resources of the second synchronization node meet the future resource requirements of both the second synchronization node and the future resource requirements of the instance to be migrated.

[0060] For example, the total future resource requirement of the second synchronization node is determined based on the future resource requirement of the second synchronization node and the future resource requirement of the instance to be migrated. The total future resource requirement of the second synchronization node is obtained by summing the future resource requirement of the second synchronization node and the future resource requirement of the instance to be migrated. The total future resource requirement of the second synchronization node includes the total resource requirement of the second synchronization node at Q future time points, where the total resource requirement of the second synchronization node at the q-th future time point is the sum of the resource requirement of the second synchronization node at the q-th future time point and the resource requirement of the instance to be migrated at the q-th future time point.

[0061] For example, if the total resource demand of the second synchronization node at Q future time points exceeds the total resource allocation available to the second synchronization node, it indicates that the resources of the second synchronization node do not meet the future resource demands of both the second synchronization node and the instance to be migrated. Conversely, if the total resource demand of the second synchronization node at Q future time points does not exceed the total resource allocation available to the second synchronization node, it indicates that the resources of the second synchronization node meet the future resource demands of both the second synchronization node and the instance to be migrated.

[0062] The calculation method for the future resource requirements of the second synchronization node can refer to the calculation method for the future resource requirements of the first synchronization node, and will not be repeated here.

[0063] Optionally, the scheduling component deletes the instance to be migrated on the first synchronization node and creates the instance to be migrated on the second synchronization node, thereby completing the migration of the instance to be migrated. For example, the scheduling component can create the instance to be migrated on the second synchronization node according to the resource allocation of the instance to be migrated before the migration, that is, the resource allocation of the recreated instance to be migrated is the same as the resource allocation of the instance to be migrated before the migration.

[0064] It should be noted that the instance to be migrated cannot perform data synchronization tasks during the migration process; that is, the migration of the instance to be migrated will affect the execution of data synchronization tasks.

[0065] To minimize the impact on data synchronization tasks, the migration of instances to be migrated can be performed during off-peak hours. During off-peak hours, the data synchronization traffic of the data synchronization cluster is less than a preset traffic threshold. The size of this preset traffic threshold can be set according to actual needs, and this application does not specifically limit it. Optionally, steps 201 and 202 above can be performed during off-peak hours. For example, steps 201 and 202 above can be performed during off-peak hours before the time point when a remote rollback is required in the future. Generally, off-peak hours are from 2:00 AM to 4:00 AM every day; therefore, steps 201 and 202 above can be performed at 2:00 AM every day. That is to say, steps 201 and 202 above can be performed on a scheduled basis.

[0066] Optional, such as Figure 2 As shown, the method also includes the following steps:

[0067] 203. The target boosting component determines the resource expansion requirements of the first synchronization instance based on the current resource usage of the first synchronization instance.

[0068] Among them, expansion is also known as boosting, and the demand for resource expansion can also be called the demand for resource boosting.

[0069] The target lift component refers to the lift component currently deployed on the first synchronization node, and the first synchronization instance refers to the synchronization instance deployed on the first synchronization node.

[0070] The first synchronization node can have multiple synchronization instances deployed (e.g., N instances), and the first synchronization instance refers to any one of these N synchronization instances.

[0071] The target lift component collects the current running data of the first synchronization instance, determines the current resource usage of the first synchronization instance based on the current running data, and determines the resource expansion requirements of the first synchronization instance based on the current resource usage.

[0072] The current resource usage of the first synchronization instance can be the average resource usage of the first synchronization instance within the current time window (e.g., 10 seconds).

[0073] For example, when the ratio between the current resource usage of the first synchronization instance and the current resource allocation of the first synchronization instance is greater than or equal to a first preset threshold (e.g., 90%), it indicates that the first synchronization instance needs to expand its resources. At this time, the amount of additional resource allocation required for the first synchronization instance can be determined, and the resource expansion requirement of the first synchronization instance can be determined based on the amount of additional resource allocation required for the first synchronization instance.

[0074] The amount of additional resource allocation required for the first synchronization instance can be set according to actual needs, and this application embodiment does not impose specific limitations on this. In one example, the amount of additional resource allocation required for the first synchronization instance is determined to be the default configured resource amount. In another example, the amount of additional resource allocation required for each synchronization instance can be determined based on the current resource allocation of that synchronization instance. For example, the product of the current resource allocation of the synchronization instance and a preset ratio can be used to determine the amount of additional resource allocation required for the synchronization instance. The preset ratio can be set according to actual needs, such as 10% or 20%, and this application does not impose specific limitations on this.

[0075] The resource expansion requirements for the first synchronization instance may include the amount of additional resource allocation needed for the first synchronization instance.

[0076] For example, when the ratio between the current resource usage of the first synchronization instance and the current resource allocation of the first synchronization instance is less than a first preset threshold, it is determined that the first synchronization instance does not have a resource expansion requirement.

[0077] 204. The target lifting component expands the resources of the first synchronization instance by utilizing the idle resources of the first synchronization node according to the resource expansion requirement.

[0078] The idle resources of the first synchronization node refer to the resources on the first synchronization node that have not yet been allocated. This can be understood as resources that are pre-configured on the first synchronization node for allocation but have not yet been allocated to any synchronization instance.

[0079] For example: The first synchronization node has 500GB of memory, of which 450GB is allocated to synchronization instances and 50GB is allocated to the target pop-up component. Three synchronization instances, A, B, and C, are deployed on the first synchronization node. A currently has a resource allocation of 100, B has a resource allocation of 50, and C has a resource allocation of 100. Therefore, the first synchronization node has 200GB of free resources.

[0080] The target expansion component determines the amount of additional resource allocation required for the first synchronization instance based on the resource expansion needs. Then, based on this additional resource allocation, the target expansion component utilizes the idle resources of the first synchronization node to increase the resource allocation for the first synchronization instance. The resource allocation for the first synchronization instance after the addition is the sum of the resource allocation before the addition of the first synchronization node and the additional resource allocation required for the first synchronization instance.

[0081] In the technical solution provided in this application embodiment, each synchronization node is equipped with a pop-up component, which is responsible for expanding the resources of the synchronization instances on that node. A scheduling component is deployed on the central node. Based on the future resource requirements of each synchronization node, the scheduling component pre-executes cross-node migrations of instances to ensure that most synchronization nodes can utilize their own resources to complete the resource expansion of synchronization instances within a certain period, thereby reducing the need for cross-node resource expansion. When the pop-up component can utilize the resources of its own node to complete the resource expansion of synchronization instances, the response speed and real-time performance of resource expansion can be effectively improved. Furthermore, when a pop-up component on a synchronization node malfunctions, it only affects the expansion tasks on that synchronization node and does not affect the expansion tasks on other synchronization nodes, resulting in a relatively small impact.

[0082] In one optional implementation, when the ratio between the current resource usage and the current resource allocation of the first synchronization instance is less than a second preset threshold (e.g., 50%), it indicates that the first synchronization instance has a resource scaling-down requirement. Scaling-down is also known as a rollback, and the resource scaling-down requirement can be referred to as a resource rollback requirement. At this time, the resource scaling-down requirement of the first synchronization instance can be determined. The target rollback component scales down the resources of the first synchronization instance according to the resource scaling-down requirement. The resource scaling-down requirement may include the amount of resource allocation that the first synchronization instance needs to reduce. The resource allocation of the first synchronization instance after scaling-down is the difference between the resource allocation of the first synchronization instance before scaling-down and the amount of resource allocation that the first synchronization instance needs to reduce.

[0083] The second preset threshold is less than the first preset threshold.

[0084] The amount of resource allocation reduction required for the first synchronization instance can be set according to actual needs, and this application embodiment does not impose a specific limitation on this. The amount of resource allocation reduction required for each synchronization instance can be determined based on the current resource allocation of the synchronization instance. For example, the product of the current resource allocation of the synchronization instance and a preset ratio can be used to determine the amount of resource allocation reduction required for the synchronization instance. For example, the preset ratio can be 10% or 20%, and this application does not impose a specific limitation on this.

[0085] In this embodiment, when the scaling-up component completes the resource scaling-down of the synchronization instance on this node, it can effectively improve the response speed and real-time performance of the resource scaling-down. Timely scaling-down can release unused resources for use by other instances, thereby improving resource utilization.

[0086] In an optional implementation, step 204 above, "the target elevation component utilizes the idle resources of the first synchronization node to expand the resource capacity of the first synchronization instance," may include:

[0087] 2041. If the idle resources of the first synchronization node meet the resource expansion requirements, the target boosting component uses the idle resources of the first synchronization node to expand the resources of the first synchronization instance.

[0088] Optionally, the resource expansion requirement includes the additional resource allocation required by the first synchronization instance. The target boosting component can determine whether the idle resources of the first synchronization node meet the resource expansion requirement based on the comparison between the amount of idle resources of the first synchronization node and the additional resource allocation required by the first synchronization instance.

[0089] For example, when the amount of idle resources of the first synchronization node is greater than or equal to the amount of additional resources required by the first synchronization instance, it is determined that the idle resources of the first synchronization node meet the resource expansion requirements.

[0090] For example, when the amount of idle resources of the first synchronization node is less than the amount of additional resources required by the first synchronization instance, it is determined that the idle resources of the first synchronization node do not meet the resource expansion requirements.

[0091] In this embodiment, the springback component can utilize the resources of this node to complete the resource expansion of the synchronization instance, which can effectively improve the response speed of the resource expansion of the synchronization instance and improve the real-time performance of the resource expansion.

[0092] The following will combine Figure 3 This section introduces the local rollout process of a serverless data synchronization system from a single-machine perspective:

[0093] like Figure 3 As shown, synchronization node 31 is equipped with a pop-up component 311, a synchronization instance 312, and a synchronization instance 313. The pop-up component 311 comprises three modules: a perception module, a decision-making module, and an execution module. The perception module obtains second-level operational data (corresponding to...) from the synchronization instance 312 in real time. Figure 3 S31 in the context refers to second-level operational information, which is further processed by the sensor module, such as calculating the average and peak values ​​within a window. The decision module acquires the processed data from the sensor module in real time and performs second-level decision-making (corresponding to...). Figure 3 If the S32 rule is met (e.g., CPU utilization reaches a 90% threshold within a 10-second window), the execution module will be invoked to dynamically expand resources (corresponding to...). Figure 3 (S33). The execution module responds to the call from the decision module, dynamically changing the resource limits of synchronization instance 312, such as CPU, memory, and IO (Input / Output). Resource changes take effect within seconds (corresponding to...). Figure 3 (S34). Among them, resource constraints are also known as resource allocation.

[0094] It should be noted that there are multiple ways to modify resource limits. For example, when the synchronization instance uses Linux source control groups for resource isolation, resource limits can be changed by modifying the source control group file corresponding to synchronization instance 312. For example, the synchronization instance can also provide an interface for dynamically modifying resource limits, which is configured to change resource limits within 1 second for execution module scheduling.

[0095] Optionally, the above method may also include the following steps:

[0096] 205. If the idle resources of the first synchronization node do not meet the resource expansion requirements, the target escalation component sends a resource shortage event to the scheduling component.

[0097] The resource shortage event carries instance information of the first synchronization instance. This instance information is used to uniquely identify the first synchronization instance, and may include, for example, the instance name, instance ID, etc.

[0098] 206. After receiving the resource shortage event, the scheduling component obtains the current resource requirement of the third synchronization node among the multiple synchronization nodes and the future resource requirement of the third synchronization node.

[0099] For example, the third synchronization node has M synchronization instances deployed, where M is an integer greater than or equal to 1. The current resource requirement of the third synchronization node is the sum of the current resource requirements of the M synchronization instances.

[0100] The current resource requirement of each synchronization instance is determined based on the current resource usage of that synchronization instance.

[0101] For example, when the ratio between the current resource usage of the synchronization instance and the current resource allocation of the synchronization instance is greater than or equal to a first preset threshold (e.g., 90%), the required additional resource allocation for the synchronization instance is determined, and the sum of the current resource allocation of the synchronization instance and the required additional resource allocation for the synchronization instance is determined as the current resource requirement of the synchronization instance.

[0102] For example, when the ratio between the current resource usage of the synchronization instance and the current resource allocation of the synchronization instance is less than a second preset threshold (e.g., 50%), the amount of resource allocation that the synchronization instance needs to reduce is determined, and the difference between the current resource allocation of the synchronization instance and the amount of resource allocation that the synchronization instance needs to reduce is determined as the current resource requirement of the synchronization instance.

[0103] The process of determining the future resource requirements of the third synchronization node can refer to the process of determining the future resource requirements of the first synchronization node in the above embodiments, and will not be repeated in this embodiment.

[0104] 207. The scheduling component determines whether the third synchronization node currently and in the future meets the receiving conditions for receiving the first synchronization instance based on the current resource demand and the future resource demand of the third synchronization node.

[0105] For example, when the total resources allocated by the third synchronization node are greater than or equal to the combined current resource requirements of the third synchronization node and the first synchronization instance, it is determined that the resources of the third synchronization node meet the combined current resource requirements of the third synchronization node and the first synchronization instance; otherwise, it is determined that the resources of the third synchronization node do not meet the combined current resource requirements of the third synchronization node and the first synchronization instance. This total requirement is the sum of the current resource requirements of the third synchronization node and the first synchronization instance.

[0106] When the resources of the third synchronization node meet the total resource requirements of the third synchronization node and the first synchronization instance, it is determined that the third synchronization node currently meets the receiving conditions.

[0107] When the resources of the third synchronization node do not meet the total resource requirements of the third synchronization node and the first synchronization instance, it is determined that the third synchronization node does not currently meet the receiving conditions.

[0108] For example, the total future resource requirement of the third synchronization node is determined based on the future resource requirement of the third synchronization node and the future resource requirement of the first synchronization instance. The total future resource requirement of the third synchronization node is obtained by summing the future resource requirements of the third synchronization node and the first synchronization instance. The total future resource requirement of the third synchronization node includes the total resource requirement of the third synchronization node at Q future time points, where the total resource requirement of the third synchronization node at the q-th future time point is the sum of the resource requirements of the third synchronization node at the q-th future time point and the resource requirements of the first synchronization instance at the q-th future time point.

[0109] For example, if the total resource demand of the third synchronization node at Q future time points exceeds the total resource allocation available to the third synchronization node, it indicates that the resources of the third synchronization node do not meet the future resource demands of both the third synchronization node and the first synchronization instance. Conversely, if the total resource demand of the third synchronization node at Q future time points does not exceed the total resource allocation available to the third synchronization node, it indicates that the resources of the third synchronization node meet the future resource demands of both the third synchronization node and the first synchronization instance.

[0110] When the resources of the third synchronization node meet both the future resource requirements of the third synchronization node and the future resource requirements of the first synchronization instance, it is determined that the third synchronization node will meet the receiving conditions in the future; wherein, the future resource requirements of the first synchronization instance are predicted based on the historical resource usage of the first synchronization instance. The prediction process can refer to the prediction process of the future resource requirements of the synchronization instance described in the above embodiments, and will not be repeated here.

[0111] When the resources of the third synchronization node do not meet the future resource requirements of the third synchronization node and the future resource requirements of the first synchronization instance, it is determined that the third synchronization node will not meet the receiving conditions in the future.

[0112] 208. When the third synchronization node currently and in the future meets the receiving conditions, the scheduling component migrates the first synchronization instance to the third synchronization node.

[0113] When the third synchronization node currently meets the receiving conditions and will not meet the receiving conditions in the future, the scheduling component will migrate the first synchronization instance to the third synchronization node. The migration process can be referred to the migration process of the first synchronization instance from the first synchronization node to the second synchronization node in the above embodiments, and will not be repeated here.

[0114] In this embodiment, if the escalation component cannot utilize the resources of this node to complete the resource expansion of the synchronization instance, remote escalation is used as a fallback to ensure availability in extreme cases.

[0115] It should be noted that there may be multiple third synchronization nodes in the data synchronization cluster that currently and in the future meet the above receiving conditions. In one example, one of these third synchronization nodes can be randomly selected as the target synchronization node for receiving the first synchronization instance. In another example, these third synchronization nodes can be sorted from highest to lowest according to their current resource demand level, and the third synchronization node with the lowest ranking can be selected as the target synchronization node for receiving the first synchronization instance. The current resource demand level of a third synchronization node is the ratio between its current resource demand and its total resources. In yet another example, the future resource demand level of a third synchronization node is determined, and the future resource demand level of a third synchronization node is the ratio between its future resource demand and its total resources. The current resource demand level and the future resource demand level of a third synchronization node are weighted and summed to obtain the target resource demand level of the third synchronization node. The multiple third synchronization nodes are then sorted according to their target resource demand levels, and the third synchronization node with the lowest ranking is selected as the target synchronization node for receiving the first synchronization instance.

[0116] The following will combine Figure 4 This section introduces the remote boosting process of a serverless data synchronization system from a cluster perspective:

[0117] like Figure 4 As shown, the data synchronization cluster includes a central node 41, synchronization nodes 42, and synchronization nodes 43. Synchronization node 41 is equipped with a synchronization instance 421 and a pop-up component 422, where the pop-up component 422 includes a perception module and a decision module. The central node 41 is equipped with a scheduling component 411, which includes a remote decision module and a scheduling module. The scheduling component 411 handles situations where the local decision module cannot execute. The pop-up component 422 obtains second-level runtime data (corresponding to...) from the synchronization instance 421 in real time. Figure 4 In S41), the perception module processes the second-level operational data, and the decision-making module makes second-level decisions based on the processed data (corresponding to...). Figure 4 In step S42), when the idle resources on synchronization node 42 cannot meet the resource expansion requirements of synchronization instance 421, the decision module reports a resource shortage event to the remote decision module in scheduling component 411 (corresponding to...). Figure 4 After the remote decision-making module verifies that there are no errors, it will call the data synchronization cluster scheduling module to reschedule the synchronization instance (corresponding to S43). Figure 4(S44). The scheduling module will schedule synchronization instance 421 based on the current resource usage and historical resource usage of nodes within the cluster (corresponding to S44). Figure 4 (S45). After synchronization instance 421 is scheduled to the new data synchronization node 43, it is taken over by the escalation component 431 on the new synchronization node 43. Figure 4 S45 includes two steps: deleting synchronization instance 421 on synchronization node 42 and creating synchronization instance 421 on synchronization node 43.

[0118] In the above embodiments, the future resource requirements of the synchronization node and the future resource requirements of the synchronization instance can be determined by the central node 11.

[0119] As an option, such as Figure 1 As shown, the data synchronization system further includes a management cluster 2, which includes management nodes 21. The number of management nodes 21 can be one or more. In this embodiment, the management nodes 21 can determine the future resource requirements of the synchronization nodes and the future resource requirements of the synchronization instances in the data synchronization cluster. The determination process can refer to the corresponding content in the above embodiments, and will not be repeated here.

[0120] Optionally, the above method may also include the following steps:

[0121] 209. The elastic component collects the resource usage of the synchronous instances deployed on this node.

[0122] The pop-up component can collect resource usage data of synchronization instances deployed on this node at preset time intervals. The preset time interval can be set according to actual needs, for example, 1 second or 2 seconds. A shorter preset time interval results in a heavier data collection task for the pop-up component, but higher accuracy in subsequent calculations based on the collected resource usage data.

[0123] 210. The escalation component sends the resource usage of the synchronization instance deployed on this node to the control node.

[0124] In this embodiment, the control node is used to predict future data, and its real-time requirements are not high. Therefore, the escalation component can send the resource usage of synchronization instances deployed on its own node to the control node in batches. This reduces network interaction between the escalation component and the control node, saving bandwidth resources. For example, the escalation component can send the resource usage of synchronization instances deployed on its own node collected within the current collection window (e.g., 1 minute, 5 minutes) to the control node. The duration of the current collection window is longer than a preset time interval.

[0125] 211. Control the resource usage sent by the storage boosting component of the control node.

[0126] The control node can store the resource usage data sent by the escalation component in an offline data warehouse, and retrieve it from the offline data warehouse when needed later.

[0127] In this embodiment, the control node is responsible for collecting and storing historical data, which reduces the load on the scheduling component and thus ensures the remote boosting performance of the scheduling component.

[0128] Optionally, the above method further includes the following steps:

[0129] 209. The control node obtains the historical resource usage of the synchronization instance deployed on the first synchronization node.

[0130] The control node obtains the historical resource usage of each of the N synchronization instances mentioned above. The historical resource usage of a synchronization instance includes the resource usage of that synchronization instance at multiple historical points in time.

[0131] For example, the control node can obtain the historical resource usage of each of the above N synchronization instances from the offline data warehouse.

[0132] 210. The control node predicts the future resource requirements of the synchronization instances deployed on the first synchronization node based on the historical resource usage of the synchronization instances deployed on the first synchronization node.

[0133] In one optional implementation, the historical resource usage includes: resource usage at multiple historical time points. The control node sorts the resource usage at multiple historical time points according to their chronological order to obtain a time series. Based on the time series, the control node uses a time series prediction algorithm to predict the future resource demand of the synchronization instance.

[0134] For example, the control node can input time series data into a time series prediction algorithm, which will then output the future resource requirements of the synchronization instance.

[0135] The future resource requirements of a synchronization instance include the resource requirements of the synchronization instance at one or more future points in time.

[0136] 211. The control node determines the future resource requirements of the first synchronization node based on the future resource requirements of the synchronization instances deployed on the first synchronization node.

[0137] For example, the sum of the future resource requirements of the N synchronization instances deployed on the first synchronization node is determined as the future resource requirement of the first synchronization node.

[0138] Optionally, in section 201 above, "the scheduling component obtains the future resource requirements of the first synchronization node among the plurality of synchronization nodes" includes:

[0139] 2011. The scheduling component sends an acquisition request to the control node.

[0140] The request is used to request the future resource requirements of the first synchronization node. The request may carry node information of the first synchronization node, which is used to uniquely identify the first synchronization node.

[0141] 2012. The scheduling component receives the future resource requirements of the first synchronization node returned by the control node.

[0142] After receiving the acquisition request, the control node determines the first synchronization node based on the node information carried in the acquisition request and returns the future resource requirements of the first synchronization node to the scheduling component.

[0143] In this embodiment, the control node is responsible for predicting future data, which reduces the load on the scheduling component and thus ensures the remote boosting performance of the scheduling component.

[0144] The following is combined Figure 5 This paper introduces the time-series prediction process of a serverless data synchronization system from a cluster perspective:

[0145] In practical applications, data synchronization systems involve multiple regions, each including one or more data synchronization clusters. The data synchronization system includes a management cluster, which contains multiple management nodes. Each region corresponds to one management node, which is used to manage one or more data synchronization clusters within that region.

[0146] like Figure 5 As shown, the data synchronization cluster within Region 1 includes synchronization node 51, synchronization node 52, and central node 53. Synchronization node 51 is equipped with synchronization instance 511 and a boosting component 512. Central node 53 is equipped with scheduling component 531, which includes a scheduling module. The control node 54, responsible for managing Region 1, is equipped with a data aggregation module, an offline data warehouse, and a time-series prediction module.

[0147] like Figure 5 As shown, the sensing module of the lifting component 512 obtains second-level operating data from the synchronization instance 511 in real time (corresponding to...). Figure 5 In S51, the data collected by the sensing module is aggregated to the data aggregation module on the control node 54 (corresponding to S51) within seconds. Figure 5 (S52). The data aggregation module will archive the data offline to the offline data warehouse on the control node 54 (corresponding to...). Figure 5(S53). The timing prediction module on the control node 54 predicts the timing characteristics of each synchronization instance (corresponding to) through a timing prediction algorithm. Figure 5 (S54). The scheduling module of scheduling component 531 obtains the temporal features (corresponding to) from the temporal prediction module. Figure 5 S55), and reschedule in advance during off-peak periods (corresponding to Figure 5 The S56 module ensures that the data synchronization module has sufficient performance guarantees during peak periods, and the scheduling module reschedules the synchronization instances in the cluster during low traffic periods based on the time-series prediction results, ensuring that the synchronization instances have sufficient resources to guarantee successful local upgrades.

[0148] In one embodiment, the above method may further include the following steps:

[0149] 212. The control node obtains the ratio between the current resource requirement of the data synchronization cluster and the total resource of the data synchronization cluster.

[0150] The control node can periodically (e.g., every 1 minute, every 10 minutes) obtain the current resource demand of the data synchronization cluster and calculate the ratio between the current resource demand of the data synchronization cluster and the total resources of the data synchronization cluster.

[0151] For example, the sum of the current resource requirements of multiple synchronization nodes in a data synchronization cluster can be used to determine the current resource requirements of the data synchronization cluster.

[0152] 213. When the ratio is greater than or equal to the first ratio threshold, the control node adds a data synchronization node to the data synchronization cluster.

[0153] The first ratio threshold can be set according to the actual situation, and this application does not make a specific limitation on it, for example: 90%.

[0154] When the ratio is greater than or equal to the first ratio threshold, it indicates that the current water level of the cluster is high and the cluster needs to be expanded, which means that new data synchronization nodes need to be added to the data synchronization cluster.

[0155] 214. When the ratio is less than the second ratio threshold, the control node reduces the number of data synchronization nodes in the data synchronization cluster.

[0156] Wherein, the first ratio threshold is greater than the second ratio threshold. The size of the first ratio threshold can be set according to actual needs, and this application embodiment does not specifically limit it, for example: 50%.

[0157] When the ratio is less than the second ratio threshold, it indicates that the current water level of the cluster is low and there is a waste of resources. The cluster needs to be scaled down, which means that the number of data synchronization nodes in the data synchronization cluster needs to be reduced.

[0158] For example, a data synchronization node with a smaller number of deployed synchronization instances can be selected from the data synchronization cluster as the node to be deleted. Before deleting the node, the synchronization instances on the node to be deleted need to be migrated to other synchronization nodes in the data synchronization cluster. The selection process for other synchronization nodes can refer to the selection process for the second or third synchronization node described above, and will not be described in detail here. The migration process can also refer to the migration process for the instances to be migrated described above, and will not be described in detail here.

[0159] The technical solution provided by the embodiments of this application can ensure that the cluster has sufficient idle resources in real time. That is, it ensures that there are enough idle resources at the cluster level to meet the needs of instance expansion. Furthermore, cluster scaling down can reduce resource waste and improve resource utilization.

[0160] Optionally, the above method may also include the following steps:

[0161] 215. The control node determines the future resource requirements of the data synchronization cluster based on the future resource requirements of each synchronization node in the data synchronization cluster.

[0162] For example, the future resource requirements of the data synchronization cluster may include: the resource requirements of the data synchronization cluster at Q future time points, where the resource requirements of the data synchronization cluster at the q-th future time point are the sum of the resource requirements of the X synchronization nodes in the data synchronization cluster at the q-th future time point.

[0163] 216. When the resources of the data synchronization cluster cannot meet the future resource requirements of the data synchronization cluster, the control node adds a data synchronization node to the data synchronization cluster.

[0164] If the resource demand of the data synchronization cluster at any of the Q future time points exceeds the total resources available for allocation to synchronization instances, it indicates that the resources of the data synchronization cluster cannot meet its future resource demands. Conversely, if the resource demand of the data synchronization cluster at any of the Q future time points does not exceed the total resources available for allocation to synchronization instances, it indicates that the resources of the data synchronization cluster meet its future resource demands.

[0165] When the resources of the data synchronization cluster cannot meet the future resource requirements of the data synchronization cluster, the control node adds a data synchronization node to the data synchronization cluster.

[0166] The technical solution provided by the embodiments of this application can ensure that the cluster has sufficient idle resources in the future (e.g., the next day), that is, at the cluster level, it can ensure that there are enough idle resources to meet the needs of instance expansion in the future.

[0167] For example, in this embodiment of the application, a basic serverless data synchronization system can be implemented using cloud services. For instance, a serverless data synchronization system can be built using Elastic Container Instance (ECI) cloud services. That is, the synchronization node in this embodiment of the application is an Elastic Container Instance. When a synchronization instance needs to be deployed, it is deployed by calling the interface provided by the Elastic Container Instance cloud service. In this way, the Elastic Container Instance can be used to ensure the resource supply of the synchronization instance.

[0168] The following is combined Figure 6 This section introduces the cluster resource management process of a serverless data synchronization system from a cluster perspective:

[0169] like Figure 6 As shown, region 600 includes data synchronization cluster 1, data synchronization cluster 2 and data synchronization cluster 3, and region 600 also includes a control node 64 for managing region 600.

[0170] Data synchronization cluster 1 is equipped with a boosting component 61, data synchronization cluster 2 is equipped with a boosting component 62, data synchronization cluster 2 is equipped with a boosting component 63, and the control node 64 is equipped with a data aggregation module, an offline data warehouse, a time series prediction module and a resource management module.

[0171] As a serverless architecture, the cluster needs to maintain a sufficient water level in real time.

[0172] Elevation components 61, 62, and 63 will aggregate second-level operational data to the data aggregation module (corresponding to...). Figure 6 (S61). The resource management module periodically queries the water level of each cluster (corresponding to...). Figure 6 In S62), if the cluster water level is greater than or equal to the first threshold, then cluster expansion is executed (corresponding to...). Figure 6 If the value is less than the second threshold (S63), then cluster scaling down is performed (corresponding to...). Figure 6 (S64). The data aggregation module will archive the data offline to the offline data warehouse (corresponding to...). Figure 6 In the S65 module, the time-series forecasting module will predict the resource usage of each cluster from the offline data warehouse (corresponding to...). Figure 6 If a resource shortage is predicted within the next day (as per S66), the resource management module will be invoked in advance to expand capacity (corresponding to...). Figure 6(S67) This ensures sufficient water levels without excessive resource waste. The second threshold is less than or equal to the first threshold.

[0173] Optionally, the above method may also include the following steps:

[0174] 217. The control node determines the total resource usage of the second synchronization instance within a historical time period based on the historical resource usage of the second synchronization instance.

[0175] The historical time period can be the most recent hour, the most recent ten minutes, etc.

[0176] For example, the historical resource usage of the second synchronization instance includes the resource usage at Y historical time points within a historical time period. The Y historical time points form an arithmetic sequence, and the common difference of this arithmetic sequence is the sampling interval used by the lifting component. The sampling interval can be 1 second.

[0177] The resource usage of the second synchronization instance at the y-th historical time point can be multiplied by the sampling interval to obtain the y-th resource usage, where y is an integer ranging from 1 to Y. This yields Y resource usage values. The sum of these Y resource usage values ​​is then determined as the total resource usage of the second synchronization instance within the historical time period.

[0178] 218. Based on the total resource usage, determine the resource usage cost of the second synchronization instance within the historical time period.

[0179] The cost of using the resources is directly proportional to the total amount of resources used.

[0180] The second synchronization instance is any one of the synchronization instances deployed in the data synchronization cluster.

[0181] The product of total resource usage and unit cost can be used to determine the resource usage cost of the second synchronization instance within the historical time period.

[0182] This embodiment provides a data synchronization system that can be billed based on usage.

[0183] The following is combined Figure 7 This document introduces the metering and billing process for serverless data synchronization systems from a regional perspective:

[0184] like Figure 7 As shown, region 700 includes data synchronization cluster 1, data synchronization cluster 2 and data synchronization cluster 3, and region 700 also includes a control node 74 for managing region 700.

[0185] Data synchronization cluster 1 is equipped with a springboard component 71, data synchronization cluster 2 is equipped with a springboard component 72, data synchronization cluster 2 is equipped with a springboard component 73, and the control node 64 is equipped with a data aggregation module, an offline data warehouse, an offline billing calculation module, and a billing module.

[0186] Elevation components 61, 62, and 63 will aggregate second-level operational data to the data aggregation module (corresponding to...). Figure 7 Step 1). The billing module will query the operational data from the data aggregation module and perform bill calculations (corresponding to...). Figure 7 Step 2) After verification, the data is pushed to the billing service (not shown) for billing. Simultaneously, both the data aggregation module and the billing module will archive the data offline to the offline data warehouse (corresponding to...). Figure 7 In step 3), the offline billing calculation module will verify the bills generated by the billing module daily (corresponding to...). Figure 7 Step 4) ensures the bill is accurate.

[0187] In summary, this application addresses the performance limitations and large failure radius of centralized population components by introducing a distributed population component. This application provides a perception module, a decision-making module, and an execution module on all synchronization nodes, responsible for second-level data acquisition, second-level decision-making, and second-level population, respectively. A time-series prediction module ensures that the vast majority of synchronization instances can complete local population within the node, while in rare cases, remote population ensures availability under extreme conditions, thus optimizing the performance limitations of centralized population components. In other words, this application can achieve a highly available data synchronization system with second-level seamless dynamic scaling. Furthermore, population is performed within a single node; if the population component within a node fails, it only affects the elastic tasks of that single node, reducing the failure radius by an order of magnitude compared to centralized population components.

[0188] Figure 8 A schematic diagram of the structure of an electronic device according to an embodiment of this application is shown. Figure 8As shown, the electronic device includes a memory 1101 and a processor 1102. The memory 1101 can be configured to store various other data to support operation on the electronic device. Examples of such data include instructions for any application or method used to operate on the electronic device. The memory 1101 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Electrically Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0189] The memory 1101 is used to store programs;

[0190] The processor 1102 is coupled to the memory 1101 and is used to execute the program stored in the memory 1101 to implement the methods provided in the above-described method embodiments.

[0191] Furthermore, such as Figure 8 As shown, the electronic device also includes: communication component 1103, power supply component 1104, and other components. Figure 8 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 8 The components shown.

[0192] For example, an electronic device may specifically be a server, which may be a physical server, a virtual server, a cloud server, etc.

[0193] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, can implement the steps or functions of the methods provided in the above-described method embodiments.

[0194] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps or functions of the methods provided in the above-described method embodiments.

[0195] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0196] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM (Read Only Memory) / RAM (Random Access Memory), magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0197] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them, and multiple embodiments can be combined with each other; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A resource scheduling method applied to a data synchronization system, characterized in that, The data synchronization system includes a central node and multiple synchronization nodes. A scheduling component is deployed on the central node, and each synchronization node is deployed with a boosting component and a synchronization instance for performing data synchronization tasks. The scheduling component obtains the future resource requirements of the first synchronization node among the plurality of synchronization nodes; the future resource requirements of the first synchronization node are predicted based on the historical resource usage of the synchronization instances deployed on the first synchronization node; When the resources of the first synchronization node cannot meet the future resource requirements of the first synchronization node, the scheduling component determines the instance to be migrated from the synchronization instances deployed on the first synchronization node and migrates the instance to be migrated to the second synchronization node among the plurality of synchronization nodes; the resources of the second synchronization node meet the future resource requirements of the second synchronization node and the future resource requirements of the instance to be migrated. The target escalation component determines the resource expansion requirements of the first synchronization instance based on the current resource usage of the first synchronization instance. The target escalation component refers to the escalation component deployed on the first synchronization node, and the first synchronization instance refers to the synchronization instance currently deployed on the first synchronization node. The target boosting component expands the resources of the first synchronization instance by utilizing the idle resources of the first synchronization node according to the resource expansion requirements.

2. The method according to claim 1, characterized in that, The resource expansion requirement includes the additional resource allocation required for the first synchronization instance; the method further includes: The target lift component determines whether the idle resources of the first synchronization node meet the resource expansion requirements based on the comparison between the amount of idle resources of the first synchronization node and the amount of additional resource allocation required by the first synchronization instance. The target lift-up component, based on the resource expansion requirement, utilizes the idle resources of the first synchronization node to expand the resources of the first synchronization instance, including: If the idle resources of the first synchronization node meet the resource expansion requirements, the target boosting component will use the idle resources of the first synchronization node to expand the resources of the first synchronization instance according to the resource expansion requirements.

3. The method according to claim 2, characterized in that, Also includes: If the idle resources of the first synchronization node do not meet the resource expansion requirements, the target escalation component sends a resource shortage event to the scheduling component. After receiving the resource shortage event, the scheduling component obtains the current resource requirement of the third synchronization node among the multiple synchronization nodes and the future resource requirement of the third synchronization node. The scheduling component determines whether the third synchronization node currently and in the future meets the receiving conditions for receiving the first synchronization instance based on the current resource demand and the future resource demand of the third synchronization node. When the third synchronization node currently and in the future meets the receiving conditions, the scheduling component will migrate the first synchronization instance to the third synchronization node.

4. The method according to claim 3, characterized in that, The scheduling component determines whether the third synchronization node currently and in the future meets the reception conditions for receiving the first synchronization instance based on the current resource demand and future resource demand of the third synchronization node, including: When the resources of the third synchronization node meet the current resource requirements of the third synchronization node and the current resource requirements of the first synchronization instance, it is determined that the third synchronization node currently meets the receiving conditions. When the resources of the third synchronization node meet the future resource requirements of both the third synchronization node and the first synchronization instance, it is determined that the third synchronization node will meet the receiving conditions in the future. The future resource requirements of the first synchronization instance are predicted based on the historical resource usage of the first synchronization instance.

5. The method according to any one of claims 1 to 4, characterized in that, The data synchronization system further includes a management and control node, and the method further includes: The control node obtains the historical resource usage of the synchronization instances deployed on the first synchronization node; The control node predicts the future resource requirements of the synchronization instances deployed on the first synchronization node based on the historical resource usage of the synchronization instances deployed on the first synchronization node. The control node determines the future resource requirements of the first synchronization node based on the future resource requirements of the synchronization instances deployed on the first synchronization node.

6. The method according to claim 5, characterized in that, The scheduling component obtains the future resource requirements of the first synchronization node among the plurality of synchronization nodes, including: The scheduling component sends an acquisition request to the control node; the acquisition request is used to request the future resource requirements of the first synchronization node. The scheduling component receives the future resource requirements of the first synchronization node returned by the control node.

7. The method according to claim 5, characterized in that, The historical resource usage includes resource usage at multiple historical points in time. The control node predicts the future resource requirements of the synchronization instances deployed on the first synchronization node based on their historical resource usage, including: The control node sorts the resource usage at the multiple historical time points according to their chronological order to obtain a time series; The control node uses a time series prediction algorithm to predict the future resource requirements of the synchronization instances deployed on the first synchronization node based on the time series.

8. The method according to claim 5, characterized in that, The central node and the plurality of synchronization nodes constitute a data synchronization cluster; the method further includes: The control node obtains the ratio between the current resource requirement of the data synchronization cluster and the total resources of the data synchronization cluster; When the ratio is greater than or equal to the first ratio threshold, the control node adds a data synchronization node to the data synchronization cluster; When the ratio is less than the second ratio threshold, the control node reduces the number of data synchronization nodes in the data synchronization cluster; Wherein, the first ratio threshold is greater than or equal to the second ratio threshold.

9. The method according to claim 8, characterized in that, Also includes: The control node determines the future resource requirements of the data synchronization cluster based on the future resource requirements of each synchronization node in the data synchronization cluster. When the resources of the data synchronization cluster cannot meet the future resource requirements of the data synchronization cluster, the control node adds a data synchronization node to the data synchronization cluster.

10. The method according to claim 5, characterized in that, Also includes: The control node determines the total resource usage of the second synchronization instance within the historical time period based on the historical resource usage of the second synchronization instance. The control node determines the resource usage cost of the second synchronization instance within the historical time period based on the total resource usage, and the resource usage cost is proportional to the total resource usage. The second synchronization instance is a synchronization instance deployed in the data synchronization cluster.

11. A data synchronization system, characterized in that, The data synchronization system includes a central node and multiple synchronization nodes. A scheduling component is deployed on the central node, and each synchronization node is deployed with a spurious component and a synchronization instance for executing data synchronization tasks. The scheduling component is configured to: obtain the future resource requirements of a first synchronization node among the plurality of synchronization nodes; the future resource requirements of the first synchronization node are predicted based on the historical resource usage of the synchronization instances deployed on the first synchronization node; when the resources of the first synchronization node cannot meet the future resource requirements of the first synchronization node, determine an instance to be migrated from the synchronization instances deployed on the first synchronization node and migrate the instance to be migrated to a second synchronization node among the plurality of synchronization nodes; the resources of the second synchronization node meet the future resource requirements of the second synchronization node and the future resource requirements of the instance to be migrated; The target expansion component is used to: determine the resource expansion requirements of the first synchronization instance based on the current resource usage of the first synchronization instance, wherein the target expansion component refers to the expansion component deployed on the first synchronization node, and the first synchronization instance refers to the synchronization instance currently deployed on the first synchronization node; and expand the resources of the first synchronization instance using the idle resources of the first synchronization node according to the resource expansion requirements.

12. An electronic device, characterized in that, include: Memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is configured to execute the program stored in the memory to implement the method of any one of claims 1 to 10.

13. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a computer, it can implement the method of any one of claims 1 to 10.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 10.

Images