Data processing methods, devices, equipment, and media for clusters under big data computing

Abstract

The present application discloses a data processing method, device, electronic device, and computer-readable medium, the method comprising: after creating an elastic scaling rule for a target cluster, first configuring a cache space corresponding to the target cluster according to the elastic scaling rule; then obtaining indicator detection status data of the load type rule item within a replenishment time range according to the cluster identifier of the target cluster and the indicator item identifier carried by the load type rule item in the elastic scaling rule, and storing the indicator detection status data in the cache space corresponding to the target cluster; subsequently, continuously obtaining the indicator detection update result corresponding to the target cluster during the polling interval by polling, and updating the storage content in the cache space corresponding to the target cluster according to the indicator detection update result, so that the purpose of elastic scaling decision for a cluster can be achieved by means of one-time replenishment + continuous polling and updating.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, device, electronic device, and computer-readable medium. Background Art

[0002] In fact, the amount of computing presented by big data computing in some application scenarios is not constant, but has obvious peaks and troughs. As a result, the cluster used to perform big data computing may have insufficient computing resources during busy times and wasted computing resources during idle times. Therefore, in order to overcome the adverse effects caused by this phenomenon, computing resources can be adjusted for the cluster at certain times (that is, elastic scaling of computing resources for the cluster).

[0003] In some technical solutions, it is possible to determine when to adjust the computing resources of a cluster based on some pre-set rules (eg, load scaling rules, etc.).

[0004] However, due to defects in the above technical solutions (for example, establishing a corresponding polling mechanism for different indicators of different clusters), the determination of these rules takes a long time, resulting in poor performance in adjusting the computing resources of the cluster. Summary of the Invention

[0005] In order to solve the above problems, the present application provides a data processing method, device, electronic device, and computer-readable medium.

[0006] In order to achieve the above objectives, the technical solutions provided by this application are as follows:

[0007] The present application provides a data processing method, applied to a cluster management device, the method comprising:

[0008] After creating an elastic scaling rule for a target cluster, configuring a cache space corresponding to the target cluster according to the elastic scaling rule; the elastic scaling rule includes a load type rule item;

[0009] According to the cluster identifier of the target cluster and the indicator item identifier carried by the load type rule item, obtain the indicator detection status data of the load type rule item within the replenishment time range, and store the indicator detection status data in the cache space corresponding to the target cluster; the replenishment time range is determined according to the indicator detection time period carried by the load type rule item and the last polling trigger time;

[0010] Determine whether the polling trigger condition is met. If so, obtain the indicator detection update result corresponding to the target cluster, and update the content stored in the cache space corresponding to the target cluster based on the indicator detection update result corresponding to the target cluster; the indicator detection update result is used to describe the indicator detection status update presented by the elastic scaling rule during the polling interval;

[0011] According to the content stored in the cache space corresponding to the target cluster, the elastic scaling decision result corresponding to the target cluster is determined, and the step of determining whether the polling trigger condition is met is continued.

[0012] In one possible implementation, before creating an elastic scaling rule for a target cluster, the cluster management device is configured to perform elastic scaling decision processing on at least one to-be-managed cluster using the same polling mechanism;

[0013] The last polling trigger time refers to the triggering moment of the last polling process under the polling mechanism;

[0014] The polling triggering condition refers to a condition pre-set for the polling mechanism and used to trigger a polling process.

[0015] In one possible implementation, obtaining the indicator detection status data of the load type rule item within the replenishment time range based on the cluster identifier of the target cluster and the indicator item identifier carried by the load type rule item includes:

[0016] The cluster identifier of the target cluster, the indicator item identifier carried by the load type rule item, and the completion time range are passed into the data completion interface, and the data completion interface is called to obtain the indicator detection status data of the load type rule item within the completion time range;

[0017] The obtaining of the indicator detection update result corresponding to the target cluster includes:

[0018] Call the polling data interface to obtain the indicator detection update result corresponding to the target cluster.

[0019] In a possible implementation manner, the cluster management device is used to manage a plurality of clusters; the plurality of clusters includes the target cluster;

[0020] The calling of the polling data interface to obtain the indicator detection update result corresponding to the target cluster includes:

[0021] Calling the polling data interface to obtain the indicator detection update results corresponding to the several clusters;

[0022] The updating process of the content stored in the cache space corresponding to the target cluster according to the indicator detection update result corresponding to the target cluster includes:

[0023] Based on the indicator detection update results corresponding to each cluster, the stored content in the cache space corresponding to each cluster is updated;

[0024] The determining, based on the content stored in the cache space corresponding to the target cluster, the elastic scaling decision result corresponding to the target cluster includes:

[0025] The elastic scaling decision result corresponding to each cluster is determined according to the stored content in the cache space corresponding to each cluster.

[0026] In one possible implementation manner, before transmitting the cluster identifier of the target cluster, the indicator item identifier carried by the load type rule item, and the padding time range to the data padding interface, the method further includes:

[0027] Read the last call time of the polling data interface from the preset cache area as the last polling trigger time;

[0028] The filling time range is determined according to the time period carried by the load type rule item and the last polling trigger time.

[0029] In one possible implementation, the method further includes:

[0030] The data stored in the preset cache area is updated using the current call time of the polling data interface.

[0031] In one possible implementation, configuring the cache space corresponding to the target cluster according to the elastic scaling rule created for the target cluster includes:

[0032] Parsing the elastic scaling rule to obtain a rule tree; the elastic scaling rule includes a plurality of rule expression objects; the plurality of rule expression objects include a load type rule item; the rule tree includes a node corresponding to each of the rule expression objects;

[0033] According to the rule tree, a cache space corresponding to the target cluster is configured, where the cache space includes a cache area corresponding to each of the rule expression objects.

[0034] In a possible implementation manner, the cache space corresponding to the target cluster includes a cache area corresponding to the load type rule item;

[0035] Storing the indicator detection status data in a cache space corresponding to the target cluster includes:

[0036] The indicator detection status data is stored in a cache area corresponding to the load type rule item.

[0037] In a possible implementation manner, the elastic scaling rule further includes a time type rule item;

[0038] The method further comprises:

[0039] Determine whether the current polling trigger time is within the time range represented by the time type rule item, and obtain a determination result;

[0040] The determining, based on the content stored in the cache space corresponding to the target cluster, the elastic scaling decision result corresponding to the target cluster includes:

[0041] An elastic scaling decision result corresponding to the target cluster is determined according to the judgment result and the storage content in the cache space corresponding to the target cluster.

[0042] The present application provides a data processing device, comprising:

[0043] A cache configuration unit, configured to configure a cache space corresponding to a target cluster according to an elastic scaling rule created for the target cluster; the elastic scaling rule includes a load type rule item;

[0044] a data completion unit, configured to obtain, based on the cluster identifier of the target cluster and the indicator item identifier carried by the load type rule item, the indicator detection status data of the load type rule item within a completion time range, and store the indicator detection status data in a cache space corresponding to the target cluster; the completion time range is determined based on the indicator detection time period carried by the load type rule item and the last polling trigger time;

[0045] A polling update unit is used to determine whether the polling trigger condition is met. If so, the indicator detection update result corresponding to the target cluster is obtained, and the storage content in the cache space corresponding to the target cluster is updated according to the indicator detection update result corresponding to the target cluster; the indicator detection update result is used to describe the indicator detection status update status presented by the elastic scaling rule during the polling interval; based on the storage content in the cache space corresponding to the target cluster, the elastic scaling decision result corresponding to the target cluster is determined, and the step of determining whether the polling trigger condition is met is continued.

[0046] The present application provides an electronic device, the device comprising: a processor and a memory;

[0047] The memory is used to store instructions or computer programs;

[0048] The processor is used to execute the instructions or computer programs in the memory so that the electronic device performs the data processing method provided in this application.

[0049] The present application provides a computer-readable medium, which stores instructions or computer programs. When the instructions or computer programs are executed on a device, the device executes the data processing method provided by the present application.

[0050] The present application provides a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the data processing method provided by the present application.

[0051] Compared with the prior art, this application has at least the following advantages:

[0052] In the technical solution provided by the present application, for a cluster management device, after creating an elastic scaling rule for a target cluster, first configure the cache space corresponding to the target cluster according to the elastic scaling rule; then, based on the cluster identifier of the target cluster and the indicator item identifier carried by the load type rule item in the elastic scaling rule, obtain the indicator detection status data of the load type rule item within the completion time range, and store the indicator detection status data in the cache space corresponding to the target cluster, so that the cache space records the status of the load type rule item within the time range of [last polling trigger time-indicator detection time period, last polling trigger time] (for example, at which moments the load indicator conditions involved in the load type rule item are met); then, through polling, continuously obtain the indicator detection update results corresponding to the target cluster to make the indicator detection more accurate. The new result is used to describe the indicator detection status update of the elastic scaling rule during the polling interval, and based on the indicator detection update result, the storage content in the cache space corresponding to the target cluster is updated so that the content recorded in the storage space for the indicator detection time period is continuously updated. This ensures the real-time nature of the storage content in the cache space, so that the elastic scaling decision result corresponding to the target cluster can be determined in real time based on the content stored in the cache space. In this way, with the help of the one-time completion + continuous polling update method, not only the purpose of elastic scaling decision for a cluster is achieved, but also the purpose of directly adding the elastic scaling decision process for the target cluster to the existing polling mechanism is achieved, thereby effectively avoiding the adverse effects caused by re-creating a new polling mechanism for the target cluster, and thus helping to improve the elastic scaling decision effect.

[0053] It can be seen that for the cluster management device, the cluster management device can adopt the same polling mechanism to perform elastic decision-making processing on all clusters with elastic scaling configurations that it manages. In this way, the cluster management device can achieve the purpose of centralized polling processing of all indicator items of all clusters, which is conducive to improving the elastic scaling decision-making effect.

[0054] In addition, for a cluster, a data completion process is performed when its rules are created, so that the cache space corresponding to the cluster stores the indicator detection status data within the time range of [last polling trigger time - indicator detection time period, last polling trigger time]. Therefore, in the subsequent continuous polling process, it is only necessary to obtain the indicator detection status update presented during the polling interval for storage and update. In this way, the purpose of centralized query for all indicators under a cluster can be achieved, thereby effectively improving the efficiency of elastic scaling decision-making for a cluster.

[0055] In addition, for the target cluster mentioned above, the time range for filling in the time range involved is determined based on the indicator detection time period carried by the load type rule item and the last polling trigger time, so that the end time involved in the time range for filling in the time range is the start time of the current polling. This can effectively overcome the adverse effects caused by the rule creation time not belonging to a certain polling trigger time (for example, the resource overhead caused by the inability to use the polling mechanism that already exists in the cluster management device), thereby achieving the purpose of smoothly joining the existing polling mechanism without missing data, which is conducive to improving the elastic scaling decision-making effect of the cluster management device for the cluster.

[0056] In addition, for the cluster management device mentioned above, it can use only one interface (for example, the polling data interface) to obtain the indicator detection status update status of all clusters during the polling interval during each polling process. In this way, it can use the same interface to achieve the purpose of using the same polling mechanism to complete the elastic scaling decision process for all clusters, thereby effectively avoiding the adverse effects caused by configuring a polling mechanism for each cluster, and thus helping to improve the elastic scaling decision-making effect of the cluster management device for the cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0058] Figure 1A flowchart of a data processing method provided in an embodiment of the present application;

[0059] Figure 2 A schematic diagram of an elastic scaling rule provided in an embodiment of the present application;

[0060] Figure 3 A schematic diagram of a cache space corresponding to a cluster provided in an embodiment of the present application;

[0061] Figure 4 A schematic diagram of updating the stored content in the cache area corresponding to a load type rule item provided in an embodiment of the present application;

[0062] Figure 5 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0063] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] To facilitate understanding, the technical terms involved in this application are explained below.

[0065] Elastic scaling refers to adjusting the computing scale of a cluster (for example, adjusting the number of task nodes).

[0066] The time scaling type is used to trigger elastic scaling according to a certain point in time or a certain time period. In addition, the load scaling type can be further divided into two types: one is the time point scaling type, which specifically refers to the single or repeated execution of elastic scaling processing at a fixed time point. For example: single, the scaling action is triggered at 8:00 on January 1, 2023; repeated, the scaling action is triggered at 8:00 every Monday. The other is the time period scaling type, which specifically refers to the repeated execution of elastic scaling processing within a fixed time period, which can be used in conjunction with the load scaling type. For example: trigger the scaling action from 8:00 to 9:00 on the 1st of each month. The trigger interval can be defined by the system, such as 5 minutes, that is, the scaling action is triggered every 5 minutes during this time period.

[0067] Load scaling triggers autoscaling based on a user-specified load metric (e.g., YetAnother Resource Negotiator (YARN) metric) and its corresponding threshold for a cluster. Specifically, scaling is triggered if the load metric exceeds (or falls below) the threshold a specified number of times within a specified time period.

[0068] The sampling interval is the time interval used to collect data from a cluster. For example, when collecting a load metric value for a cluster, you can use a metric monitoring service (such as Prometheus) to perform fixed-frequency sampling and collection on the cluster side. In this case, the time interval between two consecutive samplings can be called the sampling interval.

[0069] The polling interval is the time between consecutive polls when the cluster management device polls each cluster for relevant data (e.g., metric values). As can be seen, if the polling interval is 5 minutes and the sampling interval is 30 seconds, the cluster management device can query 10 metric values ​​for a load metric for a cluster in one poll.

[0070] Based on the above technical terms, the technical solution of this application is introduced below.

[0071] Research has found that in some technical solutions, in order to better improve the effectiveness of elastic scaling decisions, polling can be used to implement the triggering logic of rules under the load scaling type. However, this triggering method has defects, which are explained below with examples.

[0072] As an example, assume there are 1,000 clusters with autoscaling configurations, each cluster has 10 rules, each rule has 3 sub-rules (separated by "or"), and each sub-rule involves 3 load indicators (separated by "and"). Based on this assumption, we can see that the number of data queries required during each elastic decision-making process is: 1000*10*3*3=90,000 (times). Because the sampling time periods set by users for indicators in different rules in different clusters (that is, the time range within which sampling is required for each indicator) may be different, it is impossible to conduct a centralized query for each cluster and indicator. 90,000 queries are necessary, which requires establishing a corresponding polling mechanism for different indicators in different clusters. This results in a relatively long elastic decision-making process for all clusters, which leads to poor elastic scaling decision-making performance.

[0073] Based on this, the present application provides a data processing method applied to a cluster management device, which is specifically as follows: after creating an elastic scaling rule for a target cluster, first configure the cache space corresponding to the target cluster according to the elastic scaling rule; then, based on the cluster identifier of the target cluster and the indicator item identifier carried by the load type rule item in the elastic scaling rule, obtain the indicator detection status data of the load type rule item within the replenishment time range, and store the indicator detection status data in the cache space corresponding to the target cluster, so that the cache space records the status of the load type rule item within the time range of [last polling trigger time-indicator detection time period, last polling trigger time] (for example, at which moments the load indicator conditions involved in the load type rule item are met); then, through polling, continuously obtain the indicator detection update results corresponding to the target cluster, so that the The indicator detection update result is used to describe the indicator detection status update of the elastic scaling rule during the polling interval, and based on the indicator detection update result, the storage content in the cache space corresponding to the target cluster is updated, so that the content recorded in the storage space for the indicator detection time period is continuously updated. This ensures the real-time nature of the storage content in the cache space, so that the elastic scaling decision result corresponding to the target cluster can be determined in real time based on the storage content in the cache space. In this way, with the help of the one-time completion + continuous polling update method, not only the purpose of elastic scaling decision for a cluster is achieved, but also the purpose of directly adding the elastic scaling decision process for the target cluster to the existing polling mechanism is achieved, thereby effectively avoiding the adverse effects caused by re-creating a new polling mechanism for the target cluster, and thus helping to improve the elastic scaling decision effect.

[0074] It can be seen that for the cluster management device, the cluster management device can adopt the same polling mechanism to perform elastic decision-making processing on all clusters with elastic scaling configurations that it manages. In this way, the cluster management device can achieve the purpose of centralized polling processing of all indicator items of all clusters, which is conducive to improving the elastic scaling decision-making effect.

[0075] In addition, for a cluster, a data completion process is performed when its rules are created, so that the cache space corresponding to the cluster stores the indicator detection status data within the time range of [last polling trigger time - indicator detection time period, last polling trigger time]. Therefore, in the subsequent continuous polling process, it is only necessary to obtain the indicator detection status update presented during the polling interval for storage and update. In this way, the purpose of centralized query for all indicators under a cluster can be achieved, thereby effectively improving the efficiency of elastic scaling decision-making for a cluster.

[0076] In addition, since the filling time range is determined based on the indicator detection time period carried by the load type rule item and the last polling trigger time, the end time involved in the filling time range is the start time of the current polling. This can effectively overcome the adverse effects caused by the rule creation time not belonging to a certain polling trigger time (for example, the resource overhead caused by the inability to use the polling mechanism that already exists in the cluster management device), thereby achieving the purpose of smoothly joining the existing polling mechanism without missing data, which is conducive to improving the elastic scaling decision-making effect of the cluster management device for the cluster.

[0077] In addition, the present application does not limit the execution subject of the above data processing method. For example, the data processing method provided in the embodiment of the present application can be applied to a device with data processing function such as a terminal device or a server. For another example, the data processing method provided in the embodiment of the present application can also be implemented by means of a data communication process between different devices (for example, a terminal device and a server, two terminal devices, or two servers). Among them, the terminal device can be a smart phone, a computer, a personal digital assistant (PDA) or a tablet computer. The server can be a stand-alone server, a cluster server or a cloud server.

[0078] In order to help those skilled in the art better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0079] In order to better understand the technical solution provided by this application, the data processing method provided by this application is described below with reference to some drawings. Figure 1 As shown, the data processing method for cluster management equipment provided by the embodiment of the present application includes the following S1-S5. Figure 1 A flowchart of a data processing method provided in an embodiment of the present application.

[0080] S1: After creating an elastic scaling rule for a target cluster, configure the cache space corresponding to the target cluster according to the elastic scaling rule; the elastic scaling rule includes a load type rule item.

[0081] The cluster management device is used to manage at least one cluster. The present application does not limit the cluster management device. For example, it can be implemented using any existing or future device that can perform unified management on a number of clusters.

[0082] In addition, for the cluster management device provided in this application, if the cluster management device is used to manage several clusters, the cluster management device can use the processing method shown in S1-S5 to adopt the same polling mechanism to complete some management affairs of all clusters (for example, affairs related to elastic scaling decisions, etc.). This can effectively avoid the adverse effects caused by the need to adopt different polling mechanisms for some management affairs of all clusters, thereby helping to improve the management performance of the cluster management device for these clusters (especially, elastic scaling decision performance).

[0083] The target cluster refers to a cluster that needs to be added to the existing polling mechanism in the cluster management device. This application does not limit the target cluster. For example, the target cluster can refer to a newly created cluster with elastic scaling configuration, so that the target cluster can perform elastic scaling with the assistance of the cluster management device. For another example, the target cluster can also refer to a cluster that has just added elastic scaling rules, so that the cluster management device can make elastic scaling decisions for the cluster based on the elastic scaling rules.

[0084] Autoscaling rules are rules set by the user for a cluster (e.g., target cluster) that are used when making autoscaling decisions for the cluster. For example, when the target cluster is Figure 2 For cluster 1 shown in the figure, the elastic scaling rules corresponding to the target cluster can include Figure 2 The following are rules 1 to rule N, where N is a positive integer.

[0085] In addition, this application is not limited to the above elastic scaling rules. For ease of understanding, two examples are used below to illustrate.

[0086] In example 1, an elastic scaling rule may include at least a load type rule item, so that the elastic scaling rule can be triggered according to at least one or more load indicator items.

[0087] The load type rule item refers to the rule belonging to the load scaling type. For example, the load type rule item can be Figure 2 The rule item "The number of YARN queued tasks greater than 5 occurs greater than 2 times within 2 hours" is shown, or Figure 2 The rule item shown is "The number of times the YARN available memory is less than 50 Gigabytes (GB) within 30 minutes is greater than 5."

[0088] In addition, the present application does not limit the implementation method of the above load type rule item. For example, the load type rule item can be used to indicate whether a certain indicator item meets a certain condition within a certain time period. It can be seen that under one possible implementation method, the load type rule item can carry an indicator item identifier, an indicator detection time period, and an indicator judgment condition. Among them, the indicator item identifier is used to uniquely identify a load indicator (for example, the number of YARN queued tasks, etc.). The indicator detection time period is used to indicate the time period for detecting the load indicator (for example, within 2 hours, etc.). The indicator judgment condition is used to indicate the situation specified by the user for the load indicator that needs to be paid attention to when making elastic scaling decisions (for example, the number of YARN queued tasks is greater than 5).

[0089] In addition, this application does not limit the number of the above load type rule items. For example, an elastic scaling rule may include one or more load type rule items.

[0090] In the second example, the elastic scaling rule may include at least a load type rule item and a time type rule item, so that the elastic scaling rule can simultaneously trigger elastic scaling according to the status of some load indicator items and a certain time period.

[0091] The time type rule item refers to a rule belonging to the time expansion type. For example, the time type rule item can be Figure 2 The rule items shown are "the last day of each month and the first day of the month", "9 to 12 o'clock every day", or "Monday to Friday every week".

[0092] In addition, this application does not limit the above time type rule items. For example, in some application scenarios, the time type rule item can be used to limit a certain time period. Therefore, in one possible implementation, the time type rule item belongs to the time period scaling type, and the time type rule item can carry the elastic scaling requirement time period. The elastic scaling requirement time period refers to the time range within which elastic scaling processing is required.

[0093] In addition, this application does not limit the number of the above time type rule items. For example, an elastic scaling rule may include one or more time type rule items.

[0094] Based on the above information about auto-scaling rules, it can be seen that in one possible implementation, the auto-scaling rule can include one or more rule items under the load scaling type. In another possible implementation, the auto-scaling rule can include at least one rule item under the load scaling type and at least one rule item under the time scaling type, so that the auto-scaling rule is a hybrid time + load type rule.

[0095] In addition, this application does not limit the creation process of the above elastic scaling rules. For example, in some application scenarios, users can use natural language and the expression of the rule engine (for example, Figure 2 ), configure the elastic scaling rule for a cluster on the relevant interface to complete the creation process for the elastic scaling rule.

[0096] The cache space corresponding to the target cluster mentioned above is used to store data related to the elastic scaling rules created for the target cluster (for example, Figure 3 timestamps such as "timestamp-1" shown in the figure) so that the storage content in the "cache space corresponding to the target cluster" can be used to quickly complete the elastic scaling decision for the target cluster.

[0097] In addition, this application does not limit the configuration process of the "cache space corresponding to the target cluster" mentioned above. For example, to better improve the efficiency of elastic scaling decisions, the cache space corresponding to the target cluster can be configured based on the principle of trading space for time. For ease of understanding, the following example illustrates this.

[0098] As an example, the configuration process of the above “cache space corresponding to the target cluster” may specifically include the following steps 11 and 12.

[0099] Step 11: Parse the elastic scaling rule created for the target cluster to obtain a rule tree; wherein the elastic scaling rule includes several rule expression objects; the several rule expression objects include load type rule items; the rule tree includes nodes corresponding to each rule expression object.

[0100] The rule expression object refers to the object used to express a certain rule in the above “elastic scaling rules created for the target cluster”; and this application does not limit the rule expression object. For example, it can be a rule item (for example, Figure 2 The rule item "In 2 hours, the number of YARN queued tasks is greater than 5 and the number of occurrences is greater than 2" is shown), or it can be an expression composed of multiple rule items (for example, Figure 2 The rule expression "sub-rule 1" shown in FIG. 1 may also be a set of multiple expressions (for example, Figure 2 Cluster 1 rule set shown).

[0101] It can be seen that for the elastic scaling rule created for the target cluster, the elastic scaling rule may include several rule expression objects (for example, Figure 2 and certain association relationships (such as belonging relationships, or, and relationships) exist between certain objects in the plurality of rule expression objects.

[0102] The rule tree represents the elastic scaling rule in a tree structure, so that the rule tree can not only represent all the rule expression objects in the elastic scaling rule, but also represent the relationship between different rule expression objects in the elastic scaling rule. For example, when the target cluster is Figure 2 For cluster 1 shown in the figure, the rule tree corresponding to the target cluster can be Figure 2 The tree structure shown.

[0103] In addition, for the rule tree corresponding to a cluster, each node in the rule tree can represent the rule expression object existing in the elastic scaling rules created for the cluster; and for a node in the rule tree, if the node belongs to the root node, the rule expression object corresponding to the node is the cluster identifier of the cluster, so that the node can represent the rule set created for the cluster; if the node belongs to a leaf node, the rule expression object corresponding to the node is the rule item of the load scaling type or the rule item of the time scaling type existing in the cluster, so that the node can represent the load indicator or time period required to refer to when making elastic scaling decisions for the cluster; if there is a child node corresponding to the node in the rule tree, the rule expression object corresponding to the node is an expression composed of multiple rule items.

[0104] Furthermore, this application does not limit the implementation of the "parsing process" in step 11 above. For example, it can be implemented by any existing or future method that can perform parsing process on a set of rules. For another example, it can be performed according to a pre-set rule parsing strategy, which is not specifically limited in this application.

[0105] Step 12: Based on the rule tree, configure the cache space corresponding to the target cluster. The cache space includes the cache area corresponding to each rule expression object.

[0106] In this application, for the target cluster mentioned above, if the elastic scaling rule created for the target cluster includes several rule expression objects, then after parsing the rule tree from the elastic scaling rule, the cache space corresponding to the target cluster can be configured according to the rule tree so that the cache space can include the cache area corresponding to each rule expression object (for example, Figure 3 , so that the storage contents in these cache areas can represent the rule tree. For ease of understanding, the following is explained with examples.

[0107] As an example, when the target cluster above is Figure 2 Cluster 1 shown in the figure, the elastic scaling rules created for the target cluster are: Figure 2The rules shown in FIG. 4 and the cache space corresponding to the target cluster is implemented with the help of the remote dictionary service (RemoteDictionary Server, Redis), the cache space corresponding to the target cluster can include at least Figure 3 The cache area shown; and Figure 3 The storage contents of the different cache areas shown are as follows:

[0108] ① In the rule tree, except for the leaf nodes, the cache areas corresponding to the rule expression objects represented by other nodes all use data structure 1 (for example, set data structure) for data caching. Figure 3 As shown, the cache area of ​​cluster 1 identifier → set < identifier of rule 1, ..., identifier of rule N> is used to cache the value of the rule identifier set using the string "identifier of cluster 1" as the key; the cache area of ​​rule 1 identifier → set < identifier of sub-rule 1, identifier of sub-rule 2> is used to cache the value of the sub-rule identifier set using the string "identifier of rule 1" as the key; the cache area of ​​sub-rule m identifier → set < some rule items> is used to cache the value of the rule item set using the string "identifier of sub-rule m" as the key, where m is a positive integer, m≤M, and M indicates that rule 1 is composed of M sub-rules.

[0109] ② The cache areas corresponding to the load scaling rule expression objects (e.g., a load type rule item) represented by the leaf nodes in the rule tree all use data structure 2 (e.g., list data structure) for data caching. Figure 3 As shown, the identifier of the load type rule item →list <timestamp>The timestamp is the timestamp corresponding to the indicator item whose indicator value meets the above indicator judgment conditions (for example, exceeding the threshold or falling below the threshold). The insertion logic of List is to insert the indicator item when the indicator value meets the indicator judgment conditions (for example, the number of YARN queued tasks is greater than 5).

[0110] It should be noted that this application does not limit the implementation method of the "identification of load type rule item" in the above paragraph. For example, the "identification of load type rule item" can be metric#period#unit#compare#threshold#times, wherein metric is the load indicator item, period is the indicator detection time period, unit is the sampling interval, compare is the comparison operator (for example, greater than or less than, etc.), threshold is the threshold value, and times is the number of occurrences, so that the "identification of load type rule item" can represent the semantic information carried by the load type rule item, so that it is convenient to judge the obtained indicator value based on the "identification of load type rule item" later. It can be seen that when the load type rule item is Figure 2 When the rule item "The number of YARN queued tasks is greater than 5 and the number of occurrences is greater than 2 within 2 hours" is shown, the identifier of the load type rule item can be Figure 3 The string "AvailableMemory#30#minute#less#50#5" is shown.

[0111] It should also be noted that this application does not limit the implementation method of the identification of the time type rule item. For example, the "identification of the time type rule item" can be recurrenceType#(recurrenceValue)#(begin#end), where recurrenceType is the period type (for example, daily, weekly, etc.); recurrenceValue is the period value, and recurrenceValue is optional content. For example, when the period type is weekly or monthly, the recurrenceValue parameter is not empty; however, when the period type is daily, the recurrenceValue parameter can be empty; begin is the interval start time, end is the interval end time, and begin and end are also optional content. For example, when the period type is daily, begin and end can be non-empty. It can be seen that when the time type rule item is Figure 2 For the rule item "9 to 12 o'clock every day" shown in the figure, the identifier of the time type rule item can be Figure 3 The string "Daily#09:00-21:00" is shown.

[0112] It should also be noted that since the data required for the judgment process of a rule expression object of the time-stretching type (for example, a certain time-type rule item) (for example, the current time and the time range defined in the time-type rule item) is easy to obtain, in order to better save space, there is no need to configure an additional cache area for storing relevant data for the rule expression object of the time-stretching type represented by the leaf node in the above rule tree.

[0113] Based on the relevant content of steps 11 and 12 above, it can be seen that for a cluster, after creating an elastic scaling rule for the cluster, a tree structure can be parsed from the elastic scaling rule; then, based on the tree structure, cache space can be configured for the cluster to obtain the cache space corresponding to the cluster, so that the cache space includes the cache area corresponding to each rule expression object in the elastic scaling rule (for example, the cache area corresponding to the load type rule item), so that these cache areas can be used to record the elastic scaling rule and related data obtained for the elastic scaling rule (for example, timestamps, etc.).

[0114] Based on the relevant content of S1 above, it can be seen that for a cluster management device, if the cluster management device is using the same polling mechanism to implement elastic scaling decisions for multiple clusters, then after creating elastic scaling rules for the target cluster (for example, a newly created cluster or a cluster that has just added elastic scaling rules, etc.), the cache space corresponding to the target cluster can be configured according to the elastic scaling rules, so that the cache space can record the elastic scaling rules and related data obtained for the elastic scaling rules (for example, timestamps, etc.), so that the elastic scaling decisions for the target cluster can be implemented with the help of the cache space and the polling process. In this way, all clusters managed by the cluster management device and with elastic scaling decision configurations can be centralized and unified into the same polling mechanism, thereby effectively improving the efficiency of elastic scaling decisions.

[0115] S2: Based on the cluster identifier of the target cluster and the indicator item identifier carried by the load type rule item, obtain the indicator detection status data of the load type rule item within the replenishment time range, and store the indicator detection status data in the cache space corresponding to the target cluster; the replenishment time range is determined based on the indicator detection time period carried by the load type rule item and the last polling trigger time.

[0116] The cluster identifier of the target cluster is used to uniquely identify the target cluster.

[0117] The indicator item identifier refers to the string that exists in the load type rule item above and is used to identify a load indicator item; and this application does not limit the indicator item identifier. For example, when the load type rule item is Figure 2 For the rule item "Within 2 hours, the number of YARN queued tasks is greater than 5 and the number of occurrences is greater than 2", the indicator item identifier may be the string "Number of YARN queued tasks".

[0118] The indicator detection time period refers to the detection time period for a load indicator item in the load type rule item above; and this application does not limit the indicator detection time period. For example, when the load type rule item is Figure 2 For the rule item "Within 2 hours, the number of YARN queued tasks greater than 5 occurs greater than 2 times", the indicator detection time period can be the string "within 2 hours".

[0119] The last polling trigger time refers to the most recent polling time of the cluster management device used for centralized polling processing for clusters with autoscaling decision-making configurations. This "last polling trigger time" refers to the last triggering time of the polling process, as of the execution time of S2 above.

[0120] Based on the above, in one possible implementation, if the cluster management device uses the same polling mechanism to make scaling decisions for at least one unmanaged cluster before creating scaling rules for the target cluster, then the "last polling trigger time" in this context may refer to the trigger time of the last polling process under this polling mechanism. The unmanaged cluster refers to the cluster that the cluster management device needs to manage using the same polling mechanism before creating scaling rules for the target cluster.

[0121] In addition, this application does not limit the method of obtaining the above "last polling trigger time". For example, in some application scenarios, for the polling process used to perform polling processing on a cluster with elastic scaling decision configuration, each trigger time of the polling process can be updated and stored in a certain cache space, so that the "last polling trigger time" can be quickly read from the cache space later.

[0122] The padding time range refers to the historical time period required for polling the target cluster's auto-scaling decisions, so that the target cluster can directly join the existing polling mechanism with the data involved in the padding time range.

[0123] In addition, in order to be able to add the same polling mechanism without missing data, the present application also provides a determination process for the above-mentioned time range for filling, which can specifically be: according to the index detection time period carried by the above-mentioned load type rule item and the last polling trigger time, determine the time range for filling, so that the time range for filling is specifically [the last polling trigger time - the index detection time period, the last polling trigger time]. In this way, with the help of this time range for filling, it is possible to fill the data required for the last polling process of the target cluster under the existing polling mechanism, so that only the data involved in the polling interval needs to be determined subsequently, in order to achieve the purpose of one-time filling + multiple polling updates. This is beneficial to improving the efficiency of elastic scaling decision-making.

[0124] The above-mentioned "index detection status data of the load type rule item within the time range for filling" is used to represent the status presented by the load type rule item within the time range for filling; moreover, the present application does not limit the implementation manner of the "index detection status data of the load type rule item within the time range for filling". For example, it may include the acquisition time point (such as a timestamp) of the index value that meets the index determination condition carried by the load type rule item. For the sake of understanding, the following will be described in conjunction with examples.

[0125] As an example, when the above-mentioned load type rule item is Figure 2 the rule item shown as "within 2 hours, the number of occurrences where the number of YARN queued tasks is greater than 5 is greater than 2", and the last polling trigger time is 10:00, the time range for filling can be [8:00, 10:00]. Moreover, the acquisition process of the above-mentioned "index detection status data of the load type rule item within the time range for filling" can specifically be: within the time range of [8:00, 10:00], obtain the number of YARN queued tasks at a preset sampling interval (such as 30 seconds), and determine whether the number of YARN queued tasks obtained each time is greater than 5. If it is greater than 5, store the timestamp of the number of YARN queued tasks (that is, the sampling time point of the number of YARN queued tasks) in the cache area corresponding to the load type rule item, so that the cache area stores the timestamps of the number of YARN queued tasks greater than 5 that occurred within the time range of [8:00, 10:00] (such as Figure 4 the timestamps shown as "timestamp-1" and "timestamp-7", etc.), so that subsequently, the "timestamps of the number of YARN queued tasks greater than 5 that occurred within the time range of [8:00, 10:00]" can be determined as the above-mentioned "index detection status data of the load type rule item within the time range for filling".

[0126] Based on the content of the above paragraph, it can be seen that in a possible implementation method, when the above several rule expression objects include load type rule items, and the cache space corresponding to the above target cluster includes the cache area corresponding to each rule expression object, the above S2 can specifically be: based on the cluster identifier of the target cluster and the indicator item identifier carried by the load type rule item, obtain the indicator detection status data of the load type rule item within the completion time range, and store the indicator detection status data in the cache area corresponding to the load type rule item, so that it can be determined whether the load type rule item is satisfied based on the record content in the cache area corresponding to the load type rule item.

[0127] In fact, to better improve the effectiveness of elastic scaling decisions, a pre-built interface can be used to obtain the corresponding completion data for a cluster. Based on this, this application also provides a method for obtaining the "indicator detection status data of the load type rule item within the completion time range" mentioned above. Specifically, the method can be as follows: the cluster identifier of the target cluster mentioned above, the indicator item identifier carried by the load type rule item, and the completion time range are passed to the data completion interface, and the data completion interface is called to obtain the indicator detection status data of the load type rule item within the completion time range.

[0128] The data completion interface is used to obtain a list of indicator values ​​and their corresponding timestamps that meet certain conditions (for example, the condition "the number of YARN queued tasks is greater than 5" above) within a certain time range based on the cluster identifier and indicator items; and the return value of the data completion interface can be recorded as List<factValue,timestamp> The factValue is the indicator value; the timestamp is the timestamp corresponding to the indicator value.

[0129] In addition, the above data completion interface can be pre-configured according to the application scenario; and this application does not limit the working principle of the data completion interface. For example, when the above load type rule item is Figure 2 As shown in the rule item "Within 2 hours, the number of YARN queued tasks greater than 5 occurs greater than 2 times", and the above filling time range can be [8:00, 10:00], the working principle of the data filling interface can be specifically as follows: within the time range [8:00, 10:00], the number of YARN queued tasks is obtained according to the preset sampling interval (for example, 30 seconds), and the number of YARN queued tasks obtained in each sampling is determined to be greater than 5, so that the indicator value greater than 5 and the timestamp corresponding to the indicator value can be used to construct a list, and the list is returned to the cluster management device above, so that the cluster management device can use the data in the list (for example, Figure 2 The timestamp "timestamp-1" shown in the figure) is used to update the cache area corresponding to the load type rule item, so as to achieve the purpose of initialization processing of the storage content in the cache area corresponding to the load type rule item.

[0130] Based on the relevant content of S2 above, it can be seen that for a cluster management device, if the cluster management device is using the same polling mechanism to implement elastic scaling decision processing for multiple clusters to be managed, then after creating an elastic scaling rule for the target cluster, when the elastic scaling rule includes at least a load type rule item, the indicator detection status data of the load type rule item within the replenishment time range can be obtained based on the cluster identifier of the target cluster and the indicator item identifier carried by the load type rule item (for example, Figure 3 The two timestamps "timestamp-1" and "timestamp-7" shown in the figure are used to enable the indicator detection status data to represent the status of the load type rule item within the time range of [last polling trigger time - indicator detection time period, last polling trigger time] (for example, at which moments the load indicator condition represented by the load type rule item was met), and the indicator detection status data is stored in the cache area corresponding to the load type rule item. This can achieve the purpose of performing polling initialization processing under the polling mechanism on the storage content in the cache area, so that the storage content can be continuously updated using the data that appears within the polling interval based on these initialization contents. This ensures that each subsequent polling only needs to obtain the data that appears within the polling interval, thereby improving the efficiency of elastic scaling decisions.

[0131] It should be noted that this application does not limit the relative relationship between the execution time of step S2 above, "According to the cluster identifier of the target cluster and the indicator item identifier carried by the load type rule item, obtain the indicator detection status data of the load type rule item within the replenishment time range" and the execution time of step S1 above, "Configure the cache space corresponding to the target cluster according to the elastic scaling rule." For example, the two can be executed simultaneously. For another example, the two can be executed successively in a pre-set order.

[0132] S3: Determine whether the polling trigger condition is met. If so, execute the following S4-S5; if not, return to continue executing the above S3.

[0133] The polling trigger condition may be preset, for example, it may specifically be: the difference between the current time and the last polling trigger time reaches a preset polling interval (eg, 5 minutes).

[0134] In addition, the present application does not limit the polling trigger condition. For example, if before creating elastic scaling rules for the target cluster, the cluster management device is used to use the same polling mechanism to perform elastic scaling decision processing on at least one cluster to be managed, then the polling trigger condition may refer to a condition set in advance for the polling mechanism for triggering a polling process, so that any cluster (for example, the target cluster) can smoothly join the continuous polling process described by the polling mechanism according to the polling trigger condition.

[0135] Based on the relevant content of S3 above, it can be seen that for a cluster management device, the cluster management device can determine whether the difference between the current moment and the last polling trigger time reaches the polling interval. If it is lower than the polling interval, it can be determined that the polling trigger condition has not yet been met, and after waiting for a preset period of time, it can return to continue to execute the step of "determining whether the difference between the current moment and the last polling trigger time reaches the polling interval"; if it is equal to the polling interval, it can be determined that the polling trigger condition has been met, so a round of elastic scaling decision processing can be performed for each cluster with elastic scaling configuration (that is, the processing process shown in S4-S5 below), so that the purpose of concentrating the elastic scaling processing of all clusters under the same polling mechanism can be achieved, which is conducive to improving the efficiency of elastic scaling decision-making.

[0136] S4: If the polling trigger condition is met, the indicator detection update result corresponding to the target cluster is obtained, and based on the indicator detection update result corresponding to the target cluster, the storage content in the cache space corresponding to the target cluster is updated; the indicator detection update result is used to describe the indicator detection status update presented by the elastic scaling rule during the polling interval.

[0137] The indicator detection update result is used to describe the status of the elastic scaling rule corresponding to the target cluster between two adjacent pollings (that is, within a polling interval). It can be seen that when the elastic scaling rule includes the load type rule item mentioned above, the indicator detection update result corresponding to the target cluster can at least include the indicator detection status update data corresponding to the load type rule item (for example, Figure 4 The two timestamps "timestamp-14" and "timestamp-18" shown in the figure are used to enable the indicator detection status update data to represent the status of the load type rule item within the polling interval.

[0138] In addition, this application does not limit the process of obtaining the above indicator detection update results.

[0139] In practice, to better enhance elastic scaling decision-making, the "target cluster's corresponding indicator detection update results" can be obtained using a pre-defined interface. Based on this, this application also provides a possible implementation for obtaining the "target cluster's corresponding indicator detection update results," which specifically involves calling a polling data interface to obtain the target cluster's corresponding indicator detection update results.

[0140] The polling data interface is used to collect and process data for all indicators of all clusters with elastic scaling configurations. The return value of the polling data interface can be recorded as a Map. <clusterId,List<metric,factValue,timestamp> >, so that the return value can indicate the indicator values ​​and corresponding timestamps for each indicator item that each cluster needs to collect. The Map is the mapping relationship; the clusterId is the cluster identifier; the List is the list; the metric is the indicator item; the factValue is the indicator value; and the timestamp is the timestamp corresponding to the indicator value.

[0141] In addition, the above polling data interface can be pre-configured according to the application scenario; and this application does not limit the working principle of the polling data interface. For example, when the above "elastic scaling rules corresponding to the target cluster" is Figure 2 For the elastic scaling rule shown, when the current polling trigger time is 10:05 and the polling interval is 5 minutes, the working principle of the polling data interface may at least include: obtaining the number of YARN queued tasks within the time range [10:00, 10:05] according to a preset sampling interval (for example, 30 seconds), and determining whether the number of YARN queued tasks obtained in each sampling is greater than 5, so that a mapping relationship Map can be formed by using the indicator value greater than 5 and the timestamp corresponding to the indicator value, and returning the mapping relationship to the cluster management device above, so that the cluster management device can use the data in the mapping relationship (for example, Figure 4 The timestamps "timestamp-14" and "timestamp-18" shown in the figure) are used to update the cache area corresponding to the load type rule item, thereby achieving the purpose of updating the storage content in the cache area corresponding to the load type rule item.

[0142] In fact, in order to better improve efficiency, this application also provides a possible implementation of S4 above. In this implementation, when the cluster management device above is used to manage several clusters, and the several clusters include a target cluster, S4 can be specifically as follows: if the polling trigger condition is met, the polling data interface is called to obtain the indicator detection update results corresponding to the several clusters, and based on the indicator detection update results corresponding to each cluster, the storage content in the cache space corresponding to each cluster is updated. In this way, the purpose of simultaneously obtaining all cluster-related data with elastic scaling configurations through one interface can be achieved. It should be noted that the process of obtaining the indicator detection update results corresponding to each cluster is similar to the process of obtaining the "indicator detection update results corresponding to the target cluster" above. For the sake of brevity, it will not be repeated here.

[0143] In fact, in order to better improve the effect of elastic scaling decisions, the present application also provides a possible implementation method of the storage content update process. In this implementation method, for the storage content in the cache space corresponding to each cluster, it is necessary not only to use the timestamp corresponding to the indicator value that exceeds the threshold within the time period of [current polling trigger time-polling interval, current polling trigger time] to update the storage content, but also to further loop to determine whether the timestamp with the longest storage time recorded in the updated storage content has expired. If it has expired, the timestamp can be directly deleted; if it has not expired, the loop can be directly exited. In this way, the loop can be used to ensure that all expired timestamps are deleted from the cache space, thereby ensuring that only non-expired storage content exists in the cache space.

[0144] It should be noted that, in order to avoid data omission, in a possible implementation, "current polling trigger time - polling interval" in the above paragraph can be the last polling trigger time, thus ensuring that data omission does not occur.

[0145] It should also be noted that this application does not limit the implementation method of the above step of "determining whether the timestamp with the longest storage time recorded in the updated storage content has expired". For example, it can be specifically: first use the current time minus the indicator detection time period involved in the key in the cache area where the timestamp is located (for example, Figure 3 The 30-minute time period represented by the string "30#minute" or Figure 3 The time period of 2 hours represented by the string "2#hour" in the example above is obtained to obtain the time starting point (that is, the current moment - the indicator detection time period); then, it is determined whether the timestamp is lower than the time starting point. If so, it can be determined that the timestamp has expired; if not, it can be determined that the timestamp has not expired.

[0146] Based on the relevant content of S4 above, it can be known that for a cluster management device used to manage several clusters, when it is determined that the polling trigger condition is met (for example, the current time = the last polling trigger time + the polling interval), the polling data interface can be called first to obtain the indicator detection update results corresponding to the several clusters, so that the indicator detection update results include the timestamps corresponding to the indicator values ​​that exceed the threshold within the time period of [current polling trigger time - polling interval, current polling trigger time]; then, based on the indicator detection update results corresponding to each cluster, the storage content in the cache space corresponding to each cluster is updated, so that the updated storage content is newly added with the timestamps corresponding to the indicator values ​​that exceed the threshold within the time period of [current polling trigger time - polling interval, current polling trigger time]; finally, the loop is used to determine whether the timestamp with the longest storage time recorded in the updated storage content has expired. If it has expired, the timestamp can be directly deleted; if it has not expired, the loop can be directly exited, so that the final storage content (for example, Figure 4 The updated cache content shown in the figure can accurately record the timestamps corresponding to the indicator values ​​exceeding the threshold within the time range of [current polling trigger time - indicator detection time period, current polling trigger time], so that it can be directly determined whether the corresponding rule items are met based on the number of timestamps in the stored content (for example, Figure 2 (e.g., the rule item "The number of YARN queued tasks greater than 5 occurs greater than 2 times within a 2-hour period").

[0147] S5: Determine the elastic scaling decision result for the target cluster based on the content stored in the cache space corresponding to the target cluster, and return to continue executing S3 above.

[0148] Among them, the elastic scaling decision result corresponding to the target cluster is used to indicate whether elastic scaling is performed on the target cluster; and this application does not limit the method for determining the "elastic scaling decision result corresponding to the target cluster", for example, it can be specifically: first, based on the content stored in the cache space corresponding to the target cluster, determine whether the elastic scaling rules corresponding to the target cluster are met; if the elastic scaling rules are met, it can be determined that elastic scaling needs to be performed on the target cluster; if the elastic scaling rules are not met, it can be determined that elastic scaling does not need to be performed on the target cluster.

[0149] In addition, the present application does not limit the implementation method of the above step of "determining whether the elastic scaling rules corresponding to the target cluster are met based on the content stored in the cache space corresponding to the target cluster". For example, when the elastic scaling rules corresponding to the target cluster include load type rule items, it may specifically include the following steps 21-22.

[0150] Step 21: Determine whether the load type rule item is satisfied based on the content stored in the cache area corresponding to the load type rule item to obtain a judgment result of the load type rule item, so that the judgment result can indicate whether the target cluster satisfies the load type rule item.

[0151] In this application, for the current polling process, when the elastic scaling rule corresponding to the target cluster includes a load type rule item (for example, Figure 2 For example, when the rule item "Within 2 hours, the number of YARN queued tasks is greater than 5 and the number of occurrences is greater than 2" is met, the number of timestamps stored in the cache area corresponding to the load type rule item is first counted; then a determination is made as to whether the number exceeds the occurrence threshold involved in the load type rule item (for example, 2). If so, it can be determined that the target cluster satisfies the load type rule item; if not, it can be determined that the target cluster does not satisfy the load type rule item.

[0152] It can be seen that for any load type rule item, it can be determined whether the load type rule item is satisfied by judging whether the number of timestamps in the cache area corresponding to the load type rule item (for example, the size of the List) exceeds the number threshold involved in the load type rule item.

[0153] Step 22: Based on the judgment result of the load type rule item, determine the judgment result of the elastic scaling rule corresponding to the target cluster, so that the judgment result can indicate whether the target cluster satisfies the elastic scaling rule.

[0154] It should be noted that this application does not limit the implementation method of step 22.

[0155] Based on the relevant content of steps 21 to 22 above, it can be seen that in some application scenarios (such as load scaling type scenarios), for the elastic scaling rule corresponding to a cluster, the storage content in the cache space corresponding to the elastic scaling rule can be used to determine whether each load type rule item in the elastic scaling rule is satisfied; then, based on the judgment results for these load type rule items, the judgment result of the elastic scaling rule is comprehensively determined, so that the judgment result can indicate whether the target cluster satisfies the elastic scaling rule.

[0156] In fact, in some application scenarios (for example, a mixed scenario of time scaling + load scaling), the elastic scaling rules corresponding to the target cluster mentioned above involve not only load type rule items, but also time type rule items. Therefore, in order to better improve the decision-making effect, this application also provides a possible implementation method of the process of determining the "elastic scaling decision result corresponding to the target cluster" mentioned above. In this implementation method, when the "elastic scaling rules corresponding to the target cluster" mentioned above also include time type rule items (for example, Figure 2 When the rule item "9 to 12 o'clock every day" is determined, the process of determining the "auto scaling decision result corresponding to the target cluster" may specifically include the following steps 31 and 32.

[0157] Step 31: Determine whether the current polling trigger time is within the time range represented by the above time type rule item, and obtain the determination result of the time type rule item.

[0158] The above "time range represented by the time type rule item" refers to the time range specified by the time type rule item. For example, when the time type rule item is Figure 2 For the rule item "9 to 12 o'clock every day" shown in FIG. 1 , the "time range represented by the time type rule item" may be the time range of [09:00, 12:00].

[0159] The "judgment result of the time type rule item" above is used to indicate whether the target cluster meets the time type rule item.

[0160] Based on the relevant content of step 31 above, it can be known that for the current polling process, when the elastic scaling rule corresponding to the target cluster includes a time type rule item (for example, Figure 2 When the rule item "9 to 12 o'clock every day" is shown, it can be determined that the time range represented by the time type rule item is the time range of [09:00, 12:00]. Therefore, it can be determined whether the current polling trigger time is within the time range of [09:00, 12:00]. If it is, it can be determined that the time type rule item is satisfied; if not, it can be determined that the time type rule item is not satisfied.

[0161] Step 32: Based on the judgment result of the time type rule item and the storage content in the cache space corresponding to the target cluster, determine the elastic scaling decision result corresponding to the target cluster.

[0162] It should be noted that this application does not limit the implementation of step 32.

[0163] Based on the relevant content of steps 31 to 32 above, it can be seen that for the current polling process, if the elastic scaling rule corresponding to the target cluster includes a load type rule item and a time type rule item, the judgment result of the time type rule item and the judgment result of the load type rule item can be obtained first; then these judgment results are combined to comprehensively determine the judgment result of the elastic scaling rule; finally, based on the judgment result of the elastic scaling rule, the elastic scaling decision result corresponding to the target cluster is determined.

[0164] In practice, the cluster management device described above can utilize the same polling mechanism to complete the elastic scaling decision processing for all clusters. Based on this, the present application also provides a possible implementation of S5 described above, which may specifically include determining the elastic scaling decision result for each cluster based on the content stored in the cache space corresponding to each cluster. It should be noted that the process for determining the elastic scaling decision result for each cluster is similar to the process for determining the elastic scaling decision result for the target cluster described above and will not be further described here for the sake of brevity.

[0165] Based on the relevant contents of S1 to S5 above, it can be seen that for the data processing method applied to the cluster management device provided in the embodiment of the present application, after creating an elastic scaling rule for the target cluster, first configure the cache space corresponding to the target cluster according to the elastic scaling rule; then, based on the cluster identifier of the target cluster and the indicator item identifier carried by the load type rule item in the elastic scaling rule, obtain the indicator detection status data of the load type rule item within the replenishment time range, and store the indicator detection status data in the cache space corresponding to the target cluster, so that the cache space records the status of the load type rule item within the time range of [last polling trigger time-indicator detection time period, last polling trigger time] (for example, which moments have reached the load type rule item). Type rule item represented by the load indicator condition); then, through polling, continuously obtain the indicator detection update result corresponding to the target cluster, so that the indicator detection update result is used to describe the indicator detection status update presented by the elastic scaling rule during the polling interval, and according to the indicator detection update result, update the storage content in the cache space corresponding to the target cluster, so that the content recorded in the storage space for the indicator detection time period is continuously updated, thereby ensuring the real-time nature of the storage content in the cache space, so that the elastic scaling decision result corresponding to the target cluster can be determined in real time based on the storage content in the cache space. In this way, the purpose of elastic scaling decision for a cluster can be achieved by means of one-time filling + continuous polling and updating.

[0166] Among them, because the filling time range is determined according to the indicator detection time period carried by the load type rule item and the last polling trigger time, so that the end time involved in the filling time range is the start time of the current polling, it can effectively overcome the adverse effects caused by the rule creation time not belonging to a certain polling trigger time (for example, the resource overhead caused by the inability to use the cluster polling process that already exists in the cluster management device), so as to achieve the purpose of smoothly joining the polling process without missing data, which is conducive to improving the elastic scaling decision-making effect of the cluster management device for the cluster.

[0167] In addition, for a cluster, a data completion process is performed when its rules are created, so that the cache space corresponding to the cluster stores the indicator detection status data within the time range of [last polling trigger time - indicator detection time period, last polling trigger time]. Therefore, in the subsequent continuous polling process, it is only necessary to obtain the indicator detection status update presented during the polling interval for storage and update. This can effectively improve the efficiency of elastic scaling decisions for a cluster.

[0168] In addition, for the cluster management device mentioned above, it can use only one interface (for example, the polling data interface) to obtain the indicator detection status update status of all clusters during the polling interval during each polling process. In this way, it can achieve the purpose of using the same polling mechanism to complete the elastic scaling decision process for all clusters, thereby effectively avoiding the adverse effects caused by configuring a polling process for each cluster, and thus helping to improve the elastic scaling decision effect of the cluster management device for the cluster.

[0169] In fact, in order to better improve the decision-making effect, the present application also provides another possible implementation of the above data processing method, which may specifically include the following steps 41 to 47.

[0170] Step 41: After creating an elastic scaling rule for the target cluster, configure the cache space corresponding to the target cluster according to the elastic scaling rule; the elastic scaling rule includes a load type rule item.

[0171] It should be noted that for the relevant content of step 41, please refer to the relevant content of S1 above.

[0172] Step 42: Read the last call time of the polling data interface from the preset cache area as the last polling trigger time, and determine the filling time range based on the time period carried by the load type rule item above and the last polling trigger time.

[0173] Among them, the preset cache area refers to a pre-set cache space used to cache the call time of the above polling data interface; and this application does not limit the implementation method of the preset cache area, for example, it can be implemented using redis.

[0174] The "last call time of the polling data interface" mentioned above refers to the time when the polling data interface was called during the most recent polling process for centralized polling processing for the cluster with elastic scaling decision configuration, which exists in the cluster management device mentioned above.

[0175] Please refer to the above text for details on the "Completion Time Range".

[0176] It should be noted that the present application does not limit the execution time of step 42. For example, the execution time of step 42 can be later than the creation time of the elastic scaling rule corresponding to the target cluster mentioned above.

[0177] Step 43: pass the cluster identifier of the target cluster, the indicator item identifier carried by the load type rule item, and the completion time range into the data completion interface, and call the data completion interface to obtain the indicator detection status data of the load type rule item within the completion time range.

[0178] It should be noted that, for the relevant content of step 43, please refer to the above.

[0179] Step 44: Determine whether the polling trigger condition is met. If so, execute steps 45 to 47 below; if not, return to continue executing step 44 above.

[0180] It should be noted that for the relevant content of step 44, please refer to the relevant content of S3 above.

[0181] Step 45: If the polling trigger condition is met, the polling data interface is called to obtain the indicator detection update result corresponding to the target cluster, and based on the indicator detection update result corresponding to the target cluster, the storage content in the cache space corresponding to the target cluster is updated; the indicator detection update result is used to describe the indicator detection status update presented by the elastic scaling rule during the polling interval.

[0182] It should be noted that for the relevant content of step 45, please refer to the above.

[0183] Step 46: Using the current call time of the polling data interface, update the data stored in the preset cache area.

[0184] The current calling time of the polling data interface refers to the time when the polling data interface is called during the current polling process.

[0185] Based on the relevant content of step 46 above, it can be known that for the current polling process, the time when the polling data interface is called in the current polling process can be obtained first, and the time of occurrence can be recorded in the preset cache area, so that the time recorded in the preset cache area can be used as the starting time of the data collection process involved in the next polling process. In this way, the purpose of recording the trigger time of each polling with the help of the preset cache area can be achieved, so that when a new cluster wants to be added to the polling process, the time recorded in the preset cache area can be directly used to perform a data completion process.

[0186] It should be noted that the present application does not limit the execution time of step 46. For example, the execution time of step 46 is later than the execution time of the above step “calling the polling data interface”.

[0187] Step 47: Determine the elastic scaling decision result corresponding to the target cluster based on the content stored in the cache space corresponding to the target cluster, and return to continue executing S3 above.

[0188] It should be noted that for the relevant content of step 47, please refer to the relevant content of S5 above.

[0189] Based on the relevant content of steps 41 to 47 above, it can be seen that in some application scenarios, for a cluster management device, it is not only necessary to store the data involved in all clusters with elastic scaling configurations in the cache space, but also to record the polling trigger time involved in the centralized polling processing of these clusters in real time in the cache space, which is conducive to improving decision-making results.

[0190] In fact, in order to better improve the user experience, this application also provides some possible implementation methods for the creation process of the above "elastic scaling rules", which are explained below with the help of the contents shown in ①-③.

[0191] ① In this application, users can configure elastic scaling rules based on natural language, using a declarative configuration method in the user interface interaction to clearly display the rule content and associated logic. This can effectively lower the configuration threshold of elastic scaling rules, allowing users to complete rule configuration easily and quickly.

[0192] ② In this application, elastic scaling rules can be obtained by combining time scaling types or load scaling types. This can effectively eliminate the negative effects of adjusting resources based solely on load or time, reduce the judgment errors caused by users relying on experience-based estimates, improve the security and effectiveness of resource control, and make the scaling effect more accurately meet expectations.

[0193] ③ In this application, users can set the elastic scaling rules and the elastic scaling policies of each cluster separately, so that the user can adjust the elastic scaling policies by binding or unbinding certain elastic scaling rules with certain elastic scaling policies. In this way, for some elastic scaling policies with certain common requirements, only one elastic scaling rule that meets the requirements needs to be configured, which not only effectively saves the configuration cost of the elastic scaling rules, but also effectively improves the reuse rate of the elastic scaling rules, which is conducive to improving user experience.

[0194] Based on the data processing method provided in the embodiment of the present application, the embodiment of the present application also provides a data processing device. Figure 5 Explain and illustrate. Figure 5 This is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. It should be noted that for the technical details of the data processing device provided in an embodiment of the present application, please refer to the relevant content of the data processing method above.

[0195] like Figure 5 As shown, the data processing device 500 provided in this embodiment of the application includes:

[0196] A cache configuration unit 501 is configured to configure cache space corresponding to a target cluster according to an elastic scaling rule created for the target cluster; the elastic scaling rule includes a load type rule item;

[0197] The data completion unit 502 is configured to obtain, based on the cluster identifier of the target cluster and the indicator item identifier carried by the load type rule item, the indicator detection status data of the load type rule item within the completion time range, and store the indicator detection status data in the cache space corresponding to the target cluster; the completion time range is determined based on the indicator detection time period carried by the load type rule item and the last polling trigger time;

[0198] The polling update unit 503 is used to determine whether the polling trigger condition is met. If so, the indicator detection update result corresponding to the target cluster is obtained, and the storage content in the cache space corresponding to the target cluster is updated based on the indicator detection update result corresponding to the target cluster; the indicator detection update result is used to describe the indicator detection status update status presented by the elastic scaling rule during the polling interval; based on the storage content in the cache space corresponding to the target cluster, the elastic scaling decision result corresponding to the target cluster is determined, and the step of determining whether the polling trigger condition is met is continued.

[0199] In one possible implementation, before creating an elastic scaling rule for a target cluster, the cluster management device is configured to perform elastic scaling decision processing on at least one to-be-managed cluster using the same polling mechanism;

[0200] The last polling trigger time refers to the triggering moment of the last polling process under the polling mechanism;

[0201] The polling triggering condition refers to a condition pre-set for the polling mechanism and used to trigger a polling process.

[0202] In one possible implementation, the data completion unit 502 is specifically configured to: pass the cluster identifier of the target cluster, the indicator item identifier carried by the load type rule item, and the completion time range to a data completion interface, and call the data completion interface to obtain the indicator detection status data of the load type rule item within the completion time range;

[0203] The polling update unit 503 is specifically configured to call a polling data interface to obtain an indicator detection update result corresponding to the target cluster.

[0204] In a possible implementation manner, the cluster management device is used to manage a plurality of clusters; the plurality of clusters includes the target cluster;

[0205] The polling update unit 503 is specifically used to: call the polling data interface to obtain the indicator detection update results corresponding to the several clusters; update the storage content in the cache space corresponding to each cluster based on the indicator detection update results corresponding to each cluster; determine the elastic scaling decision result corresponding to each cluster based on the storage content in the cache space corresponding to each cluster.

[0206] In a possible implementation manner, the data processing device 500 further includes:

[0207] A data reading unit is configured to read the last call time of the polling data interface from a preset cache area as the last polling trigger time before the cluster identifier of the target cluster, the indicator item identifier carried by the load type rule item, and the completion time range are passed into the data completion interface;

[0208] A time determination unit is used to determine the filling time range according to the time period carried by the load type rule item and the last polling trigger time.

[0209] In a possible implementation manner, the polling update unit 503 is further configured to update the data stored in the preset cache area using the current call time of the polling data interface.

[0210] In a possible implementation manner, the cache configuration unit 501 is specifically configured to:

[0211] Parsing the elastic scaling rule to obtain a rule tree; the elastic scaling rule includes a plurality of rule expression objects; the plurality of rule expression objects include a load type rule item; the rule tree includes a node corresponding to each of the rule expression objects;

[0212] According to the rule tree, a cache space corresponding to the target cluster is configured, where the cache space includes a cache area corresponding to each of the rule expression objects.

[0213] In a possible implementation manner, the cache space corresponding to the target cluster includes a cache area corresponding to the load type rule item;

[0214] The data completion unit 502 is specifically configured to store the indicator detection status data in a cache area corresponding to the load type rule item.

[0215] In a possible implementation manner, the elastic scaling rule further includes a time type rule item;

[0216] The polling update unit 503 is specifically used to: determine whether the current polling trigger time is within the time range represented by the time type rule item, and obtain a judgment result; based on the judgment result and the storage content in the cache space corresponding to the target cluster, determine the elastic scaling decision result corresponding to the target cluster.

[0217] Based on the relevant content of the above-mentioned data processing device 500, it can be known that for the data processing device 500 provided in the present application, after creating an elastic scaling rule for the target cluster, first configure the cache space corresponding to the target cluster according to the elastic scaling rule; then, based on the cluster identifier of the target cluster and the indicator item identifier carried by the load type rule item in the elastic scaling rule, obtain the indicator detection status data of the load type rule item within the replenishment time range, and store the indicator detection status data in the cache space corresponding to the target cluster, so that the cache space records the status of the load type rule item within the time range of [last polling trigger time-indicator detection time period, last polling trigger time] (for example, at which moments the load type rule item is reached). ), and then, by polling, continuously obtain the indicator detection update result corresponding to the target cluster, so that the indicator detection update result is used to describe the indicator detection status update presented by the elastic scaling rule during the polling interval, and according to the indicator detection update result, update the storage content in the cache space corresponding to the target cluster, so that the content recorded in the storage space for the indicator detection time period is continuously updated, thereby ensuring the real-time performance of the storage content in the cache space, so that the elastic scaling decision result corresponding to the target cluster can be determined in real time based on the storage content in the cache space. In this way, the purpose of elastic scaling decision for a cluster can be achieved by means of one-time filling + continuous polling and updating.

[0218] In addition, an embodiment of the present application also provides an electronic device, which includes a processor and a memory: the memory is used to store instructions or computer programs; the processor is used to execute the instructions or computer programs in the memory, so that the electronic device executes any implementation of the data processing method provided in the embodiment of the present application.

[0219] See also Figure 6 , which shows a schematic structural diagram of an electronic device 600 suitable for implementing the embodiments of the present disclosure. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0220] like Figure 6 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the electronic device 600 are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0221] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0222] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0223] The electronic device provided by the embodiment of the present disclosure and the method provided by the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0224] An embodiment of the present application further provides a computer-readable medium, in which instructions or computer programs are stored. When the instructions or computer programs are executed on a device, the device executes any implementation of the data processing method provided in the embodiment of the present application.

[0225] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0226] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0227] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0228] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device can perform the method.

[0229] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0230] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0231] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit / module does not, in some cases, limit the unit itself.

[0232] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0233] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0234] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems or devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0235] It should be understood that in this application, "at least one (item)" means one or more, and "more" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or plural.

[0236] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprise," "include," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a set of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0237] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0238] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.< / timestamp>

Claims

1. A data processing method for clusters under big data computing, characterized in that: Applied to a cluster management device, the method includes: After creating an elastic scaling rule for a target cluster, configuring a cache space corresponding to the target cluster according to the elastic scaling rule; the elastic scaling rule includes a load type rule item; According to the cluster identifier of the target cluster and the indicator item identifier carried by the load type rule item, obtain the indicator detection status data of the load type rule item within the replenishment time range, and store the indicator detection status data in the cache space corresponding to the target cluster; the replenishment time range is determined according to the indicator detection time period carried by the load type rule item and the last polling trigger time; Determine whether the polling trigger condition is met. If so, obtain the indicator detection update result corresponding to the target cluster, and update the content stored in the cache space corresponding to the target cluster based on the indicator detection update result corresponding to the target cluster; the indicator detection update result is used to describe the indicator detection status update presented by the elastic scaling rule during the polling interval; Determine the elastic scaling decision result corresponding to the target cluster based on the content stored in the cache space corresponding to the target cluster, and continue to execute the step of determining whether the polling trigger condition is met; Before creating an elastic scaling rule for a target cluster, the cluster management device is used to perform elastic scaling decision processing on at least one to-be-managed cluster using the same polling mechanism; The last polling trigger time refers to the triggering moment of the last polling process under the polling mechanism; The polling triggering condition refers to a condition pre-set for the polling mechanism and used to trigger a polling process.

2. The method according to claim 1, characterized in that The acquiring, based on the cluster identifier of the target cluster and the indicator item identifier carried by the load type rule item, indicator detection status data of the load type rule item within the replenishment time range includes: The cluster identifier of the target cluster, the indicator item identifier carried by the load type rule item, and the completion time range are passed into the data completion interface, and the data completion interface is called to obtain the indicator detection status data of the load type rule item within the completion time range; The obtaining of the indicator detection update result corresponding to the target cluster includes: Call the polling data interface to obtain the indicator detection update result corresponding to the target cluster.

3. The method according to claim 2, characterized in that The cluster management device is used to manage a plurality of clusters; the plurality of clusters includes the target cluster; The calling of the polling data interface to obtain the indicator detection update result corresponding to the target cluster includes: Calling the polling data interface to obtain the indicator detection update results corresponding to the several clusters; The updating process of the content stored in the cache space corresponding to the target cluster according to the indicator detection update result corresponding to the target cluster includes: Based on the indicator detection update results corresponding to each cluster, the stored content in the cache space corresponding to each cluster is updated; The determining, based on the content stored in the cache space corresponding to the target cluster, the elastic scaling decision result corresponding to the target cluster includes: The elastic scaling decision result corresponding to each cluster is determined according to the stored content in the cache space corresponding to each cluster.

4. The method according to claim 2, characterized in that Before transmitting the cluster identifier of the target cluster, the indicator item identifier carried by the load type rule item, and the filling time range to the data filling interface, the method further includes: Read the last call time of the polling data interface from the preset cache area as the last polling trigger time; The filling time range is determined according to the time period carried by the load type rule item and the last polling trigger time.

5. The method according to claim 2, characterized in that The method further comprises: The data stored in the preset cache area is updated using the current call time of the polling data interface.

6. The method according to claim 1, characterized in that Configure the cache space corresponding to the target cluster based on the elastic scaling rules created for the target cluster, including: Parsing the elastic scaling rule to obtain a rule tree; the elastic scaling rule includes a plurality of rule expression objects; the plurality of rule expression objects include a load type rule item; the rule tree includes a node corresponding to each of the rule expression objects; According to the rule tree, a cache space corresponding to the target cluster is configured, where the cache space includes a cache area corresponding to each of the rule expression objects.

7. The method according to claim 1, characterized in that The cache space corresponding to the target cluster includes a cache area corresponding to the load type rule item; Storing the indicator detection status data in a cache space corresponding to the target cluster includes: The indicator detection status data is stored in a cache area corresponding to the load type rule item.

8. The method according to any one of claims 1 to 7, characterized in that The elastic scaling rule also includes a time type rule item; The method further comprises: Determine whether the current polling trigger time is within the time range represented by the time type rule item, and obtain a determination result; The determining, based on the content stored in the cache space corresponding to the target cluster, the elastic scaling decision result corresponding to the target cluster includes: An elastic scaling decision result corresponding to the target cluster is determined according to the judgment result and the storage content in the cache space corresponding to the target cluster.

9. A data processing device for clusters under big data computing, characterized in that: include: A cache configuration unit, configured to configure a cache space corresponding to a target cluster according to an elastic scaling rule created for the target cluster; The elastic scaling rule includes a load type rule item; a data completion unit, configured to obtain, based on the cluster identifier of the target cluster and the indicator item identifier carried by the load type rule item, the indicator detection status data of the load type rule item within a completion time range, and store the indicator detection status data in a cache space corresponding to the target cluster; the completion time range is determined based on the indicator detection time period carried by the load type rule item and the last polling trigger time; A polling update unit, configured to determine whether a polling trigger condition is met, and if so, to obtain an indicator detection update result corresponding to the target cluster, and to update the content stored in the cache space corresponding to the target cluster based on the indicator detection update result corresponding to the target cluster; The indicator detection update result is used to describe the indicator detection status update presented by the elastic scaling rule during the polling interval; based on the content stored in the cache space corresponding to the target cluster, the elastic scaling decision result corresponding to the target cluster is determined, and the step of determining whether the polling trigger condition is met is continued; Before creating an elastic scaling rule for a target cluster, the cluster management device is used to perform elastic scaling decision processing on at least one to-be-managed cluster using the same polling mechanism; The last polling trigger time refers to the triggering moment of the last polling process under the polling mechanism; The polling triggering condition refers to a condition pre-set for the polling mechanism and used to trigger a polling process.

10. An electronic device, characterized in that: The device includes: a processor and a memory; The memory is used to store instructions or computer programs; The processor is configured to execute the instructions or computer program in the memory, so that the electronic device executes the method according to any one of claims 1 to 8.

11. A computer-readable medium, characterized in that The computer-readable medium stores instructions or a computer program, and when the instructions or the computer program are executed on a device, the device is caused to execute the method according to any one of claims 1 to 8.

Images