A method for rapid response to abnormalities of distributed computing resources
By monitoring the computing resource load of distributed nodes in real time, calculating abnormal capacity and filtering abnormal resources, the problems of difficulty in detecting and slow response in distributed systems are solved, and the rapid response and stability of the system are improved.
Patent Information
- Application Number
- CN202411139157.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-20
AI Technical Summary
In distributed computing systems, traditional fault detection methods are difficult to effectively identify abnormal behaviors, resulting in system performance degradation and difficulty in achieving rapid response, increasing the instability of the system and difficulty in detecting faults.
By monitoring the computing resource load of distributed nodes in real time, calculating the abnormal capacity, filtering out abnormal resources, and starting a fast response mechanism, reducing manual intervention in an automated way, improving the system's flexibility and computing task processing efficiency.
It realizes rapid response to distributed systems, reduces system performance degradation caused by exceptions, improves computing stability and high availability, and optimizes computing task processing and efficiency.
Smart Images

Figure CN118656220B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of response optimization, and in particular to a method for rapid response to abnormalities of distributed computing resources. Background Art
[0002] With the rapid development of information technology, distributed computing systems are widely used in complex computing tasks. The current mainstream methods for quickly responding to abnormalities in distributed computing resources, such as combining automated scheduling with prediction models or combining machine learning prediction models to identify potential resource bottlenecks or overloads so that measures can be taken in advance to avoid failures, are widely used in a variety of distributed tasks.
[0003] Since distributed systems involve multiple computing nodes and resources, the complexity and uncertainty of their computing processes increase, and the system is more prone to node failures, delays, resource exhaustion and other problems. The difficulty of monitoring and fault detection increases accordingly. Traditional fault detection methods face problems such as diverse fault types, intermittent failures, and high overhead of anomaly detection in distributed environments. More importantly, in distributed tasks, the distinction between normal and abnormal behaviors is relatively complex, making it difficult to achieve effective detection through simple rules or threshold alarms.
[0004] Therefore, a rapid response method for distributed computing resource anomalies by continuously detecting system performance and resource usage and setting relevant alarm mechanisms is the key to minimizing the system performance degradation caused by anomalies and further improving the stability and high availability of distributed systems. Summary of the invention
[0005] The purpose of the present invention is to propose a method for rapid response to anomalies of distributed computing resources to solve one or more technical problems existing in the prior art and at least provide a beneficial option or create conditions.
[0006] The present invention provides a method for rapid response to abnormalities of distributed computing resources, which monitors the load of computing resources in distributed nodes in real time to obtain monitoring data, calculates the abnormal tolerance of each distributed node through the monitoring data, screens out abnormal computing resources according to the abnormal tolerance of each distributed node, and initiates rapid response for abnormal computing resources in the distributed nodes. The method monitors the computing resource load of distributed nodes in real time, screens out abnormal resources by calculating the abnormal tolerance and initiates a rapid response mechanism, can fully improve the computing stability of the nodes, reduce manual intervention with an automated response method, enhance the elasticity and adjustment ability of the entire distributed system to cope with load abnormalities, and effectively optimize the processing process and efficiency of computing tasks.
[0007] In order to achieve the above object, according to one aspect of the present invention, a method for quickly responding to an abnormality of a distributed computing resource is provided, the method comprising the following steps:
[0008] Monitor the load of computing resources in distributed nodes in real time to obtain monitoring data;
[0009] By monitoring the data, the abnormal tolerance of each distributed node is calculated;
[0010] Filter out abnormal computing resources based on the abnormal tolerance of each distributed node;
[0011] In distributed nodes, a quick response is initiated for abnormal computing resources.
[0012] Furthermore, the distributed node is a physical node or a virtual node, there are multiple distributed nodes, each distributed node is configured with a CPU and / or a GPU (ie, computing resources), and all distributed nodes are used to execute computing tasks and work together.
[0013] Furthermore, the computing resources in the distributed node refer to the CPU and / or GPU configured in the distributed node; the load of the computing resources in the distributed node refers to the CPU utilization per second or the GPU utilization per second configured in the distributed node, or refers to the sum of the CPU utilization per second and the GPU utilization per second configured in the distributed node.
[0014] Furthermore, the load of computing resources in distributed nodes is monitored in real time, and the method for obtaining monitoring data is specifically as follows:
[0015] Let the number of distributed nodes be N, let Dnode(i) be the i-th one of the N distributed nodes, set the time period T to be Tk (i.e., time period T consists of Tk seconds), and within time period T, record the load of the computing resources in each distributed node in real time at intervals of one second, and record Tk load data in total. Then, each distributed node corresponds to Tk load data within time period T, and the array LC(i) is used to store the load data corresponding to Dnode(i);
[0016] The arrays LC(1), LC(2), ..., LC(N) corresponding to the N distributed nodes Dnode(1), Dnode(2), ..., Dnode(N) are used as monitoring data.
[0017] Preferably, the specific value of Tk is set to [5000, 8000], and the time period T is generally selected as a sub-period in the process of distributed nodes executing large-scale computing tasks.
[0018] Further, the method for calculating the anomaly tolerance of each distributed node through monitoring data is specifically as follows:
[0019] Set the sub-algorithm as: create a blank array Bv i (for storing all sub-tolerance values), filter out all sub-tolerance values in the array LC(i), and add all sub-tolerance values in the array LC(i) to the array Bv i ; where the definition of the sub-tolerance value in LC(i) is: represent any value in LC(i) as LC(i,x). If LC(i,x) satisfies LC(i,x) < LC(i,x - 1) ≤ LC(i,x + 1), or LC(i,x) satisfies LC(i,x + 1) ≤ LC(i,x - 1) < LC(i,x), then LC(i,x) is called a sub-tolerance value in LC(i);
[0020] In the sub-algorithm, traverse i from i = 1 to i = N, and then obtain N arrays Bv1, Bv2, …, Bv N ;
[0021] Denote that the array Bv i contains M i values (since the number of sub-tolerance values in the array LC(i) is different, so M i is different), represent the j-th value in the array Bv i as Bv i (j), where j is the serial number, j = 1, 2, …, M i , and represent the average value of the array Bv i as Z i (that is, the average value of the M i values in the array Bv i );
[0022] Represent the anomaly tolerance of the distributed node Dnode(i) as Abt(i). The calculation method of Abt(i) is: add the M i values in the array Bv i respectively to all values in the arrays in N arrays Bv1, Bv2, …, Bv N whose array average value is less than or equal to Z i to obtain M i first sub-values;
[0023] Accumulate these M i first sub-values with Z i to obtain a second sub-value;
[0024] Accumulate all values in N arrays Bv1, Bv2, …, Bv N whose array average value is greater than or equal to Z iAll the values in the array of get the third sub-value;
[0025] The value obtained by dividing the third sub-value by the second sub-value is used as the abnormal tolerance Abt(i) of the distributed node Dnode(i).
[0026] Beneficial effect of this step: In solving large-scale computing tasks, computing resources in multiple distributed nodes are often required to work in coordination (in parallel). In the long-duration solving process, computing resource anomalies may occur due to various reasons, such as thread deadlock, data overload or network attack, etc. Therefore, it is necessary to monitor the status of computing resources abnormally to prevent the solving task from being interrupted or delayed. The method of this step calculates the sub-capacity value and abnormal tolerance by real-time analysis of the monitoring data. The sub-capacity value represents the value in the monitoring data of each distributed node that changes from the previous and next values (if a certain value, that is, the non-sub-capacity value, is consistent with the change trend of the previous and next values, that is, the slope is the same, and it is rising or falling, then the value is relatively not of high analytical value). The abnormal tolerance obtained based on the sub-capacity value and the monitoring data of other distributed nodes as the calculation basis reflects the weight of the abnormal risk of the corresponding distributed node relative to other distributed nodes in the current solving task. The abnormal tolerance of all distributed nodes is associated and screened to locate the abnormal computing resources, which can warn of computing resource anomalies in advance. At the same time, a rapid response mechanism is used to ensure that the computing task can be completed continuously and stably, thereby reducing computing interruptions and improving computing efficiency.
[0027] The principle of this step is as follows: the calculation of abnormal tolerance is mainly based on the sub-capacity value in the monitoring data. Since there are a lot of monitoring data, in order to improve the accuracy of the results, only part of the data that can reflect the status of computing resources needs to be selected. Therefore, the sub-capacity value with inconsistent change trends before and after the monitoring data is selected, and the abnormal tolerance is calculated based on the sub-capacity value. The calculation formula of the abnormal tolerance utilizes the comprehensive weights of other data (i.e., the monitoring data of all distributed nodes except Abt(i)) and target data (i.e., the monitoring data of Abt(i)), and uses the mean value for secondary balance, thereby completing the calculation of the abnormal tolerance, ensuring the accuracy and real-time nature of the calculation, and providing a reliable basis for the screening and rapid response of abnormal resources.
[0028] Furthermore, according to the abnormal tolerance of each distributed node, the method for screening out abnormal computing resources is specifically as follows:
[0029] S1, store N abnormal tolerances Abt(1), Abt(2), ..., Abt(N) in array Abt, record w i=|Abt(i+1)-Abt(i)|, traverse i from i=1 to i=N-1 to obtain w(1), w(2), ..., w(N-1), and take the maximum value among w(1), w(2), ..., w(N-1) as w MAX , take the minimum value among Abt(1), Abt(2),…,Abt(N) and record it as Abt(MIN) (minimum abnormal tolerance), and record Ck=Abt(MIN)+w MAX ;
[0030] Create a blank array Zk, which is used to store the abnormal distributed node serial numbers, set the variable k=1, and go to S2;
[0031] S2, if Abt(k) is greater than Ck, add the current value of k to the array Zk and go to S3;
[0032] S3, if the value of variable k is less than N, increase the value of k by 1 and go to S2; if the value of k is equal to or greater than N, go to S4;
[0033] S4, let the array Zk contain R values, and use Zk(r) to represent the r-th value in the array Zk, where r is the sequence number, r=1, 2, …, R; mark the computing resources in the R distributed nodes Dnode(Zk(1)), Dnode(Zk(2)), …, Dnode(Zk(R)) as abnormal computing resources.
[0034] The beneficial effect of this step is: by comparing the difference in the abnormal tolerance corresponding to each distributed node (calculating w i ), the abnormal computing resources are screened out based on the minimum abnormal tolerance and the maximum difference value, which can accurately screen out all values within the abnormal tolerance that are significantly different from other abnormal tolerances, and mark the computing resources corresponding to these values as abnormal computing resources, thereby accurately marking the abnormal nodes.
[0035] The principle of this step is: since the loads between distributed nodes will affect each other, it is difficult to accurately reflect the degree of abnormality of the nodes only by sorting the size of the abnormality tolerance as the screening basis. Therefore, the correlation of the abnormality tolerance of each node (that is, calculating the difference value) is used as the screening basis; consider the following situation: when there are several outliers (values that are significantly different from other abnormal tolerances) in N abnormal tolerances, for the neighboring values of these outliers (the previous value or the next value of the outlier), w MAX The selection of w will inevitably be generated from the difference between these outliers and their neighboring values, because the difference between the values that are not outliers and their neighboring values is very small, so they will naturally not be selected as w MAX , when w is selected MAX After that, add w to the minimum abnormal tolerance MAXAs the screening threshold Ck, it can be seen that most of the values close to the minimum abnormal tolerance Abt(MIN) are smaller than the threshold Ck. Only those abnormal tolerances far away from the minimum abnormal tolerance will exceed Ck. Therefore, the computing resources corresponding to these abnormal tolerances far away from the minimum abnormal tolerance (that is, Abt(k) is greater than Ck, and Ck accurately defines how large the value is to be defined as far away) are marked as abnormal computing resources to complete the screening process.
[0036] Furthermore, in a distributed node, for an abnormal computing resource, a method for initiating a quick response is specifically: suspending computing tasks being processed by the abnormal computing resource, and transferring these tasks to other distributed nodes.
[0037] The beneficial effect of this step is: through automated anomaly detection and response, and by timely suspending the task processing of abnormal computing resources, it is avoided that it further affects the performance of the entire distributed system, and the elasticity and adaptability of the cluster are enhanced.
[0038] Preferably, for abnormal computing resources, the steps of initiating a quick response method also include: marking the distributed node where the abnormal computing resource is located as an abnormal node, and the current computing task is no longer assigned to the abnormal node. At the same time, the abnormal node is isolated from the distributed cluster composed of multiple distributed nodes, and all abnormal nodes are restarted or restored and added to the monitoring.
[0039] The beneficial effects of the present invention are as follows: the method monitors the computing resource load of distributed nodes in real time, screens out abnormal resources by calculating the abnormal tolerance and initiates a rapid response mechanism, which can fully improve the computing stability of the nodes, reduce manual intervention with an automated response method, enhance the elasticity and adjustment capabilities of the entire distributed system to cope with load anomalies, and effectively optimize the processing process and efficiency of computing tasks. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 Shown is a flow chart of a method for rapid response to abnormalities of distributed computing resources. DETAILED DESCRIPTION
[0041] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention, so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments and features in the embodiments of this application can be combined with each other without conflict.
[0042] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.
[0043] like Figure 1 The flowchart of a method for rapidly responding to abnormalities in distributed computing resources according to the present invention is shown below. Figure 1 A method for quickly responding to anomalies of distributed computing resources according to an embodiment of the present invention is described below.
[0044] The present invention provides a method for quickly responding to abnormalities of distributed computing resources, the method comprising the following steps:
[0045] Monitor the load of computing resources in distributed nodes in real time to obtain monitoring data;
[0046] By monitoring the data, the abnormal tolerance of each distributed node is calculated;
[0047] Filter out abnormal computing resources based on the abnormal tolerance of each distributed node;
[0048] In distributed nodes, a quick response is initiated for abnormal computing resources.
[0049] Furthermore, the distributed nodes are physical nodes or virtual nodes. There are multiple distributed nodes, each of which is configured with a CPU and / or a GPU (i.e., computing resources). All distributed nodes are used to execute computing tasks and work together to achieve high-performance computing solutions and large-scale data processing.
[0050] Furthermore, the computing resources in the distributed node refer to the CPU and / or GPU configured in the distributed node; the load of the computing resources in the distributed node refers to the CPU utilization per second or the GPU utilization per second configured in the distributed node, or refers to the sum of the CPU utilization per second and the GPU utilization per second configured in the distributed node.
[0051] Furthermore, the load of computing resources in distributed nodes is monitored in real time, and the method for obtaining monitoring data is specifically as follows:
[0052] Let the number of distributed nodes be N, and denote Dnode(i) as the i-th among these N distributed nodes, where i is the serial number, i = 1, 2, …, N. Set a time period T of length Tk (i.e., the time period T consists of Tk seconds). During the time period T, record the load of computing resources in each distributed node in real time at one-second intervals, and a total of Tk load data are recorded. Then each distributed node corresponds to Tk load data during the time period T, and store the load data corresponding to Dnode(i) in the array LC(i).
[0053] Take the arrays LC(1), LC(2), …, LC(N) corresponding to the N distributed nodes Dnode(1), Dnode(2), …, Dnode(N) as monitoring data.
[0054] Specifically, the specific value of Tk is set to 6000.
[0055] Furthermore, the method for calculating the exception tolerance of each distributed node through the monitoring data is as follows:
[0056] Set the sub-algorithm as: create a blank array Bv i (for storing all sub-tolerance values), screen out all sub-tolerance values in the array LC(i), and add all sub-tolerance values in the array LC(i) to the array Bv i ; where the definition of the sub-tolerance value in LC(i) is: represent any value in LC(i) as LC(i,x). If LC(i,x) satisfies LC(i,x) < LC(i,x - 1) ≤ LC(i,x + 1), or LC(i,x) satisfies LC(i,x + 1) ≤ LC(i,x - 1) < LC(i,x), then LC(i,x) is called a sub-tolerance value in LC(i).
[0057] In the sub-algorithm, traverse i from i = 1 to i = N, and then obtain N arrays Bv1, Bv2, …, Bv N ;
[0058] Denote the array Bv i contains M i values (since the number of sub-tolerance values in the array LC(i) is different, so M i is different), represent the j-th value in the array Bv i as Bv i (j), where j is the serial number, j = 1, 2, …, M i , represent the average value of the array Bv i as Z i (i.e., the average value of the M i values in the array Bv i ), then each array Bv iCorresponding to an array mean Z i ;
[0059] Abt(i) represents the abnormal tolerance of the distributed node Dnode(i). The calculation method of Abt(i) is: Substitute the array Bv i M i values, respectively, with N arrays Bv1, Bv2, ..., Bv N The average value of the group is less than or equal to Z i Add all the values in the array to get M i The first subvalue;
[0060] This M i The first subvalue and Z i Accumulate and obtain the second sub-value;
[0061] Accumulate N arrays Bv1, Bv2, ..., Bv N The average value of the group is greater than or equal to Z i All the values in the array of get the third sub-value;
[0062] The value obtained by dividing the third sub-value by the second sub-value is taken as the abnormal tolerance Abt(i) of the distributed node Dnode(i);
[0063] The corresponding mathematical expression is:
[0064] ;
[0065] In the formula, ∑BvGM i {} and ∑BvSM i {} represent BvGM i The sum of all values within {} and BvSM i The sum of all values in {}, BvGM i {} represents N arrays Bv1, Bv2, ..., Bv N The average value of the group is less than or equal to Z i The set of all values in the array, BvSM i {} represents N arrays Bv1, Bv2, ..., Bv N The average value of the group is greater than or equal to Z i The set of all values in the array.
[0066] Furthermore, according to the abnormal tolerance of each distributed node, the method for screening out abnormal computing resources is specifically as follows:
[0067] S1, store N abnormal tolerances Abt(1), Abt(2), ..., Abt(N) in array Abt, record w i=|Abt(i+1)-Abt(i)|, traverse i from i=1 to i=N-1 to obtain w(1), w(2), ..., w(N-1), and take the maximum value among w(1), w(2), ..., w(N-1) as w MAX , take the minimum value among Abt(1), Abt(2),…,Abt(N) and record it as Abt(MIN) (minimum abnormal tolerance), record Ck=Abt(MIN)+w MAX ;
[0068] Create a blank array Zk, which is used to store the abnormal distributed node serial numbers, set variable k=1, and go to S2; variable k is used to traverse Abt(1) to Abt(N), and Abt(k) represents the kth value corresponding to k in N values Abt(1), Abt(2),…,Abt(N);
[0069] S2, if Abt(k) is greater than Ck, add the current value of k to the array Zk and go to S3;
[0070] S3, if the value of variable k is less than N, increase the value of k by 1 and go to S2; if the value of k is equal to or greater than N, go to S4;
[0071] S4, let the array Zk contain R values, and use Zk(r) to represent the r-th value in the array Zk, where r is the sequence number, r=1, 2, …, R; mark the computing resources in the R distributed nodes Dnode(Zk(1)), Dnode(Zk(2)), …, Dnode(Zk(R)) as abnormal computing resources.
[0072] Table 1 is a comparative diagram of the effects of the above-mentioned method of screening out abnormal computing resources and the method of screening out abnormal computing resources based on setting an early warning threshold on the screening accuracy; Method 1 is a screening method based on setting an early warning threshold, and Method 2 is a screening method of screening out abnormal computing resources based on abnormal tolerance in the present invention; the specific early warning threshold in Method 1 is set by empirical rules;
[0073] The distributed nodes in the experimental environment all use Dell PowerEdge R740 with the same hardware configuration. The computing tasks distributed to the distributed nodes include high-load data processing tasks and network-intensive tasks. Fault injection includes malicious occupation of CPU utilization, increased network latency, and memory leakage.
[0074] The comparative experiment sets up 5 control groups. The total number of distributed nodes selected in each control group is 100, 80, 60, 40, and 20 respectively. Multiple nodes are randomly selected in each control group at a ratio of 1:5 through the ChaosMonkey tool and faults are injected into these multiple nodes to obtain abnormal nodes. A time period of 8000 seconds is set and all distributed nodes in each control group are allowed to process computing tasks during this period. At the same time, abnormal computing resources are screened using methods one and two. After the end of this period, the accuracy of abnormal resources marked by method two is significantly higher than that of abnormal resources marked by method one.
[0075] The screening accuracy rates of method 1 in the five control groups were 35.00%, 37.50%, 75.00%, 62.50% and 100% respectively; the screening accuracy rates of method 2 in the five control groups were 90.00%, 81.25%, 83.33%, 87.50% and 100% respectively. It can be seen that when the total number of distributed nodes is large, the screening effect of the method for screening abnormal computing resources provided by the present invention is significantly better than the screening method based on setting an early warning threshold.
[0076]
[0077] Table description: Tr is the total number of distributed nodes selected in each group, NoarS is the number of abnormal nodes selected in each group, NoarBymethod1 and NoarBymethod2 represent the correct number of abnormal nodes identified by executing method 1 and method 2, respectively.
[0078] Furthermore, in a distributed node, for an abnormal computing resource, a method for initiating a quick response is specifically: suspending computing tasks being processed by the abnormal computing resource, and transferring these tasks to other distributed nodes.
[0079] Specifically, for abnormal computing resources, the steps of initiating a rapid response method also include: marking the distributed node where the abnormal computing resource is located as an abnormal node, and the current computing task is no longer assigned to the abnormal node. At the same time, the abnormal node is isolated from the distributed cluster composed of multiple distributed nodes, and all abnormal nodes are restarted or restored and added to the monitoring.
[0080] The present invention provides a method for rapid response to abnormalities of distributed computing resources, by real-time monitoring of the load of computing resources in distributed nodes, obtaining monitoring data, calculating the abnormal tolerance of each distributed node through the monitoring data, screening out abnormal computing resources according to the abnormal tolerance of each distributed node, and starting a rapid response for abnormal computing resources in the distributed nodes. The method can fully improve the computing stability of the node by monitoring the computing resource load of the distributed node in real time, and screening out abnormal resources and starting a rapid response mechanism by calculating the abnormal tolerance, thereby reducing manual intervention with an automated response method, enhancing the elasticity and adjustment ability of the entire distributed system to cope with load anomalies, and effectively optimizing the processing process and efficiency of computing tasks. Although the description of the present invention has been quite detailed and several embodiments have been described in particular, it is not intended to be limited to any of these details or embodiments or any special embodiments, so as to effectively cover the predetermined scope of the present invention. In addition, the above describes the present invention with embodiments that the inventor can foresee, and its purpose is to provide a useful description, and those non-substantial changes to the present invention that have not yet been foreseen can still represent equivalent changes to the present invention.
Claims
1. A method for rapid response to abnormalities of distributed computing resources, characterized in that: The method comprises the following steps: Monitor the load of computing resources in distributed nodes in real time to obtain monitoring data; By monitoring the data, the abnormal tolerance of each distributed node is calculated; Filter out abnormal computing resources based on the abnormal tolerance of each distributed node; In distributed nodes, initiate a quick response to abnormal computing resources; The method for real-time monitoring of the load of computing resources in distributed nodes and obtaining monitoring data is as follows: Suppose the number of distributed nodes is N, Dnode(i) is the i-th one of the N distributed nodes, and a time period T is set to be Tk. Within the time period T, the load of the computing resources in each distributed node is recorded in real time at intervals of one second, and a total of Tk load data are recorded. Then, each distributed node corresponds to Tk load data within the time period T, and the array LC(i) is used to store the load data corresponding to Dnode(i); the arrays LC(1), LC(2), ..., LC(N) corresponding to the N distributed nodes Dnode(1), Dnode(2), ..., Dnode(N) are used as monitoring data; The method for calculating the abnormal tolerance of each distributed node through monitoring data is as follows: Set the sub-algorithm as: create an empty array Bv i , filter out all sub-capacity values in the array LC(i), and add all sub-capacity values in the array LC(i) to the array Bv i ; where the definition of the sub-capacity value in LC(i) is: represent any value in LC(i) as LC(i,x). If LC(i,x) satisfies LC(i,x) < LC(i,x - 1) ≤ LC(i,x + 1), or LC(i,x) satisfies LC(i,x + 1) ≤ LC(i,x - 1) < LC(i,x), then LC(i,x) is called a sub-capacity value in LC(i); In the sub-algorithm, traverse i from i=1 to i=N, and then obtain N arrays Bv1, Bv2, …, Bv N ; Remember array Bv i Contains M i Value, in Bv i (j) represents the array Bv i The jth value in M, j is the serial number, j=1,2,…,M i , with Z i Represents array Bv i The average value of Abt(i) represents the abnormal tolerance of the distributed node Dnode(i). The calculation method of Abt(i) is: Substitute the array Bv i M i values, respectively, with N arrays Bv1, Bv2, ..., Bv N The average value of the group is less than or equal to Z i Add all the values in the array to get M i The first sub-value; this M i The first subvalue and Z i Accumulate and get the second sub-value; accumulate N arrays Bv1, Bv2, ..., Bv N The average value of the group is greater than or equal to Z i All values in the array of are obtained to obtain the third sub-value; the value obtained by dividing the third sub-value by the second sub-value is used as the abnormal tolerance Abt(i) of the distributed node Dnode(i); According to the abnormal tolerance of each distributed node, the method of screening out abnormal computing resources is as follows: S1, store N abnormal tolerances Abt(1), Abt(2), ..., Abt(N) in array Abt, record w i =|Abt(i+1)-Abt(i)|, traverse i from i=1 to i=N-1 to obtain w(1), w(2), ..., w(N-1), and take the maximum value among w(1), w(2), ..., w(N-1) as w MAX , take the minimum value among Abt(1), Abt(2),…,Abt(N) and record it as Abt(MIN), record Ck=Abt(MIN)+w MAX ; Create a blank array Zk, which is used to store the abnormal distributed node serial numbers, set the variable k=1, and go to S2; S2, if Abt(k) is greater than Ck, add the current value of k to the array Zk and go to S3; S3, if the value of variable k is less than N, increase the value of k by 1 and go to S2; if the value of k is equal to or greater than N, go to S4; S4, let the array Zk contain R values, and use Zk(r) to represent the r-th value in the array Zk, where r is the sequence number, r=1, 2, …, R; mark the computing resources in the R distributed nodes Dnode(Zk(1)), Dnode(Zk(2)), …, Dnode(Zk(R)) as abnormal computing resources.
2. A method for rapid response to abnormal distributed computing resources according to claim 1, characterized in that: A distributed node is a physical node or a virtual node. There are multiple distributed nodes. Each distributed node is configured with a CPU and / or a GPU. All distributed nodes are used to execute computing tasks and work together.
3. A method for rapid response to abnormal distributed computing resources according to claim 1, characterized in that: The computing resources in the distributed node refer to the CPU and / or GPU configured in the distributed node; the load of the computing resources in the distributed node refers to the CPU utilization per second or the GPU utilization per second configured in the distributed node, or the sum of the CPU utilization per second and the GPU utilization per second configured in the distributed node.
4. A method for rapid response to abnormal distributed computing resources according to claim 1, characterized in that: In a distributed node, for an abnormal computing resource, a method for initiating a quick response is specifically: suspending computing tasks being processed by the abnormal computing resource and transferring these tasks to other distributed nodes.
