A method and system for online detection of host activity

Abstract

This invention belongs to the field of computing power scheduling technology, and particularly relates to an online detection method and system for host activity. First, it acquires the sampling sequence of monitoring indicators and the corresponding Unix timestamp sequence of the target host within a preset time window, and performs anomaly detection on the sampling sequence. Second, it performs a (-N, N) transformation on the Unix timestamp sequence to obtain an N-transformation sequence. Then, using the sampling sequence, the activity threshold of the preset monitoring indicators, and the N-transformation sequence, it calculates the active time percentage, weight, and activity amplitude. Using the active time percentage and activity amplitude, it calculates the activity index of the target host. Finally, it acquires new sampling data, repeats the above operations, and calculates a new activity index for the target host. The activity index proposed in this invention solves the problem of comparative analysis between different hosts, thereby providing a scientific and reasonable basis for judgment in computing power scheduling, load balancing, and energy consumption reduction, and improving the accuracy of corresponding decisions.

Description

Technical Field

[0001] This invention belongs to the field of computing power scheduling technology, and in particular relates to an online detection method and system for host activity. Background Technology

[0002] Computing power is a new core productive force in the digital economy era. With the implementation of the national integrated big data center system and the East-West data center project, the importance of computing power scheduling to the development of the computing power economy is becoming increasingly prominent. Based on ubiquitous network connectivity, computing power scheduling integrates multi-level computing resources, improves the utilization efficiency of computing resources through unified management and collaborative scheduling of heterogeneous computing resources, provides unified computing power services to the outside world, and achieves consistency in user experience and flexibility in services.

[0003] Resource status awareness and assessment is one of the common fundamental technologies for building computing power scheduling systems. The ability to assess the operational status of computing resources in a timely and accurate manner directly affects the effectiveness of scheduling decisions. Detecting the activity of hosts is one of the effective ways to perceive and assess resource status. Hosts with high activity can reduce migration and maintain system stability, while hosts with low activity can be assigned more tasks or shut down, thereby improving resource utilization or reducing data center energy consumption.

[0004] For example, Chinese patent application publication number CN106789424A discloses a server activity detection system and method. The server activity detection system includes: a monitoring data acquisition module for collecting server monitoring data; an operation and maintenance event acquisition module for collecting server operation and maintenance events and obtaining server operation and maintenance data from the operation and maintenance events; a data preprocessing module for preprocessing the monitoring data and the operation and maintenance data, extracting feature information, and generating input vectors for a training module; a training module for training on pre-prepared server sample data to obtain a set of support vectors; and a detection module for using the support vectors to detect the activity of the server to be detected and outputting the detection results.

[0005] For example, Chinese patent CN112182100A discloses a method and system for synchronizing state data of a cloud management platform. It configures an activity convergence factor for various operations that affect the state changes of various cloud computing resources. When an operation that affects the state changes of a resource occurs, the activity is set to the maximum value and the resource is assigned to the active zone. The activity of the resource is deducted according to the corresponding activity convergence factor. When the activity is 0, the resource is assigned to the quiescent zone. The state synchronization thread pool periodically generates a fixed number of state synchronization threads and allocates them to the active zone and the quiescent zone in a preset ratio. Each thread obtains the latest state of a resource in the corresponding zone and updates it synchronously in the database of the cloud management platform.

[0006] However, existing methods for detecting host activity primarily involve threshold detection based on the average value of resource sampling data over a period of time; for example, a CPU utilization exceeding 35% indicates an active host. While this method is simple and convenient, it ignores trend changes in the sampling data sequence. For instance, a CPU utilization drop from 60% to 10% and a CPU utilization increase from 10% to 60% both have an average of 35%, but their activity levels are clearly not directly equivalent. Furthermore, the importance of sampling data at different time periods for assessing activity varies; sampling data closer to the detection time is more important for determining activity. Existing technologies largely ignore these issues. To address these problems, this invention provides an online host activity detection method. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to address the shortcomings of the prior art by providing an online method for detecting host activity. This method not only solves the problem of the influence of trends on judging host activity by using an updatable activity index, but also realizes comparative analysis between different hosts.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A method for online detection of host activity includes the following steps:

[0010] S1: Obtain the sampling sequence of monitoring metrics of the target host within a preset time window and the corresponding Unix timestamp sequence;

[0011] S2: Perform outlier detection and replacement on the sampled sequence, and transform the Unix timestamp sequence to obtain the N-transformation sequence;

[0012] S3: Calculate the percentage of active time using the sampling sequence and the active threshold of the preset monitoring indicators;

[0013] S4: Using the activity threshold of the preset monitoring indicators, the sampling sequence and the N-transformation sequence, the weight and activity amplitude are calculated;

[0014] S5: Calculate the activity index of the target host using the percentage of active time and the activity level;

[0015] S6: Obtain new sampling data, update the sampling sequence and timestamp sequence, repeat S2 to S5, and calculate the new activity index of the target host.

[0016] To achieve the above objectives, the present invention provides the following technical solution:

[0017] Specifically, the preset time window length is The calculation formula is:

[0018] ,

[0019] in, This indicates the current moment when the activity detection is performed. This indicates the first historical sample taken for activity detection. time;

[0020] The sampling sequence is ,in Indicates the distance to time The most recent sample, i.e., the latest sampled data; the sampling sequence. The corresponding Unix timestamp sequence is ,in Indicates the first A timestamp; a Unix timestamp refers to converting the time corresponding to the original sample into a Unix timestamp.

[0021] Specifically, the outlier detection method is the Isolation Forest algorithm, and its specific process is as follows:

[0022] The sampling sequence is input into the Isolation Forest algorithm. If the output label is 1, it is a normal value; if it is -1, it is an outlier.

[0023] The outlier replacement method is as follows: Assume the sampling sequence middle If the value is an outlier, the replacement value is calculated using the following formula. ,

[0024] ,

[0025] in, Indicates outlier Replacement value, Indicates the first sampled sequence i -1 non-abnormal sampled value, Indicates the first sampled sequence i+ One non-abnormal sampled value;

[0026] If the abnormal value is and Then, the replacement value is calculated using the following formula:

[0027] ,

[0028] ,

[0029] in, and For sequential outliers and Replacement value, , , and These are the non-abnormal sampled values ​​in the sampling sequence.

[0030] Specifically, in S2 The specific conversion process is as follows:

[0031] Using the Unix timestamp sequence The transformed sequence is calculated using the following formula. The formula is as follows:

[0032] ,

[0033] in, Indicates the th element in the transformed sequence One value, Represents the 1st in a Unix timestamp sequence A timestamp, This represents the first timestamp in the Unix timestamp sequence. Represents the 1st in a Unix timestamp sequence A timestamp, express The set of natural numbers under transformation.

[0034] Specifically, the formula for calculating the percentage of active time is as follows:

[0035] ,

[0036] in, Indicates the percentage of active time. Represents the sampling sequence The Middle Each sample value, Indicates preset monitoring indicators The activity threshold Indicates the number of sampled sequences. Specifically:

[0037] ,

[0038] in, express The value of .

[0039] Specifically, the weights in S4 The specific calculation method is as follows:

[0040] ,

[0041] in, Indicates the first Each weight, Represents the transformed sequence The One value, e Represents the natural base;

[0042] The specific calculation method for the activity level is as follows:

[0043] ,

[0044] in, Indicates the level of activity.

[0045] Specifically, the activity index The calculation method is as follows:

[0046] ,

[0047] in, Indicates the activity index, Indicates the percentage of active time. Indicates the level of activity.

[0048] An online host activity detection system includes a data acquisition module, an anomaly detection module, a data preprocessing module, and an activity index calculation module.

[0049] The data acquisition module is used to obtain historical sampling data and sampling time of the corresponding monitoring indicators of the target host within a set time interval from the monitoring system, as well as real-time sampling data and sampling time.

[0050] The anomaly detection module is used to detect and replace anomalies in both historical and real-time collected data.

[0051] The data preprocessing module is used to convert the raw time of the sampled data into a Unix timestamp and to process the sampled data. Transformation;

[0052] The activity index calculation module is used to calculate the percentage of active time, the activity level, and the activity index.

[0053] The present invention provides a storage medium storing instructions, wherein when a computer reads the instructions, the computer executes an online host activity detection method as described in any of the preceding claims.

[0054] An electronic device according to the present invention includes a processor and the above-described storage medium, wherein the processor executes instructions in the storage medium.

[0055] Compared with the prior art, the beneficial effects of the present invention are:

[0056] 1. This invention addresses the trend changes in the comprehensive sampled data sequence and proposes an activity index that can comprehensively quantify the activity level of a host. This facilitates comparative analysis between different hosts, thereby providing a scientific and reasonable basis for judgment on computing power scheduling, load balancing, and energy consumption reduction, and improving the accuracy of corresponding decisions.

[0057] 2. This invention addresses the problems of other weight generation methods by using the sigmoid function to generate non-linear weights between (0,1). This allows data at different sampling times to have different contributions. Furthermore, the closer the weights generated by the sigmoid function are to 0 or 1, the smaller the difference in weight changes, which better reflects the actual meaning of activity changes.

[0058] 3. This invention utilizes N Transformation, transforming the time series to Inside, and integer N The value is between 5 and 8, which makes the time parameter more standardized, thus improving the diversity and balance of the weights generated by the sigmoid function. Attached Figure Description

[0059] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0060] Figure 1 This is a schematic diagram of the online host activity detection method according to Embodiment 1 of the present invention;

[0061] Figure 2 This is a trend graph of the CPU utilization sampling sequence of host 1 in Embodiment 2 of the present invention;

[0062] Figure 3 This is a trend graph of the CPU utilization sampling sequence of host 2 in Embodiment 2 of the present invention;

[0063] Figure 4 This is a block diagram of an online host activity detection system according to Embodiment 3 of the present invention; Detailed Implementation

[0064] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0065] Example 1:

[0066] Existing methods for detecting host activity primarily involve thresholding the average of resource sampling data over a period of time. However, this method ignores the trend changes within the sampling data sequence. Research into existing technologies reveals that sampling data from different time periods offers different assessments of activity levels, thus addressing this deficiency. Therefore, this invention provides an embodiment: an online method for detecting host activity. Please refer to [link to relevant documentation]. Figure 1 The specific steps include:

[0067] S1: Obtain the sampling sequence of monitoring metrics of the target host within a preset time window and the corresponding Unix timestamp sequence;

[0068] S2: Perform outlier detection and replacement on the sampled sequence, and perform [further processing] on the Unix timestamp sequence. Transform to obtain the N-transform sequence;

[0069] S3: Utilizing sampling sequences and preset monitoring indicators Activity threshold The percentage of active time was calculated.

[0070] S4: Utilize preset monitoring indicators Activity threshold The weights and activity amplitudes are calculated from the sampling sequence and the N-transform sequence.

[0071] S5: Calculate the activity index of the target host using the percentage of active time and the activity level;

[0072] S6: Acquire new sampling data Update the sampling sequence and timestamp sequence Repeat steps S2 to S5 to calculate the new activity index of the target host.

[0073] Specifically, the preset time window length is The calculation formula is:

[0074] ,

[0075] in, This indicates the current moment when the activity detection is performed. This indicates the first historical sample taken for activity detection. time;

[0076] The sampling sequence is ,in Indicates the distance to time The most recent sample, i.e., the latest sampled data; the sampling sequence. The corresponding Unix timestamp sequence is ,in Indicates the first A timestamp; a Unix timestamp refers to converting the time corresponding to the original sample into a Unix timestamp.

[0077] Specifically, the outlier detection method is the Isolation Forest algorithm, and its specific process is as follows:

[0078] The sampled sequence is input into the Isolation Forest algorithm. If the output label is 1, it is a normal value; if it is -1, it is an outlier.

[0079] The outlier replacement method is as follows: Assume the sampling sequence middle If the value is an outlier, the replacement value is calculated using the following formula. ,

[0080] ,

[0081] in, Indicates outlier Replacement value, Indicates the first sampled sequence i -1 non-abnormal sampled value, Indicates the first sampled sequence i+ One non-abnormal sampled value;

[0082] If the abnormal value is and Then, the replacement value is calculated using the following formula:

[0083] ,

[0084] ,

[0085] in, and The outlier and Replacement value, , , and These are the non-abnormal sampled values ​​in the sampling sequence.

[0086] Specifically, in S2 The specific conversion process is as follows:

[0087] Using the Unix timestamp sequence The transformed sequence is calculated using the following formula. The formula is as follows:

[0088] ,

[0089] in, Indicates the th element in the transformed sequence One value, Represents the 1st in a Unix timestamp sequence A timestamp, This represents the first timestamp in the Unix timestamp sequence. Represents the 1st in a Unix timestamp sequence A timestamp, express The set of natural numbers under transformation.

[0090] Specifically, the formula for calculating the percentage of active time is as follows:

[0091] ,

[0092] in, Indicates the percentage of active time. Represents the sampling sequence The Middle Each sample value, Indicates preset monitoring indicators The activity threshold Indicates the number of sampled sequences. Specifically:

[0093] ,

[0094] in, express The value of .

[0095] Specifically, the weights in S4 The specific calculation method is as follows:

[0096] ,

[0097] in, Indicates the first Each weight, Represents the transformed sequence The One value, e Represents the natural base;

[0098] The specific calculation method for the activity level is as follows:

[0099] ,

[0100] in, Indicates the level of activity.

[0101] Specifically, the activity index The calculation method is as follows:

[0102] ,

[0103] in, Indicates the activity index, Indicates the percentage of active time. Indicates the level of activity.

[0104] Example 2:

[0105] To better illustrate the online host activity detection method provided in Example 1, this example uses the CPU utilization of host 1 and host 2 as a sampling sequence to calculate the activity index and perform comparative analysis.

[0106] Specifically, the CPU utilization sampling sequence of host 1 is: [73.0395, 65.2886, 69.3409, 54.3349, 28.9465, 24.8818, 31.4, 32.5733, 34.6533, 23.337]; its trend is as follows. Figure 2 As shown, it exhibits a downward trend;

[0107] Specifically, the CPU utilization sampling sequence of host 2 is: [20.7457, 30.9652, 32.934, 25.9477, 40.8978, 38.5191, 56.9229, 66.1957, 56.2311, 55.1978]; its trend is as follows. Figure 3 As shown, it exhibits an upward trend.

[0108] Specifically, the process of outlier detection and replacement in the sampling sequence is as follows:

[0109] First, the CPU utilization sampling sequences of host 1 and host 2 are input into the Isolation Forest algorithm. Python is used to calculate the output label for each sample value. In the CPU utilization sampling sequence of host 1, the label corresponding to 73.0395 is -1, and the labels corresponding to the other 9 sequence values ​​are 1. In the CPU utilization sampling sequence of host 2, the label corresponding to 66.1957 is -1, and the labels corresponding to the other 9 sequence values ​​are 1. Therefore, using the following formula, the replacement value for 73.0395 can be calculated as 67.31475.

[0110] ,

[0111] in, The outlier Replacement value, , These are non-abnormal sampled values ​​in the sampling sequence;

[0112] The replacement value for 66.1957 is calculated to be 56.577 using the following formula.

[0113] ,

[0114] in, Indicates outlier Replacement value, Indicates the first sampled sequence i -1 non-abnormal sampled value, Indicates the first sampled sequence i+ One non-abnormal sample value.

[0115] Specifically, The specific conversion process is as follows:

[0116] The obtained Unix timestamp sequence =[1660266611, 1660266671, 1660266731, 1660266791, 1660266851, 1660266911, 1660266971, 1660267031, 1660267091, 1660267151] Enter into Transform the (-5, 5) sequence using the transformation formula to obtain the transformed sequence. The formula is as follows:

[0117] ,

[0118] in, Indicates the th element in the transformed sequence One value, Represents the 1st in a Unix timestamp sequence A timestamp, This represents the first timestamp in the Unix timestamp sequence. Represents the 1st in a Unix timestamp sequence A timestamp, express The set of natural numbers under transformation.

[0119] Specifically, using a CPU utilization rate of 30% as the threshold, the activity indices of host 1 and host 2 were calculated to be 85.60 and 194.29 respectively using the following formula:

[0120] ,

[0121] in, Indicates the activity index, Indicates the percentage of active time. Indicates the level of activity;

[0122] The calculation method is as follows:

[0123] ,

[0124] in, Indicates the percentage of active time. Represents the sampling sequence The Middle Each sample value, Indicates preset monitoring indicators The activity threshold Indicates the number of sampled sequences. Specifically:

[0125] ,

[0126] in, express The value of .

[0127] The calculation method is as follows:

[0128] ,

[0129] in, Indicates the level of activity. Indicates the first Each weight is calculated as follows:

[0130] ,

[0131] in, Represents the transformed sequence The One value, e Represents the natural base;

[0132] Specifically, acquiring new sampling data Update the sampling sequence and timestamp sequence Repeat the above calculation process to obtain the new activity index of the target host.

[0133] The above calculations show that host 2 is more active than host 1. However, the mean values ​​of host 1 and host 2 are 43.207 and 41.493 respectively, indicating that host 1 is more active than host 2. This shows that using only the mean value for detection will affect the activity of the host due to the trend changes in the sampled data sequence. The method proposed in this invention effectively solves this problem by using sampled data from different time periods, thus improving the accuracy of computer resource allocation decisions.

[0134] Example 3:

[0135] Please see Figure 4 The present invention provides an embodiment of an online host activity detection system, which includes a data acquisition module, an anomaly detection module, a data preprocessing module, and an activity index calculation module.

[0136] The data acquisition module is used to obtain historical sampling data and sampling time of the corresponding monitoring indicators of the target host within a set time interval from the monitoring system, as well as real-time sampling data and sampling time.

[0137] The anomaly detection module is used to detect and replace anomalies in both historical and real-time collected data.

[0138] The data preprocessing module is used to convert the raw time of the sampled data into a Unix timestamp and to process the sampled data. Transformation;

[0139] The activity index calculation module is used to calculate the percentage of active time, the activity level, and the activity index.

[0140] Example 4:

[0141] An embodiment of the present invention provides a storage medium storing instructions, which, when read by a computer, cause the computer to execute any of the above-mentioned online host activity detection methods.

[0142] Example 5:

[0143] An electronic device according to the present invention includes a processor and the above-described storage medium, wherein the processor executes instructions in the storage medium.

[0144] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for online detection of host activity, characterized in that, Includes the following steps: S1: Obtain the sampling sequence of monitoring metrics of the target host within a preset time window and the corresponding Unix timestamp sequence; S2: Perform outlier detection and replacement on the sampled sequence, and perform [further processing] on the Unix timestamp sequence. Convert, obtain N Transform sequence; S3: Utilizing sampling sequences and preset monitoring indicators Activity threshold The percentage of active time was calculated. S4: Utilize preset monitoring indicators Activity threshold , sampling sequence and N Transform the sequence and calculate the weights and activity levels; S5: Calculate the activity index of the target host using the percentage of active time and the activity level; S6: Acquire new sampling data Update the sampling sequence and timestamp sequence Repeat steps S2 to S5 to calculate the new activity index of the target host. In S2 The specific conversion process is as follows: Using the Unix timestamp sequence The transformed sequence is calculated using the following formula. The formula is as follows: ， in, Indicates the th element in the transformed sequence One value, Represents the 1st in a Unix timestamp sequence A timestamp, This represents the first timestamp in the Unix timestamp sequence. Represents the 1st in a Unix timestamp sequence A timestamp, express The set of natural numbers under transformation; The specific formula for calculating the percentage of active time is as follows: ， in, Indicates the percentage of active time. Represents the sampling sequence The Middle Each sample value, Indicates preset monitoring indicators The activity threshold Indicates the number of sampled sequences. Specifically: ， in, express The value; The weights in S4 The specific calculation method is as follows: ， in, Indicates the first Each weight, Represents the transformed sequence The One value, e Represents the natural base; The specific calculation method for the activity level is as follows: ， in, Indicates the level of activity; The activity index The calculation method is as follows: ， in, Indicates the activity index, Indicates the percentage of active time. Indicates the level of activity.

2. The online host activity detection method according to claim 1, characterized in that, The preset time window length in S1 is The calculation formula is: ， in, This indicates the current moment when the activity detection is performed. This indicates the first historical sampling for activity detection. time; The sampling sequence is: ,in Indicates the distance to time The most recent sample, i.e., the latest sampled data; The sampling sequence The corresponding Unix timestamp sequence is ,in Indicates the first A timestamp; The Unix timestamp refers to converting the time corresponding to the original sample into a Unix timestamp.

3. The online host activity detection method according to claim 2, characterized in that, The specific method for anomaly detection is the Isolation Forest algorithm, and its specific process is as follows: The sampling sequence is input into the Isolation Forest algorithm. If the output label is 1, it is a normal value; if it is -1, it is an outlier. The specific steps of the outlier replacement method are as follows: Assuming the sampling sequence middle If the value is an outlier, the replacement value is calculated using the following formula. , ， in, Indicates outlier Replacement value, Indicates the first sampled sequence i -1 non-abnormal sampled value, Indicates the first sampled sequence i+ One non-abnormal sampled value; If the abnormal value is and Then, the replacement value is calculated using the following formula: ， ， in, and The outlier and Replacement value, , , and These are non-abnormal sampled values ​​in the sampling sequence.

4. A host activity online detection system, implemented based on the host activity online detection method as described in any one of claims 1-3, characterized in that, The system includes a data acquisition module, an anomaly detection module, a data preprocessing module, and an activity index calculation module; The data acquisition module is used to acquire historical sampling data and sampling time of the corresponding monitoring indicators of the target host within a set time interval, as well as real-time sampling data and sampling time, from the monitoring system. The anomaly detection module is used to detect and replace anomalies in the collected historical data and real-time data. The data preprocessing module is used to convert the raw time of the sampled data into a Unix timestamp and perform data processing on the sampled data. Transformation; The activity index calculation module is used to calculate the percentage of active time, the activity level, and the activity index.

5. A storage medium, characterized in that, The storage medium stores instructions that, when read by a computer, cause the computer to execute an online host activity detection method as described in any one of claims 1-3.

6. An electronic device, characterized in that, The device includes a processor and the storage medium of claim 5, wherein the processor executes the online host activity detection method of any one of claims 1-3 in the storage medium.

Images