Method for detecting abnormality of resource usage behavior and method for constructing fingerprint model

Abstract

This application discloses anomaly detection methods and fingerprint model construction methods for resource usage behavior, relating to the field of computer technology, particularly to artificial intelligence fields such as deep learning, large-scale models, and cloud computing. The specific implementation scheme is as follows: acquiring resource indicator data for a target time window; wherein, the resource indicator data includes resource indicators at each sampling time point within the target time window acquired during the execution of the target computing task; determining the first similarity between the resource indicator data and the resource usage behavior fingerprint model corresponding to the task type to which the target computing task belongs; wherein, the resource usage behavior fingerprint model is used to characterize the resource usage pattern of the task type; and detecting abnormal resource usage behavior of the target computing task based on the first similarity, and obtaining the detection results.

Description

Technical Field

[0001] This application relates to the field of computer technology, particularly to the fields of artificial intelligence such as deep learning, large models, and cloud computing, and specifically to a method for detecting abnormal resource usage behavior and a method for constructing a fingerprint model of resource usage behavior. Background Technology

[0002] In real-world computing environments, various computing tasks (such as data analysis, model training, and service deployment) continuously consume system resources such as CPU (Central Processing Unit), memory, GPU (Graphics Processing Unit), and disk I / O (Input / Output). However, some computing tasks may consume system resources without authorization or beyond reasonable limits, thereby interfering with the normal execution of other tasks and reducing overall computing efficiency. Summary of the Invention

[0003] This application provides a method for detecting abnormal resource usage behavior and a method for constructing a resource usage behavior fingerprint model. The specific solution is as follows:

[0004] According to one aspect of this application, a method for detecting abnormal resource usage behavior is provided, comprising: Acquire resource indicator data for the target time window; wherein, the resource indicator data includes resource indicators at each sampling time point within the target time window acquired during the execution of the target computing task; Determine the first similarity between the resource indicator data and the resource usage behavior fingerprint model corresponding to the task type to which the target computing task belongs; wherein, the resource usage behavior fingerprint model is used to characterize the resource usage pattern of the task type; Based on the first similarity score, abnormal resource usage behavior of the target computing task is detected, and the detection results are obtained.

[0005] According to another aspect of this application, a method for constructing a resource usage behavior fingerprint model is provided, comprising: Acquire resource indicator data for multiple time windows; the resource indicator data includes resource indicators at each sampling time point within the corresponding time window acquired during the historical computing task execution process. Statistical features are extracted from the same resource feature dimension in the resource indicator data to obtain the statistical feature vector of the corresponding time window; Clusters are obtained by clustering statistical feature vectors from multiple time windows, and behavioral fingerprint templates of the clusters are extracted; wherein, a behavioral fingerprint template is used to characterize a resource usage pattern. Based on the behavioral fingerprint templates of each cluster, the resource usage behavior fingerprint model corresponding to the target task type to which the historical computing tasks belong is determined; wherein, the resource usage behavior fingerprint model is used to characterize the resource usage pattern of the target task type.

[0006] According to another aspect of this application, a resource usage behavior anomaly detection device is provided, comprising: The first acquisition module is used to acquire resource indicator data for the target time window; wherein, the resource indicator data includes resource indicators at each sampling time point within the target time window acquired during the execution of the target computing task. The first determining module is used to determine the first similarity between the resource indicator data and the resource usage behavior fingerprint model corresponding to the task type to which the target computing task belongs; wherein, the resource usage behavior fingerprint model is used to characterize the resource usage pattern of the task type; The detection module is used to detect abnormal resource usage behavior of the target computing task based on the first similarity and obtain the detection results.

[0007] According to another aspect of this application, an apparatus for constructing a resource usage behavior fingerprint model is provided, comprising: The first acquisition module is used to acquire resource indicator data for multiple time windows; wherein, the resource indicator data includes resource indicators at each sampling time point within the corresponding time window acquired during the historical computing task execution process; The first extraction module is used to extract statistical features from the same resource feature dimension in the resource indicator data to obtain the statistical feature vector of the corresponding time window. The second extraction module is used to obtain clusters obtained by clustering statistical feature vectors of multiple time windows, and to extract behavioral fingerprint templates of the clusters; wherein, a behavioral fingerprint template is used to characterize a resource usage pattern. The first determining module is used to determine the resource usage behavior fingerprint model corresponding to the target task type to which the historical computing tasks belong, based on the behavior fingerprint template of each cluster; wherein, the resource usage behavior fingerprint model is used to characterize the resource usage pattern of the target task type.

[0008] According to another aspect of this application, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described in the above embodiments.

[0009] According to another aspect of this application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to perform the method described in the above embodiments.

[0010] According to another aspect of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described in the above embodiments.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of this application. Wherein: Figure 1 A flowchart illustrating a method for detecting abnormal resource usage behavior according to an embodiment of this application; Figure 2 A flowchart illustrating a method for detecting abnormal resource usage behavior according to another embodiment of this application; Figure 3 A flowchart illustrating a method for detecting abnormal resource usage behavior according to another embodiment of this application; Figure 4 A flowchart illustrating a method for detecting abnormal resource usage behavior according to an embodiment of this application; Figure 5 A flowchart illustrating a method for constructing a resource usage behavior fingerprint model according to an embodiment of this application; Figure 6 A flowchart illustrating a method for constructing a resource usage behavior fingerprint model according to an embodiment of this application; Figure 7 A schematic diagram illustrating the calling process between the modules of the resource usage behavior anomaly detection system provided in this application embodiment; Figure 8 This is a schematic diagram of the structure of a resource usage behavior anomaly detection device provided in an embodiment of this application; Figure 9 A schematic diagram of the structure of a resource usage behavior fingerprint model construction device provided in an embodiment of this application; Figure 10 This is a block diagram of an electronic device used to implement the resource usage behavior anomaly detection method of the embodiments of this application. Detailed Implementation

[0013] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0014] It should be noted that the acquisition, storage, use, and processing of data in this application comply with the relevant provisions of national laws and regulations and do not violate public order and good morals.

[0015] The following description, with reference to the accompanying drawings, describes a method for detecting abnormal resource usage behavior, a method for constructing a resource usage behavior fingerprint model, an apparatus, an electronic device, and a storage medium according to embodiments of this application.

[0016] In some embodiments, resource security detection methods typically rely on process IDs, container identifiers, or static permission configurations to detect abnormal resource usage behavior. However, in some scenarios (such as multi-tenant cluster environments), it is difficult to identify implicit resource occupation behavior under legitimate scheduling. For example, a user task may secretly occupy CPU, memory, bandwidth, or I / O resources on a shared node, thereby affecting the main computing task and causing a decrease in overall computing efficiency.

[0017] Based on this, this application proposes a method for detecting abnormal resource usage behavior. Figure 1 This is a flowchart illustrating a method for detecting abnormal resource usage behavior according to an embodiment of this application.

[0018] The resource usage behavior anomaly detection method of this application embodiment can be executed by the resource usage behavior anomaly detection device of this application embodiment, which can be configured in an electronic device.

[0019] Among them, electronic devices can be any device with computing capabilities, such as personal computers, mobile terminals, servers, etc. Mobile terminals can be hardware devices with various operating systems, touch screens and / or displays, such as in-vehicle devices, mobile phones, tablets, personal digital assistants, wearable devices, etc.

[0020] like Figure 1 As shown, the method for detecting abnormal resource usage behavior includes: Step 101: Obtain resource indicator data for the target time window.

[0021] In this application, resource index data may include resource indexes at each sampling time point within the target time window obtained during the execution of the target computing task.

[0022] For example, the target time window can be any time window during the execution of the target computing task, which can be understood as the current time window. For example, the target computing task can be a model training task, a data processing task, a graph computing task, or other high-performance computing tasks, etc. The target computing task can be a distributed computing task or a single-node computing task, without limitation.

[0023] For example, the target computation task can be a large model training task (such as a large model pre-training task, a large model fine-tuning task, etc.), a large model inference task, a large model validation task, a deep learning model training task (such as a deep learning model pre-training task, a deep learning model fine-tuning task, etc.), a deep learning model inference task, or a deep learning model validation task, etc.

[0024] For example, the resource metrics at each sampling time point may include, but are not limited to, metrics for computing resources, storage resources, I / O resources, and communication resources. For example, each resource metric can be considered a resource feature dimension, allowing for the acquisition of multi-dimensional resource metrics at each sampling time point.

[0025] For example, metrics for computing resources can include GPU metrics (such as GPU utilization, video memory usage, video memory bandwidth, etc.), CPU metrics (such as CPU utilization, number of threads, context switching rate, etc.), storage resource metrics can include memory metrics (such as memory utilization, swap utilization, etc.), I / O resource metrics can include disk read / write speed, I / O latency, etc., and communication resource metrics can include network bandwidth, etc.

[0026] For example, the resource indicator data for a time window can be in the form of an array, a matrix, or other mathematical expression, and this application does not limit it in this way.

[0027] For example, resource metric streams can be received in real time through the cluster monitoring interface, at each sampling time point. The feature vector can be represented as ,in, Indicates the quantity of resource indicators. Indicates the first The value of a resource indicator at sampling time point t. Data is received at a fixed sampling period (e.g., 1 second) to form a real-time feature sequence. ,in, This indicates the number of sampling time points.

[0028] It should be noted that the resource indicator data for each time window can be the raw data or the data after standardization and normalization, and this application does not limit it.

[0029] Step 102: Determine the first similarity between the resource indicator data and the resource usage behavior fingerprint model corresponding to the task type to which the target computing task belongs.

[0030] In this application, the resource usage behavior fingerprint model can be used to characterize the resource usage pattern of the task type to which the target computing task belongs. The resource usage pattern can be understood as the resource usage pattern of the computing task of that task type during operation. For example, the number of resource usage patterns for a task type can be one or more, and there is no limitation on this.

[0031] For example, the first similarity score can be used to characterize the degree of similarity between resource indicator data and resource usage behavior fingerprint model within a target time window. The higher the first similarity score, the closer the resource usage within the target time window is to normal.

[0032] For example, the resource usage behavior fingerprint model can be pre-constructed based on resource indicator data obtained during the operation of historical computing tasks of the same type as the target computing task. For specific construction methods, please refer to the following embodiments.

[0033] Step 103: Based on the first similarity, detect abnormal resource usage behavior of the target computing task and obtain the detection results.

[0034] In this application, the resource usage behavior within the target time window can be determined based on the relationship between the first similarity and the first threshold corresponding to the task type to which the target computing task belongs, thereby obtaining the judgment result of the target time window, and the detection result of the target computing task can be determined based on the judgment result of the target time window.

[0035] For example, if the first similarity is less than the first threshold, it can be determined that the resource usage behavior within the target time window is abnormal; if the first similarity is greater than or equal to the first threshold, it can be determined that the resource usage behavior within the target time window is normal.

[0036] For example, determining the detection result of the target computing task based on the judgment result of the target time window may include: determining whether the target computing task has abnormal resource usage behavior based on the judgment result of the target time window and the judgment result of the time window adjacent to the target time window, and then determining the detection result of the target computing task based on the judgment result of the target computing task.

[0037] For example, if the judgment results of the target time window and the two adjacent time windows before it are both abnormal resource usage behavior, it can be determined that the resource usage behavior of the target computing task is abnormal.

[0038] For example, the detection results may include, but are not limited to, the first similarity, the degree of deviation, the judgment result of the target time window, and the judgment result of the target computation task. Among them, the degree of deviation is equal to the difference between 1 and the first similarity.

[0039] For example, the first threshold corresponding to different task types can be the same or different, and can be set according to actual needs. This application does not limit this.

[0040] The resource usage behavior anomaly detection method of this application embodiment can be applied to computing task scenarios with relatively stable resource usage patterns, such as model training scenarios, and can solve the problem of implicit occupation of cluster resources and impaired task stability during model training.

[0041] In this embodiment, resource indicator data during the execution of computing tasks is collected within a target time window and compared with a resource usage behavior fingerprint model pre-built for that task type. This quantifies the consistency between the current running behavior and the expected normal behavior, enabling abnormal detection of resource usage behavior in the target computing task. This allows for the identification of abnormal behavior under legitimate scheduling, significantly improving the sensitivity and accuracy of resource usage behavior detection. Furthermore, because the model is customized based on task type, it takes into account the differences in resource usage behavior under different business loads, achieving adaptive detection of resource usage behavior for computing tasks of different task types and improving the accuracy of detection.

[0042] Figure 2 This is a flowchart illustrating a method for detecting abnormal resource usage behavior, provided in another embodiment of this application.

[0043] like Figure 2 As shown, the method for detecting abnormal resource usage behavior includes: Step 201: Obtain resource indicator data for the target time window.

[0044] In this application, step 201 can be implemented in any of the embodiments of this application, so it will not be described again here.

[0045] Step 202: Extract statistical features from the same resource feature dimension in the resource indicator data to obtain the statistical feature vector of the target time window.

[0046] In this application, statistical features can be extracted from the same resource feature dimension at each time point within the target time window to obtain the statistical features of each resource feature dimension. Based on the statistical features of each resource feature dimension, the statistical feature vector of the target time window is obtained.

[0047] For example, statistical feature extraction may include, but is not limited to, calculating the mean, variance, kurtosis, and periodicity. Kurtosis can be used to measure the sharpness of the resource feature distribution, and periodicity can be calculated using the autocorrelation function. Therefore, the statistical features of a resource feature dimension can include the mean, variance, kurtosis, and periodicity of that resource feature dimension.

[0048] For example, the first Peak value of each resource characteristic dimension It can be calculated using the following formula (1): (1) in, This indicates the number of sampling time points within the target time window. Indicates the first Each resource feature dimension at the sampling time point The value, Indicates the first The mean of each resource characteristic dimension, Indicates the first The variance of each resource feature dimension.

[0049] For example, the first The periodicity of each resource feature dimension can be calculated using the following formula (2): (2) in, Indicates a time lag.

[0050] Since the units of measurement of different resource feature dimensions may be different, in some embodiments, if the resource indicator data is raw data, that is, the resource indicator data is the original value of the resource indicator, the resource indicator data can be normalized and standardized first, and then statistical feature extraction can be performed to obtain the statistical feature vector of the target time window.

[0051] For example, the resource feature dimensions can be normalized based on the minimum and maximum values ​​corresponding to each resource feature dimension in the resource indicator data to obtain normalized indicator data. Then, the normalized indicator data can be standardized based on the mean and variance corresponding to the resource feature dimensions to obtain standardized indicator data. Finally, statistical feature extraction can be performed on the standardized indicator data to obtain the statistical feature vector of the target time window.

[0052] For example, the minimum, maximum, mean, and variance used for normalization and standardization are statistically obtained during the execution of historical computing tasks of the same type as the target computing task.

[0053] For example, the representation of normalized and standardized indicator data can be matrices. In one matrix, each row represents the data for each resource feature dimension at the corresponding sampling time point, and each column represents the value of a resource feature dimension at each sampling time point. For instance, normalized indicator data can be called a normalized feature matrix, and standardized indicator data can be called a standardized feature matrix.

[0054] Step 203: Determine the second similarity between the statistical feature vector and the resource usage behavior fingerprint model.

[0055] In this application, the resource usage behavior fingerprint model can be constructed based on the statistical extraction of resource indicator data obtained from each time window during the execution of historical computing tasks, and the statistical feature vectors of each time window obtained from the statistical feature extraction. The historical computing tasks and the target computing task are computing tasks of the same type.

[0056] Since different task types have different resource usage patterns, such as different training stages of a large model (e.g., pre-training, fine-tuning), in some embodiments, the task type of the target computation task can be identified first, and then the resource usage behavior fingerprint model corresponding to that task type can be obtained.

[0057] For example, the task type label of the target computing task can be obtained from the task scheduling system, and the task type of the target computing task can be determined based on the label.

[0058] For example, the average similarity between the resource indicator data of each time window and the resource usage behavior fingerprint model corresponding to each task type can be calculated, and the task type with the highest average similarity can be used as the task type of the target time window.

[0059] Here, the average similarity corresponding to the task type can be the average of the similarities between the resource indicator data and each behavior fingerprint template in the resource usage behavior fingerprint model corresponding to that task type.

[0060] Taking the model training scenario as an example, the average similarity between the resource index data of the target time window and the resource usage behavior fingerprint model corresponding to each training stage is calculated, and the training stage with the highest average similarity is taken as the training stage of the target computing task.

[0061] In some embodiments, a resource usage behavior fingerprint model may include multiple behavior fingerprint templates. A behavior fingerprint template may be used to characterize a resource usage pattern of a task type. A behavior fingerprint template may include a resource behavior template, which includes a mean vector. The mean vector is used to describe the resource usage pattern and may include the mean of the same statistical feature of each resource feature dimension.

[0062] For example, the similarity between the statistical feature vector and the mean vector in the behavioral fingerprint template can be determined, referred to as behavioral similarity, and the highest behavioral similarity among multiple behavioral fingerprint templates is determined as the second similarity. It can be understood that the behavioral similarity between the statistical feature vector and the mean vector in the behavioral fingerprint template can also be described as the behavioral similarity between the resource indicator data of the target time window and the behavioral fingerprint template.

[0063] For example, the cosine similarity between the statistical feature vector and the mean vector in the behavioral fingerprint template can be calculated, and the cosine similarity can be used as the behavioral similarity. For example, for stable resource indicator data, the cosine similarity can be calculated, and the closer the cosine similarity is to 1, the more similar the data is.

[0064] For example, behavioral similarity between the statistical feature vector and the mean vector in the behavioral fingerprint template can also be calculated based on DTW (Dynamic Time Warping), for example, using formula (3): (3) in, Representing resource indicator data Statistical feature vectors and behavioral fingerprint templates mean vector in The dynamic time-normalized distance between them; This represents the smoothing coefficient, used to control the rate of similarity decay.

[0065] For example, for resource indication data with periodic fluctuations, behavioral similarity can be calculated based on DTW.

[0066] Therefore, if the resource usage behavior fingerprint model includes multiple behavior fingerprint templates, the behavior similarity between the statistical feature vector and the mean vector in each behavior fingerprint template can be determined. By measuring the similarity between resource indicator data and behavior fingerprint templates through behavior similarity, the accuracy of detecting abnormal resource usage behavior can be improved.

[0067] Step 204: Determine the first similarity based on the second similarity.

[0068] In this application, the second similarity can be used as the first similarity.

[0069] Step 205: Based on the first similarity, detect abnormal resource usage behavior of the target computing task and obtain the detection results.

[0070] In this application, step 205 can be implemented in any of the embodiments of this application, so it will not be described in detail here.

[0071] In this embodiment, statistical features are extracted from the same resource feature dimension of resource indicator data within the target time window. The original time-series data is compressed and abstracted into a highly discriminative statistical feature vector. Then, a second similarity is calculated based on the vector and the resource usage behavior fingerprint model corresponding to the task type. Finally, the first similarity used for anomaly judgment is determined. The original time-series data, which is high-dimensional, redundant, and susceptible to transient disturbances, is transformed into a low-dimensional, stable statistical representation rich in distribution characteristics. This not only significantly improves the accuracy and robustness of similarity comparison, but also effectively distinguishes between normal fluctuations and real anomalies, while reducing computational complexity. This provides reliable support for real-time and efficient detection of resource usage behavior anomalies.

[0072] To avoid false alarms due to short-term fluctuations, in some embodiments of this application, the following method can also be used to calculate the first similarity between the resource indicator data of the target time window and the resource usage behavior fingerprint model: the feature vectors of each sampling time point within the target time window can be calculated with the behavior fingerprint template, and the similarity of each sampling time point can be averaged to obtain the overall similarity of the target time window. This overall similarity is used as the behavior similarity between the resource indicator data of the target time window and the behavior fingerprint template, and the largest behavior similarity among multiple behavior fingerprint templates is used as the first similarity.

[0073] For example, the overall similarity can be calculated using the following formula (4): (4) in, It represents the overall similarity, and can also be called "moving average similarity" or "smoothed average similarity"; This indicates the length of the target time window, which is also the number of sampling time points within the target time window; Indicates the sampling time point The similarity between the feature vector and the mean vector of the behavioral fingerprint template.

[0074] Therefore, the similarity between the resource indicator data and the behavioral fingerprint template within a time window can be determined based on the average similarity between the resource indicator data and the behavioral fingerprint template at each sampling time point within the time window. By smoothing and correcting the overall similarity of the time window, the accuracy of similarity calculation can be improved, and false alarms due to short-term fluctuations can be avoided.

[0075] Figure 3 This is a flowchart illustrating a method for detecting abnormal resource usage behavior, provided in another embodiment of this application.

[0076] like Figure 3 As shown, the method for detecting abnormal resource usage behavior includes: Step 301: Obtain resource indicator data for the target time window.

[0077] Step 302: Determine the first similarity between the resource indicator data and the resource usage behavior fingerprint model corresponding to the task type to which the target computing task belongs.

[0078] In this application, steps 301-302 can be implemented in any of the embodiments of this application, so they will not be described in detail here.

[0079] Step 303: Determine the detection result based on at least one of the following: the magnitude relationship between the first similarity and the first threshold corresponding to the task type, the mean of the first similarity and the third similarity, the change in key indicators when the target computing task has unauthorized resource occupation behavior, the contribution of key indicators to the first similarity when the target computing task has unauthorized resource occupation behavior, and the correlation of similarity corresponding to adjacent time windows when the target computing task has unauthorized resource occupation behavior.

[0080] In some embodiments of this application, if the first similarity is greater than or equal to the first threshold corresponding to the task type, the resource usage behavior within the target time window can be considered normal, and a detection result indicating that there is no abnormal resource usage behavior within the target time window can be generated. Here, "no abnormal resource usage behavior within the target time window" can be understood as the target computing task having normal resource usage behavior within the target time window.

[0081] For example, if the first similarity is greater than or equal to the first threshold, a category labeling result can be generated for the target time window. This category labeling result is used to indicate that the resource usage behavior within the target time window is normal. Based on this category labeling result, the detection result of the target computing task is determined.

[0082] In some embodiments of this application, if the first similarity is less than a first threshold corresponding to the task type, it indicates that a category labeling result is generated for the target time window. This category labeling result is used to indicate that there is abnormal resource usage behavior in the target time window. Based on the first similarity, the abnormality level of the abnormal resource usage behavior in the target time window is determined, and a detection result is generated based on the abnormality level. For example, the detection result may include, but is not limited to, the abnormality level corresponding to the target time window.

[0083] Since resource usage behavior may fluctuate, in order to improve the accuracy of detection results, for example, the abnormality level of abnormal resource usage behavior in the target time window can be determined based on the average of the first similarity and the third similarity.

[0084] The third similarity can be the similarity between time windows adjacent to the target time window. The third similarity is used to characterize the degree of similarity between the resource indicator data of adjacent time windows and the resource usage behavior fingerprint model. In other words, the third similarity is the similarity between the resource indicator data of adjacent time windows and the resource usage behavior fingerprint model.

[0085] For example, the anomaly level of the target time window can be determined based on the average of the first and third similarities, combined with the threshold range corresponding to different anomaly levels.

[0086] For example, setting three thresholds. It is a mild abnormality threshold. It is the threshold for moderate abnormality. It is the threshold for severe anomalies, where, The anomaly level can be determined using the following formula (5): (5) Among them, the abnormal level Indicates mild abnormality, abnormality level This indicates a moderate abnormality, with an abnormality level of [missing information]. Indicates a serious abnormality; It can be the average of the first and third similarities, or it can be the first similarity of the target time window; there are no restrictions on this.

[0087] In other words, if the first similarity of the target time window is greater than or equal to If the resource usage behavior within the target time window is considered normal, then the first similarity of the target time window is less than [a certain value]. If the resource usage behavior within the target time window is considered abnormal, then the level of abnormality can be determined based on the average of the first and third similarities.

[0088] For example, the number of time windows adjacent to the target time window can be one or more, and there is no limitation on this.

[0089] For example, suppose , , The similarity of three consecutive time windows, including the target time window, was 0.72, 0.70, and 0.68, respectively. The similarity of the target time window was 0.68. The mean of these three similarities was 0.7, which is less than 0.75 and greater than 0.6. Therefore, the anomaly level was determined to be moderate.

[0090] Therefore, by determining the anomaly level by the mean similarity between resource indicator data based on the target time window and its adjacent time windows and the resource usage behavior fingerprint model, the problem of inaccurate judgment caused by instantaneous anomalies in resource usage behavior can be avoided, thus improving the accuracy of anomaly level determination.

[0091] In some embodiments, if the category labeling result of the target time window is abnormal, it can be determined whether the target computing task has unauthorized resource occupation behavior. If so, the occupation type to which the unauthorized resource occupation behavior belongs is determined, and then a detection result indicating abnormal resource usage behavior in the target time window is generated based on the task identifier of the target computing task, the task type to which the target time task belongs, the category labeling result, the occupation type, etc.

[0092] For example, the type of occupancy can include explicit occupancy and implicit occupancy. For example, explicit occupancy is manifested by a sudden increase or decrease in resource indicators, such as GPU memory being occupied by external processes or an abnormal increase in CPU load; implicit occupancy is manifested by a disruption of resource usage patterns, but the overall load does not change significantly, such as an internal thread abnormally occupying GPU memory.

[0093] For example, the occupancy type can be determined in the following way: the occupancy type can be determined based on at least one of the following: the change in key indicators when the target computing task has unauthorized resource occupancy behavior, the contribution of key indicators to the first similarity when the target computing task has unauthorized resource occupancy behavior, and the correlation of the similarity corresponding to adjacent time windows when the target computing task has unauthorized resource occupancy behavior.

[0094] For example, key indicators are resource indicators associated with unauthorized resource usage behavior. Key indicators can be understood as the main characteristics that cause abnormal resource usage behavior of the target computing task. Key indicators can be determined based on the contribution of each resource indicator to the first similarity. For example, the resource indicator with the largest contribution can be used as the key indicator.

[0095] For example, the contribution of resource indicators can be calculated using the Shapley value according to the following formula (6): (6) in, Indicators of resources Contribution to the first similarity; Represents the set of all resource metrics; This represents a subset of resource indicators, which is a set. Excluding resource indicators A subset outside of; Representing a subset Contribution to the first similarity; Representing a subset The size of the subset. The number of resource indicators included; Representing a subset The factorial of the size; Representing a subset The size of the subset. The number of resource indicators included; Representing a subset The factorial of the size; In other words, a subset represents all possible subsets. The proportion; Representing a subset Add resource indicators Changes in contributions afterward.

[0096] For example, the change in a key indicator can be used to characterize how much the current key indicator deviates from its historical average.

[0097] For example, the correlation of similarity between adjacent time windows can measure the stability of resource usage in adjacent time windows. The similarity between adjacent time windows can be the similarity between resource indicator data of each time window in adjacent time windows and the resource usage behavior fingerprint model.

[0098] As an example, if the absolute value of the change in a key indicator is greater than the third threshold, and the contribution of the key indicator is greater than the fourth threshold, the occupancy type can be determined as a visible intrusion.

[0099] For example, the change in key indicators can be calculated using the following formula (7): (7) in, Key indicators The change in; Key indicators At the sampling time point The value; Key indicators The mean value over the target computational task; Key indicators Variance on the target computation task.

[0100] if The absolute value is greater than the third threshold, key indicator Feature contribution If the value exceeds the fourth threshold, the occupancy type is determined to be a visible intrusion.

[0101] As an example, the correlation can be calculated using the following formula (8): (8) in, Indicates time window The similarity between resource indicator data and resource usage behavior fingerprint models. Indicates time window Previous time window The similarity between resource indicator data and resource usage behavior fingerprint models. This represents the correlation function.

[0102] if If the resource index is less than the fifth threshold and the fluctuation of the resource index is concentrated on a single resource index, then the occupancy type can be determined as implicit occupancy.

[0103] In some embodiments, the following method can be used to determine that the target computing task has unauthorized resource occupation behavior: the feature contribution variance can be determined based on the feature contribution vector, and an anomaly score can be determined based on the first similarity, the feature contribution variance and the similarity change rate. If the anomaly score is greater than a second threshold, the target computing task has unauthorized resource occupation behavior based on at least one of the resource usage behavior category, feature contribution vector and deviation degree in multiple consecutive time windows.

[0104] For example, the feature contribution vector can include the contribution of the resource indicators to the first similarity under each resource feature dimension. Specifically, the contribution of each resource indicator to the first similarity can be calculated using the above contribution calculation formula.

[0105] For example, the average contribution can be calculated based on the feature contribution of each resource indicator in the feature contribution vector, and the variance of the feature contribution can be calculated based on the feature contribution of each resource indicator and the average contribution.

[0106] For example, the similarity change rate can be used to characterize the trend of the similarity between resource indicator data and resource usage behavior fingerprint model over time for multiple consecutive time windows, and the anomaly score can be used to characterize the degree of anomaly in abnormal resource usage behavior in a target time window.

[0107] As an example, the anomaly score can be calculated using the following formula (9): (9) in, Indicates abnormal rating; Indicates the target time window The corresponding first similarity; Indicates the degree of deviation; Indicates the rate of change in similarity; Indicates the variance of the feature contribution; , and Represents the weighting coefficient, where .

[0108] For example, if the resource usage behavior category is abnormal for multiple consecutive time windows, it can be determined that the target computing task has engaged in unauthorized resource usage. Here, "multiple consecutive time windows" can refer to multiple consecutive time windows including the target time window, such as the target time window and time windows preceding and adjacent to it.

[0109] For example, if the resource usage behavior category is abnormal for multiple consecutive time windows and there is a resource indicator in the feature contribution vector with a contribution greater than the fourth threshold, it can be determined that the target computing task has unauthorized resource occupation behavior.

[0110] For example, if the resource usage behavior category is abnormal in multiple consecutive time windows, there are resource indicators in the feature contribution vector with a contribution greater than the fourth threshold, and the absolute value of the change in the resource indicator is greater than the third threshold, it can be determined that the target computing task has unauthorized resource occupation behavior.

[0111] For example, if the resource usage behavior category is abnormal for multiple consecutive time windows and the correlation of the similarity between adjacent time windows is less than the fifth threshold, it can be determined that the target computing task has unauthorized resource usage behavior.

[0112] Therefore, by determining the anomaly score based on the first similarity, the variance of feature contribution, and the similarity change rate, the anomaly score can not only reflect the intensity of the anomaly, but also characterize its structural and temporal evolution characteristics. When the anomaly score exceeds the second threshold, by further combining the stability of resource use behavior categories and feature contribution patterns in continuous time windows, it is possible to effectively distinguish between occasional fluctuations and persistent unauthorized behavior, thereby significantly reducing the false alarm rate.

[0113] In some embodiments, if the anomaly level of the target time window reaches a preset level, it can be determined that the target computing task has engaged in unauthorized resource occupation.

[0114] In other words, when the anomaly level of the target time window reaches the preset level (such as moderate anomaly or severe anomaly) or the anomaly score exceeds the second threshold, the occupancy type identification process can be triggered to identify the occupancy type to which the unauthorized resource occupancy behavior belongs.

[0115] In some embodiments, if the category labeling result of the target time window is abnormal, the abnormality level corresponding to the target time window can be determined, and it can be determined whether the target computing task has unauthorized resource occupation behavior. If it does, the occupation type to which the unauthorized resource occupation behavior belongs can be determined, and then the detection result of the target computing task can be generated according to the abnormality level and occupation type.

[0116] Therefore, when the similarity between the resource indicator data of the target time window and the resource usage behavior fingerprint model is not lower than the first threshold, it is judged as no anomaly; when the similarity is lower than the first threshold, the anomaly level is further determined by combining the average resource usage and the judgment criteria such as the change of key indicators, feature contribution, and cross-window correlation, to accurately distinguish the specific types of unauthorized use. This not only improves the accuracy and robustness of anomaly detection, but also realizes interpretable diagnosis from "whether it is abnormal" to "what kind of anomaly, how serious, and why it is abnormal", providing high-precision, fine-grained identification and classification of abnormal resource usage behavior, and providing a structured decision basis for subsequent alarm classification.

[0117] In some embodiments, the first threshold can be determined as follows: the first average value and standard deviation of the similarity between resource indicator data and resource usage behavior fingerprint model in multiple time windows during the operation of historical computing tasks can be determined, and the first threshold can be determined based on the first average value and standard deviation.

[0118] For example, the average similarity between resource indicator data and resource usage behavior fingerprint model over multiple time windows during the execution of historical computing tasks can be calculated, which is also known as the first average or the first average similarity.

[0119] For example, the first threshold can be calculated using the following formula (10). : (10) in, This represents the average similarity of multiple time windows during the execution of a historical computing task. The standard deviation representing the similarity of multiple time windows during the execution of a historical computing task; This represents an empirical coefficient, and its value range could be (1.5, 2).

[0120] In this embodiment, the average and standard deviation of historical computing tasks of the same task type are dynamically calculated based on their similarity to resource usage behavior fingerprint models across multiple time windows. A first threshold for this task type is then adaptively set accordingly, thereby achieving task-awareness and data-driven optimization of the anomaly detection benchmark. Since different task types exhibit fundamental differences in resource usage patterns, fluctuation amplitudes, and temporal sequences, a globally uniform threshold is insufficient to account for the normal behavioral characteristics of various tasks. This embodiment utilizes real-world operational data from similar historical computing tasks to construct a statistical model, enabling the first threshold to accurately reflect the central trend and reasonable fluctuation boundaries of normal behavior under this task type. This improves the accuracy of the first threshold and consequently enhances the accuracy of anomaly detection within the time window.

[0121] To improve the accuracy of real-time detection of abnormal resource behavior, in some embodiments, the first threshold can be dynamically adjusted based on the resource usage behavior in the target computing task.

[0122] For example, a second average value of the similarity between resource indicator data and resource usage behavior fingerprint model in multiple time windows during the operation of the target computing task can be determined, and a first threshold can be adjusted based on the second average value to obtain an adjusted first threshold.

[0123] For example, the second average value could be the average similarity between the resource indicator data of each time window and the resource usage behavior fingerprint model during the execution of the target computing task.

[0124] As an example, the first threshold can be adjusted using the following formula (11): (11) in, This represents the adjusted first threshold. Indicates the first threshold; This represents the smoothing coefficient, and its value range can be (0.8, 0.9). This represents the average similarity over a recent period, such as the second average.

[0125] Therefore, the first threshold can be dynamically adjusted based on the similarity of multiple time windows during the execution of the target calculation task, thereby improving the accuracy of subsequent abnormal behavior detection.

[0126] In this embodiment, by integrating multi-dimensional criteria to comprehensively determine the detection results, it not only determines whether the first similarity of the current time window is lower than the first threshold corresponding to the task type, but also introduces the average similarity of adjacent windows, temporal context information such as time correlation, and the abnormal sensitivity of key resource indicators such as the change of specific indicators under the unauthorized behavior and their contribution to the overall similarity. This constructs a dynamic and three-dimensional anomaly decision-making mechanism. This multi-factor joint judgment method can effectively overcome the defects of single threshold detection being susceptible to noise interference and difficult to identify hidden or gradual unauthorized behaviors. Through the consistency analysis of the similarity of adjacent windows, it can filter out occasional fluctuations and enhance detection stability. It can also focus on the highly sensitive key indicators and their pattern contributions under the unauthorized scenario, which can significantly improve the identification accuracy of malicious resource occupation. It can achieve high accuracy and low false alarm rate detection of abnormal use of computing task resources and unauthorized occupation behavior.

[0127] In some embodiments of this application, the resource usage behavior fingerprint model may include a behavior fingerprint template. A behavior fingerprint template may include a resource behavior template and a semantic behavior description, whereby the semantic behavior description describes the semantics of the behavior fingerprint template. For example, the semantic behavior description may be determined based on the semantic feature vectors of historical computing tasks of the same task type as the target computing task. For instance, the semantic behavior description may include task type, task framework, algorithm type, etc.

[0128] In some embodiments, abnormal resource usage behavior of the target computing task can be detected based on the first similarity and combined with the fourth similarity between the semantic feature vector of the target computing task and the semantic behavior description, so as to obtain the detection result.

[0129] For example, semantic feature vectors can include the task type, computation stage, algorithm type, distributed topology, batch processing parameters, etc., of the target computation task. For instance, for a model training task, the computation stage can include forward propagation, backpropagation, gradient synchronization, etc., the algorithm type can be Transformer, ResNet (Residual Network), GNN (Graph Neural Network), etc., and the distributed topology can include the number of nodes (i.e., the total number of nodes participating in the task), the number of worker nodes (i.e., the number of worker nodes actually executing the computation task), the number of parameter servers, etc., where a parameter server is an architectural component used to manage the parameters of a machine learning model.

[0130] Taking the target computation task as the model pre-training task as an example, the training parameters of the model pre-training task (such as the number of samples for each training, learning rate, gradient synchronization interval, total number of model parameters, optimizer type, etc.) can constitute the semantic feature vector of the model pre-training task.

[0131] For example, the semantic feature vector of the target computing task can be obtained, and the fourth similarity between the semantic feature vector of the target computing task and the semantic behavior description can be determined. Based on the first similarity and the fourth similarity, the target similarity can be determined. Then, based on the target similarity, abnormal resource usage behavior of the target computing task can be detected to obtain the detection result.

[0132] For example, the first similarity and the fourth similarity can be weighted to obtain the target similarity.

[0133] For example, the "first similarity" in the above embodiment can be replaced with the target similarity. Therefore, the specific process of detecting abnormal resource usage behavior of the target computing task based on the target similarity can be found in the above embodiment, which describes the specific process of detecting abnormal resource usage behavior of the target computing task based on the first similarity. So it will not be repeated here.

[0134] In some embodiments, "semantic behavior description" may also be described as "semantic behavior template" or "semantic behavior label," etc. This application does not limit the name of "semantic behavior description."

[0135] In this embodiment, if the behavioral fingerprint template includes a semantic behavioral description, the semantic feature vector of the target computing task can be obtained, and the similarity between the semantic feature vector and the semantic behavioral description can be determined. Based on the similarity and the first similarity, abnormal resource usage behavior of the target computing task can be detected, which can improve the accuracy of the detection results.

[0136] In this embodiment, a fourth similarity is introduced between semantic feature vectors and semantic behavior descriptions, and fused with the first similarity of resource indicators to form a more comprehensive target similarity. The dual-dimensional similarity fusion enhances the perception of the computing task context, enabling cross-modal alignment verification of resource usage behavior and task semantics. This allows anomaly detection to not only focus on whether resource usage is abnormal, but also to determine whether it conforms to the semantic behavior description. This effectively reduces false alarms caused by differences in task characteristics and can identify hidden anomalies where resource performance is normal but behavior semantics do not match, thereby improving the accuracy and semantic consistency of detection.

[0137] Figure 4 This is a flowchart illustrating a method for detecting abnormal resource usage behavior according to an embodiment of this application.

[0138] like Figure 4 As shown, the method for detecting abnormal resource usage behavior includes: Step 401: Obtain resource indicator data for the target time window.

[0139] Step 402: Determine the first similarity between the resource indicator data and the resource usage behavior fingerprint model corresponding to the task type to which the target computing task belongs.

[0140] Step 403: Based on the first similarity, detect abnormal resource usage behavior of the target computing task and obtain the detection results.

[0141] In this application, steps 401-403 can be implemented in any of the embodiments of this application, so they will not be described in detail here.

[0142] Step 404: Determine the corresponding response strategy based on the task type, exception level, and occupation type.

[0143] In this application, a mapping relationship between the exception level, occupation type, and response strategy for each task type can be pre-established. The task type, the exception level, and occupation type corresponding to the target time window are calculated based on the target, and the corresponding response strategy is determined by querying the mapping relationship. For example, the response strategy may include a response action.

[0144] As an example, for the pre-training phase, the mapping relationship is shown in Table 1 below: Table 1

[0145] For example, if the target computation task is of type pre-training, anomaly level Lev=3, and occupancy type is explicit occupancy, then the response strategy includes the set of response actions R={GPU isolation, task pause, administrator alert}.

[0146] Step 405: Execute the response strategy and obtain the response results.

[0147] In this application, the response strategy can be executed by calling the node control interface to obtain the response result, which may include whether the response was successful or failed. For example, for distributed computing tasks, the response action in the response strategy can be executed through the cluster control interface.

[0148] For example, response actions may include resource isolation, task pause, administrator alert, fingerprint model reconstruction, etc.

[0149] As an example, for resource isolation, for explicit occupancy, the container can limit video memory and computing resources; for implicit occupancy, abnormal threads can be temporarily frozen or tasks can be migrated to independent nodes. For task suspension, the task scheduling system's API can be called to pause task execution and prevent further resource damage. For administrator alerts, alert information can be pushed through a message queue, and the alert information can include occupancy type, feature contribution, occupancy time, etc. For model reconstruction, for cases where implicit occupancy causes abnormal task status, fingerprint model updates can be triggered.

[0150] In some embodiments, a delay mechanism may be introduced before the response strategy is executed to prevent false alarms.

[0151] For example, a response strategy can be executed after a target time delay, wherein the target time is calculated as shown in the following formula (12): (12) in, Indicates the target time. This indicates the maximum delay factor (e.g., 10 seconds). This represents the first similarity score corresponding to the target time window.

[0152] Optionally, when the target computing task triggers an unauthorized access response simultaneously with other computing tasks, that is, when multiple computing tasks need to execute a response strategy, the response strategy can be executed according to the task priority of the multiple computing tasks, such as prioritizing the high-priority computing task that has deviated significantly.

[0153] For example, the scheduling priority of a computation task can be determined based on the task priority and the first similarity, and the response strategy can be executed in descending order of scheduling priority.

[0154] For example, the scheduling priority can be determined using the following formula (13): (13) in, Represents computational task scheduling priority, Represents computational task Task priority, Represents computational task The corresponding first similarity.

[0155] To facilitate timely feedback of detection results, optionally, alarm information on abnormal resource usage behavior of the target computing task can be generated based on the detection results and response results. The alarm level can be determined according to the abnormality level corresponding to the target time window, and the alarm strategy corresponding to the alarm level can be adopted to send alarm information to the target object and obtain the target object's feedback on the alarm information.

[0156] For example, the target group could be operations and maintenance personnel, or other personnel responsible for system security.

[0157] For example, alarm information may include, but is not limited to, alarm subject, task identifier of the target computing task, task type, anomaly level, occupancy type, first similarity, deviation degree, key indicators, response action, response result, occupancy event time, node information, etc.

[0158] For example, a certain alarm message includes: alarm topic is "severe resource occupation alarm", task ID is Task_20260202_001, task type is pre-training, first similarity is 0.58, occupation type is explicit occupation, and response actions include GPU isolation and task pause.

[0159] For example, a mapping relationship between anomaly level, alarm level, and alarm policy can be established in advance. By querying this mapping relationship, the alarm level and alarm policy corresponding to the anomaly level in the target time window can be determined.

[0160] As an example, the mapping relationship between anomaly level, alarm level, and alarm policy is shown in Table 2 below: Table 2

[0161] As shown in Table 2, real-time alerts for unauthorized access events can be achieved through multiple channels (such as SMS, email, and message queues). Here, "unauthorized access event" refers to an event related to unauthorized access behavior.

[0162] For example, the target object can perform feedback operations such as confirmation and annotation on alarm information in the visual interface, and the system generates and returns feedback results based on the target object's feedback operations. For example, the type of feedback result (i.e., feedback type) can include confirmed occupancy, false alarm marking, and delayed confirmation. If it is confirmed occupancy, the system records it as a real occupancy event; if it is marked as a false alarm, the system records it as a false alarm sample; if it is delayed confirmation, the system does not process it and waits for further data. Here, "system" can refer to a resource usage behavior anomaly detection system.

[0163] Therefore, by combining the anomaly detection results of the target computing task with the response results of the executed response strategy, alarm information with context awareness is generated. The alarm level is dynamically determined according to the severity of the abnormal behavior, thereby triggering a differentiated alarm strategy matching the level, accurately pushing alarms to the target object, and further collecting the target object's feedback on the alarm to form a closed-loop feedback mechanism. This not only improves the accuracy and operability of alarms, but also continuously optimizes subsequent detection and response logic through feedback data, significantly enhancing the system's ability to govern abnormal resource usage behavior and its adaptive security protection level.

[0164] Optionally, when multiple occupancy events trigger alarms simultaneously, the alarm priority of the occupancy events can be determined based on their anomaly level, deviation degree, and maximum feature contribution, and alarm information can be pushed in descending order of alarm priority. Here, the maximum feature contribution refers to the largest contribution to the first similarity among the various resource indicators corresponding to the occupancy event, and the various resource indicators corresponding to the occupancy event refer to the resource indicators corresponding to the computing task to which the occupancy event belongs.

[0165] For example, the priority of an occupancy event can be calculated using the following formula (14): (14) in, Indicates the event of occupancy. Alarm priority, Indicates the event of occupancy. The anomaly level corresponding to the target time window, Indicates the event of occupancy. The degree of deviation from the corresponding target time window, Indicates the event of occupancy. The corresponding maximum feature contribution.

[0166] Optionally, the first threshold can be updated based on the feedback and detection results to obtain an updated first threshold. The first threshold can be used to determine whether resource usage behavior within a target time window is normal, i.e., the first threshold in the above embodiment.

[0167] For example, the feedback results from the target object can be converted into feedback labels. For instance, a feedback label of 1 indicates confirmed occupancy; a feedback label of 0 indicates a false alarm; and a feedback label of -1 indicates pending confirmation (i.e., delayed confirmation).

[0168] For example, the detection results can be converted into detection labels, which can be used to characterize whether the target computing task has engaged in unauthorized resource usage. For instance, a detection label of 1 indicates unauthorized usage; a detection label of 0 indicates normal resource usage and that no unauthorized resource usage is considered to exist.

[0169] For example, the first threshold can be updated based on the feedback label and the detection label to obtain the updated first threshold.

[0170] As an example, the first threshold can be updated using the following formula (15): (15) in, This represents the updated first threshold. Indicates the first threshold. Indicates feedback label, This indicates the detection label.

[0171] Therefore, the first threshold can be dynamically updated based on the feedback from the target object, thereby improving the accuracy of detecting abnormal resource usage behavior.

[0172] Optionally, if the number of confirmed occupancy events exceeds the threshold, a new behavior fingerprint template can be regenerated.

[0173] Optionally, detection results, response results, etc., can be encapsulated into structured event objects and stored.

[0174] For example, the detection time, the task identifier of the target calculation task, the task type, the first similarity, the degree of deviation, the anomaly level, the occupancy type, the response strategy, the key indicators, the response results, etc. can be encapsulated into a structured event object and stored.

[0175] For example, data management can be achieved by combining a time-series database with an indexing engine.

[0176] For example, time-series data storage is used to store time-series data such as similarity and resource utilization rates for consecutive time windows. Each record contains fields such as timestamp, task identifier, and resource metrics. The timestamp here can be the time when the currently detected event occurred or the time index corresponding to the target time window.

[0177] For example, offline detection events and response logs can be stored through an event index, supporting multi-condition retrieval. Each event record includes a unique event identifier, anomaly level, occupancy type, response action, and response result.

[0178] For example, composite indexes can be created for task identifiers, timestamps, anomaly levels, etc., to support fast retrieval by task, time, or severity.

[0179] In this embodiment, by comprehensively considering three dimensions—task type, anomaly level, and occupancy type—a suitable response strategy is dynamically matched and executed, achieving refined and scenario-aware automated handling. Through multi-dimensional criterion-based decision-making, interference with legitimate high-load tasks can be reduced. This not only improves the accuracy and effectiveness of security responses but also enhances the stability, interpretability, and operational efficiency of the entire computing environment.

[0180] In order to achieve multi-dimensional visualization, in some embodiments of this application, the original data can be modeled and aggregated to obtain similarity trend models, feature contribution models, event distribution models, etc.

[0181] For example, the moving average and fluctuation range of similarity over time can be calculated, and a similarity trend model can be described based on the moving average and fluctuation range. This similarity trend model can be used to draw a similarity trend curve.

[0182] For example, the moving average is also called the moving average similarity. The moving average similarity can be calculated using the above formula (4), and the fluctuation range can be calculated using the following formula (16): (16) in, Indicates the standard deviation of the fluctuation. Represents the moving average similarity. Indicates the length of the time window.

[0183] For example, the contribution of each resource indicator can be normalized to obtain the normalized contribution. Based on the normalized contribution, a feature contribution model can be constructed. This feature contribution model can be used to draw feature contribution radar charts or feature contribution bar charts, etc.

[0184] As an example, the contribution of resource indicators can be normalized using the following formula (17): (17) in, Indicates the first The contribution of each resource indicator after normalization Indicates the first The contribution of each resource indicator This indicates the quantity of resource indicators.

[0185] For example, an event distribution model can be obtained by statistically analyzing the distribution of the number of events at different anomaly levels, such as the proportion of events at each anomaly level to the total number of events. This model can then be used to draw a pie chart of the anomaly level distribution.

[0186] In some embodiments of this application, the resource usage behavior of computing tasks, similarity change trends, detection results, etc., can also be displayed in a dynamic chart format.

[0187] In some embodiments, a visualization engine can dynamically generate multi-dimensional charts, and the displayed content may include a comparison chart of computational task fingerprints, a similarity trend chart, a feature contribution radar chart, an event timeline chart, etc.

[0188] For example, the horizontal axis of the computation task fingerprint comparison chart represents time, and the vertical axis represents resource utilization (such as the utilization of GPU, CPU, memory, etc.). This comparison chart includes real-time monitoring data curves, behavioral fingerprint template mean curves, etc. When the two deviate significantly, the chart is highlighted with the target color.

[0189] For example, a similarity trend graph is a curve used to show how similarity changes over time, and when the similarity is below a first threshold, outliers are automatically marked.

[0190] For example, a feature contribution radar chart is used to display key indicators that cause anomalies, with the radius of each dimension representing the magnitude of the contribution.

[0191] For example, an event timeline can display detected events in chronological order, with different colors representing different color levels, and clicking on an event node allows you to view detailed event information.

[0192] Optionally, the data view can be refreshed every preset time interval (e.g., 5 seconds) to ensure real-time display.

[0193] Figure 5 This is a flowchart illustrating a method for constructing a resource usage behavior fingerprint model according to an embodiment of this application.

[0194] like Figure 5 As shown, the method for constructing this resource using a behavioral fingerprint model includes: Step 501: Obtain resource indicator data for multiple time windows.

[0195] In this application, resource indicator data may include resource indicators at each sampling time point within a corresponding time window obtained during the execution of historical computing tasks. Each time window has corresponding resource indicator data, and different time windows have the same length.

[0196] In this application, the explanations of resource index data and resource indexes at each sampling time point can be found in the explanations in the above embodiments. The explanations of historical calculation tasks can be found in the explanations of target calculation tasks in the above embodiments, so they will not be repeated here.

[0197] For example, the resource metrics at each sampling time point within each time window can be multi-dimensional data, such as GPU utilization, CPU utilization, etc.

[0198] For example, the resource indicator data for a time window can be in the form of an array, a matrix, or other mathematical expression, and this application does not limit it in this way.

[0199] It should be noted that the resource indicator data for each time window can be the raw data or the data after standardization and normalization, and this application does not limit it.

[0200] For example, the raw data collected within a time window can be represented as a time series matrix, as shown in the following formula (18): (18) in, This indicates the number of samples taken within a time window, i.e., the number of sampling time points. Indicates the quantity of resource indicators; Indicates the first The first sampling The value of each resource indicator.

[0201] Step 502: Extract statistical features from the same resource feature dimension in the resource indicator data to obtain the statistical feature vector of the corresponding time window.

[0202] Since resource usage during training has periodic fluctuations (e.g., GPU utilization decreases during gradient synchronization), this application can employ a sliding time window mechanism for aggregation.

[0203] In some embodiments, for any given time window, statistical features can be extracted from the resource indicator data of that time window for the same resource feature dimension, resulting in the statistical features corresponding to that resource feature dimension. Thus, the statistical features of each resource feature dimension constitute a statistical feature vector. Here, one feature dimension corresponds to one resource indicator; for example, GPU utilization can be considered a resource feature dimension.

[0204] For example, statistical feature extraction may include, but is not limited to, calculating the mean, variance, kurtosis, periodicity, etc. Kurtosis can be used to measure the sharpness of the distribution of resource features, and periodicity can be calculated by the autocorrelation function. Kurtosis and periodicity can be calculated using the formulas in the above embodiments.

[0205] Assume the window length is The sliding step size is Then the first The data contained in each time window is shown in the following formula (19): (19) For each time window, the mean can be calculated using the following formula (20): (20) in, It is the mean, representing the th The average level of each resource indicator within the time window; Indicates the sampling time point The first collection Resource indicators.

[0206] For example, assuming the time window length is 10 seconds, and the GPU utilization sequence within a certain time window is [95,96,94,97,95,96,94,95,96,95], then the mean of the GPU within that time window is 95.3, the variance is 0.84, and the kurtosis is 2.7.

[0207] For example, the first The statistical feature vector of each time window can be represented as:

[0208] in, They represent the first The resource feature dimension, that is, the first The mean, variance, kurtosis, and periodicity characteristics of each resource indicator.

[0209] So, The statistical feature vectors of each time window can form a statistical feature matrix, as shown in the following formula (21): (twenty one) Step 503: Obtain clusters by clustering statistical feature vectors of multiple time windows, and extract behavioral fingerprint templates of the clusters.

[0210] In this application, some clustering methods such as K-means clustering algorithm and DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm can be used to cluster the statistical feature vectors of multiple time windows to obtain clusters.

[0211] In some embodiments, the statistical feature vector of each time window can be regarded as a sample, and multiple samples can be clustered to obtain a cluster. Each cluster may include at least one sample, that is, the statistical feature vector of at least one time window.

[0212] Since the statistical feature vectors in a cluster have similar characteristics, behavioral fingerprint templates that can characterize resource usage patterns can be extracted from the cluster based on the statistical feature vectors in the cluster.

[0213] For example, a behavioral fingerprint template can be extracted from each cluster. A behavioral fingerprint template can be used to characterize a resource usage pattern under the target task type to which the historical computing task belongs.

[0214] For example, for each cluster, statistical features can be extracted to obtain a behavioral fingerprint template for each cluster.

[0215] As an example, for each cluster, the mean and variance of each statistical feature dimension in each statistical feature vector of the cluster can be calculated. The mean of each statistical feature dimension constitutes the mean vector, and the variance of each statistical feature dimension constitutes the variance vector. Based on the mean vector and the variance vector, the behavioral fingerprint template of the cluster can be determined.

[0216] For example, a clustered behavioral fingerprint template may include a mean vector and a variance vector. The mean vector can be used to describe the resource usage pattern, and the variance vector can be used to characterize the fluctuation range of the resource usage pattern.

[0217] Step 504: Based on the behavioral fingerprint templates of each cluster, determine the resource usage behavior fingerprint model corresponding to the target task type to which the historical computing task belongs.

[0218] In this application, the resource usage behavior fingerprint model corresponding to the target task type may include the behavior fingerprint template of each cluster.

[0219] For example, if three clusters are obtained through clustering, then the resource usage behavior fingerprint model can include three behavior fingerprint templates.

[0220] It is understood that the resource usage behavior fingerprint model construction method of the embodiments of this application can be used to construct resource usage behavior fingerprint models corresponding to different task types.

[0221] For example, we can construct resource usage behavior fingerprint models corresponding to the pre-training stage of a large model and resource usage behavior fingerprint models corresponding to the fine-tuning stage of a large model, thereby enabling the construction of resource usage behavior fingerprint models that adapt to different training stages of a large model.

[0222] For example, if a computing task has different running stages or different processing modes during execution, the resource usage behavior fingerprint model construction method of this application embodiment can be used to construct a corresponding resource usage behavior fingerprint model.

[0223] In this embodiment, statistical features are extracted from resource indicator data collected across multiple time windows for historical computing tasks, forming statistical feature vectors for each time window. Clustering is then used to identify clusters with similar resource usage patterns, and behavioral fingerprint templates are extracted from each cluster. Finally, a corresponding resource usage behavior fingerprint model is constructed for each target task type. This cluster-driven behavior fingerprint template generation effectively accommodates reasonable behavioral diversity within the same task type while isolating differences in resource usage patterns across different task types. This significantly improves the targeting, accuracy, robustness, and generalization ability of subsequent anomaly detection in resource usage behavior. Furthermore, the statistical feature-based modeling approach balances noise robustness and computational efficiency, providing scalable and generalizable technical support for establishing automated behavioral baselines for large-scale, multi-type computing tasks.

[0224] Figure 6 This is a flowchart illustrating a method for constructing a resource usage behavior fingerprint model according to an embodiment of this application.

[0225] like Figure 6 As shown, the method for constructing this resource using a behavioral fingerprint model includes: Step 601: Obtain resource indicator data for multiple time windows.

[0226] In this application, step 601 can be implemented in any of the embodiments of this application, so it will not be described again here.

[0227] Step 602: Extract statistical features from the same resource feature dimension in the resource indicator data to obtain the statistical feature vector of the corresponding time window.

[0228] In this application, step 602 can be implemented in any of the embodiments of this application, so it will not be described again here.

[0229] In some embodiments, if the resource indicator data is the raw data of the resource indicators, since the resource indicators may have anomalies such as sampling loss and instantaneous jitter, the resource indicator data can be anomaly detected and cleaned before statistical feature extraction.

[0230] For example, for the resource indicator data of each time window, the mean and variance of each resource feature dimension can be calculated, and anomaly detection can be performed on the resource indicators at different sampling time points under each resource feature dimension. If there are anomalies in the resource indicators, they can be corrected. The correction strategy can adopt the neighborhood interpolation method.

[0231] As an example, Z-score can be used for anomaly detection. If the absolute value of the Z-score of a resource indicator at a certain sampling time point under any resource feature dimension is greater than the preset value, the resource indicator can be considered abnormal. The average of the two normal values ​​of the resource indicator before and after that resource feature dimension can be used as the corrected value at that sampling time point.

[0232] For example, if the sequence of GPU utilization within a time window is [95,96,94,97,30,96,94], where 30 is an outlier, and after calculating the Z-score, it is found that it deviates from the mean by more than 3 times the standard deviation, then it is replaced with the mean of the two normal values ​​before and after it, 97 and 96, that is, 30 is replaced with (97+96) / 2=96.5.

[0233] Because different resource metrics have different dimensions, such as video memory being in MB and CPU utilization being in percentage, in order to improve accuracy, the resource metrics at each sampling time point in the original resource metric data or the resource metric data after anomaly detection can be normalized and standardized.

[0234] For example, the same resource feature dimension in the resource indicator data can be normalized according to the maximum and minimum values ​​corresponding to the same resource feature dimension within the time window to obtain normalized resource indicator data. Then, the normalized resource indicator data can be standardized to obtain standardized resource indicator data for the time window. Finally, statistical features can be extracted from the same resource feature dimension in the standardized resource indicator data to obtain a statistical feature vector.

[0235] In other words, for any resource feature dimension in the resource indicator data, the maximum and minimum values ​​of the resource indicators under that resource feature dimension can be determined, and then the Min-Max (minimum value - maximum value) method can be used to normalize the resource indicators at each sampling time point under that resource feature dimension.

[0236] For example, for normalized resource indicator data, the mean and variance of any resource feature dimension in the normalized resource indicator data can be determined. Based on the mean and variance, Z-score is used to standardize the normalized resource indicators at each sampling time point under the resource feature dimension, so that the mean of the resource feature dimension is 0 and the standard deviation is 1.

[0237] For example, after performing anomaly detection, normalization, and standardization on the original resource indicator data within a time window, statistical feature extraction can be performed on the same resource feature dimension in the processed resource indicator data to obtain the statistical feature vector of that time window.

[0238] Step 603: Obtain the semantic feature vector of the historical computing task.

[0239] To improve clustering accuracy, semantic features of historical computing tasks can be introduced for clustering. In this application, semantic feature vectors can be used to indicate the attribute information of historical computing tasks. For example, attribute information may include, but is not limited to, the task type, computing stage, algorithm type, distributed topology, batch processing parameters, etc. of historical computing tasks.

[0240] In model training scenarios, to establish a correlation between resource usage behavior and training semantics, training parameters can be obtained from the training task scheduling system, such as the number of samples per training session, learning rate, gradient synchronization interval, total number of model parameters, optimizer type, etc. These semantic parameters constitute the semantic feature vector of the model training task.

[0241] Step 604: The semantic feature vector and the statistical feature vector of the time window are fused to obtain the joint feature vector of the time window.

[0242] In this application, the semantic feature vector can be concatenated with the statistical feature vector of each time window to obtain the joint feature vector of each time window.

[0243] For example, the semantic feature vector is , The number of semantic feature dimensions is represented by the number of dimensions. The semantic features are concatenated with the statistical feature vectors of each time window to obtain the joint feature vector, as shown in the following formula (22): (twenty two) in, Indicates time window The joint eigenvectors, Indicates time window Statistical eigenvectors.

[0244] Optionally, for each time window, the resource indicators and semantic feature vectors at each sampling time point within the time window can be concatenated first to obtain the fused feature vector corresponding to each sampling time point. Then, statistical feature extraction is performed on the same resource feature dimension within the time window. Since the semantic feature vector and the resource indicators are different data types, no statistical feature processing is performed on the semantic feature vector to obtain the joint feature vector for each time window. The joint feature vector includes the statistical feature vector and the semantic feature vector.

[0245] In some embodiments, to avoid bias caused by the difference in the dimensions of different statistical features in the statistical feature vector, the statistical feature vectors of multiple time windows can be standardized to obtain standardized feature vectors of multiple time windows. Then, the semantic feature vector is concatenated with the standardized feature vector of each time window to obtain the joint feature vector of each time window.

[0246] For example, all features in the statistical feature vector can be standardized so that their mean is 0 and their variance is 1.

[0247] For example, the statistical feature vector of each time window can be represented as: , Representing feature dimension, Indicates the first The first sample The values ​​of each feature, where the features can be understood as statistical features in the statistical feature vector (such as mean, variance, kurtosis, etc.), can be standardized using the following formula (23): (twenty three) in, Indicates the first The mean of each feature, Indicates the first The standard deviation of each feature.

[0248] Therefore, by standardizing the statistical feature vectors of multiple time windows, we can ensure that different resource feature dimensions are comparable on the same scale, thus providing a unified distance metric basis for subsequent cluster analysis.

[0249] Step 605: Cluster the joint feature vectors of multiple time windows to obtain clusters, and extract the behavioral fingerprint templates of the clusters.

[0250] In this application, each cluster includes a joint feature vector of at least one time window. The statistical feature vector and semantic feature vector in the joint feature vector of each cluster are processed differently to obtain the behavioral fingerprint template of the cluster.

[0251] In some embodiments, the joint feature vector of each time window can be used as a sample, and clustering algorithms such as K-Means and DBSCAN can be employed to cluster the samples in the cluster. For example, clustering can be performed using appropriate clustering algorithms based on the data distribution characteristics.

[0252] As an example, the K-Means clustering algorithm can be used for clustering. The goal of the K-Means clustering algorithm is to minimize the average variance within clusters, as shown in the following formula (24): (twenty four) in, Represents the number of clusters, Indicates the first A cluster, Indicates the first The center of a cluster, Indicates belonging to the first A sample of a cluster.

[0253] For example, the algorithm iteration process of K-Means clustering is as follows: (1) Random initialization (2) Assign each sample to the nearest cluster center; (3) Update the cluster center to the mean of the samples within the cluster; (4) Repeat steps (2)-(3) until convergence.

[0254] For example, if the resource indicator data is noisy or has a non-spherical distribution, the DBSCAN clustering algorithm can be used.

[0255] In some embodiments, for each cluster, statistical features can be extracted from the statistical feature vectors in each joint feature vector of the cluster to obtain the resource behavior template of the cluster. Semantic category statistics can be performed on the semantic feature vectors in each joint feature vector of the cluster to obtain the semantic behavior description of the cluster. Then, based on the resource behavior template and the semantic behavior description, the behavior fingerprint template of the cluster can be determined.

[0256] For example, extracting statistical features from the statistical feature vectors of each joint feature vector in a cluster to obtain the cluster's resource behavior template can be achieved in the following way: The mean and variance of the statistical feature vectors in the cluster can be calculated for the same statistical feature dimension to obtain the mean and variance of each statistical feature dimension. Based on the mean of each statistical feature dimension, a mean vector is obtained; based on the variance of each statistical feature dimension, a variance vector is obtained. Based on the mean vector and variance vector, the resource behavior template is obtained. Here, the mean vector describes the resource usage pattern, and the variance vector characterizes the fluctuation range of the resource usage pattern.

[0257] As an example, the first The mean vector of each cluster variance vector Statistical characteristics constitute a resource behavior template, which can be represented by the following formula (25): (25) The K-Means clustering algorithm was applied to the pre-training data, with a cluster size of 3, resulting in three typical resource usage patterns: Cluster C1: GPU utilization is stable at 95%, with 38GB of VRAM usage; Cluster C2: GPU utilization fluctuates significantly (80–95%), with active I / O; Cluster C3: GPU utilization drops to 60%, while network bandwidth increases. The mean and variance of each statistical feature dimension in each cluster were calculated, yielding the mean vector and variance vector for each cluster. Based on the mean and variance vectors of each cluster, a behavioral fingerprint template was obtained.

[0258] Therefore, by calculating the mean and variance of the statistical characteristics of resource indicators respectively, a mean vector and a variance vector are constructed. Based on the mean vector and variance vector, a resource behavior template is generated. This not only constructs the central trend of normal behavior, but also constructs a dynamic tolerance range, so that subsequent anomaly detection can not only identify anomalous mutations that deviate significantly from the mean, but also tolerate changes that conform to historical fluctuation patterns. This enables adaptive modeling of resource usage behavior and significantly improves the accuracy and robustness of anomaly detection.

[0259] For example, the semantic behavior description of a cluster may include, but is not limited to, the cluster's main semantic label, semantic distribution, semantic constraints, etc.

[0260] For example, for each cluster, the semantic values ​​of the samples in the cluster can be counted, and the semantic value with the most occurrences can be used as the main semantic label of the cluster. For instance, if a cluster has 3 samples, and the task type of each sample is training, i.e., task_type=training, then the main semantic label of the cluster is pre-trained, which can be represented as semantic_label=training.

[0261] For example, semantic distribution can refer to the distribution of different attribute values ​​of the same semantic attribute among samples in a cluster.

[0262] For example, semantic constraints may include, but are not limited to, task framework, algorithm type, etc.

[0263] For example, the cluster's resource behavior template and semantic behavior description can be used as a behavior fingerprint template. That is, the behavior fingerprint template can include the resource behavior template and the semantic behavior description.

[0264] As an example, the resource usage behavior fingerprint model can be represented by the following equation (26): (26) in, yes A resource behavior template, yes Each semantic behavior description, resource behavior template and semantic behavior description with the same subscript are extracted from the same cluster, forming a behavior fingerprint template, such as... and This forms a behavioral fingerprint template.

[0265] In this embodiment, resource behavior templates representing the typical usage patterns of the cluster are extracted from the resource dimension, and frequently occurring semantic behavior descriptions are summarized from the task semantic dimension. The two are then integrated to construct a structured behavior fingerprint template, realizing the collaborative modeling of resource behavior and task semantics. This not only makes the fingerprint template interpretable, facilitating subsequent detection and problem localization, but also enhances the ability to detect anomalies. Without the need for manual annotation, a multi-dimensional behavior baseline that combines pattern representativeness, semantic readability, and security sensitivity is constructed, which can significantly improve the accuracy and reliability of identifying hidden abnormal behaviors in complex computing environments.

[0266] Step 606: Based on the behavioral fingerprint templates of each cluster, determine the resource usage behavior fingerprint model corresponding to the target task type to which the historical computing task belongs.

[0267] In this application, step 606 can be implemented in any of the embodiments of this application, so it will not be described in detail here.

[0268] In some embodiments, the task type identifier of historical computing tasks can be obtained from the task scheduling system. Based on the identifier, the target task type to which the historical computing task belongs can be determined. Thus, during the data acquisition phase, resource indicator data under the target task type can be obtained. Based on the resource indicator data, a resource usage behavior fingerprint model corresponding to the target task type can be constructed.

[0269] For example, the label of the current stage can be obtained from the model training scheduling system, and the training stage of the historical training task can be determined based on the label. The training stage can be directly labeled when collecting resource indicator data.

[0270] Since a computational process may contain computational tasks of multiple task types, and the resource characteristics of different task types may change differently, for example, the training process of a large model usually contains multiple stages (such as pre-training, fine-tuning, etc.), and the resource usage patterns of different stages are significantly different. Therefore, in some embodiments, if the computational process to which the historical computational task belongs includes computational tasks of multiple task types, different task types can be identified through resource usage behavior change detection.

[0271] For example, statistical feature vectors corresponding to two adjacent time windows in the calculation process can be obtained, and the resource feature change rate can be determined based on the statistical feature vectors corresponding to the two adjacent time windows. Then, the target task type can be determined based on the relationship between the resource feature change rate and the type switching threshold.

[0272] For example, the rate of change of resource characteristics can be used to characterize the degree of change of resource characteristics between adjacent time windows in the computation process.

[0273] As an example, the rate of change of resource characteristics can be calculated using the following formula (27): (27) in, Indicates time window The corresponding standardized feature statistics vector, Indicates time window The corresponding standardized feature statistical vector.

[0274] For example, if the rate of change of resource characteristics is greater than the type switching threshold corresponding to the target task type, it can be determined that the computing process is in the process of running a historical computing task of the target task type, that is, the task type of the historical computing task is the target task type.

[0275] For example, the type switching threshold corresponding to the target task type is the threshold used to determine whether other types should switch to the target task type. It can be understood that in a computational process, if there are multiple task type switching points, then each switching point has a corresponding type switching threshold.

[0276] For example, the type switching threshold can be determined based on the statistical distribution of resource characteristics that change in the previous calculation task, or dynamically adjusted based on the quantile values ​​in the distribution of cluster profile coefficients and resource characteristic change rates.

[0277] As an example, the silhouette coefficient of a cluster can be calculated using the following formula (28): (28) in, This represents the average distance from a sample to other samples within the same cluster; This represents the average distance from a sample to the nearest cluster, which is the average distance between the sample and all samples in the nearest cluster. The closer the result is to 1, the better the clustering effect.

[0278] Taking model training as an example, a model training process can include a pre-training stage, a fine-tuning stage, etc. If the type switching threshold (also known as the stage switching threshold in training scenarios) is greater than the type switching threshold corresponding to the fine-tuning stage, the current computation process can be considered to be in the fine-tuning stage. The type switching threshold for this stage is derived from the type switching threshold at the switching point between the pre-training and fine-tuning stages. This stage switching threshold can be adaptively determined based on the statistical distribution of resource features changing in historical training tasks, or dynamically adjusted based on the quantiles of the distribution of cluster silhouette coefficients and resource feature change rates.

[0279] For example, if the GPU utilization rate drops sharply from 95% to 60% and the video memory usage decreases by 20%, the rate of change increases significantly, indicating a switch from the "training phase" to the "validation phase".

[0280] For example, in a large model training task, by detecting the rate of change of resource features, it is found that the first 200 seconds are the "pre-training phase" and the last 200-300 seconds are the "validation phase". Based on the resource indicator data from the first 200 seconds, a fingerprint model of resource usage behavior corresponding to the pre-training phase can be constructed, and based on the resource indicator data from the last 200-300 seconds, a fingerprint model of resource usage behavior corresponding to the validation phase can be constructed.

[0281] In this embodiment, the statistical feature vectors of two adjacent time windows in the computation process are obtained, and their differences are calculated to quantify the rate of change of resource characteristics in the time series. Based on the relationship between this rate of change and a preset type switching threshold, it is determined whether the task type has switched, thereby dynamically classifying and identifying the target task type to which historical computation tasks belong. This leverages the characteristic that resource behavior typically undergoes significant jumps at task boundaries, transforming the task type identification problem into the detection of resource pattern mutation points, avoiding reliance on task metadata or manual annotation. Furthermore, since the statistical feature vectors have filtered out instantaneous noise, the calculated rate of change more stably reflects real task switching rather than random fluctuations, improving the accuracy of type classification. In addition, this method can adapt to complex computation processes involving multi-task mixed execution, enabling fine-grained, adaptive task type boundary discovery, laying the foundation for subsequently building an accurate resource usage behavior fingerprint model.

[0282] In this embodiment, by fusing the semantic feature vector of historical computing tasks with the statistical feature vector of the corresponding time window, a joint feature vector containing both resource behavior and task semantic information is constructed. Clustering is performed based on the joint feature vector, so that the clustering results not only reflect the similarity of resource usage patterns, but also take into account the consistency of task semantics. This can improve the ability to distinguish behavior patterns and be used for behavior pattern interpretation or auxiliary judgment.

[0283] Furthermore, this clustering method can effectively avoid misclassifying different semantic tasks into the same category due to their occasional similar resource consumption (for example, "data preprocessing" and "model inference" may have similar CPU loads but are essentially different). At the same time, it prevents the same semantic task from being fragmented into multiple clusters due to resource pattern dispersion caused by fluctuations in the operating environment. The resulting clusters more accurately correspond to typical behavior categories in real business scenarios, laying the foundation for building a high-fidelity, interpretable resource usage behavior fingerprint model and significantly improving the accuracy of anomaly detection and semantic alignment capabilities.

[0284] To improve the accuracy of the resource usage behavior fingerprint model, in some embodiments of this application, the resource usage behavior fingerprint model can be updated in real time.

[0285] For example, a statistical feature vector corresponding to any time window during the execution of the target computing task can be obtained, and the feature similarity between the statistical feature vector corresponding to the time window and any behavioral fingerprint template can be determined. If the feature similarity is less than the anomaly threshold corresponding to the target task type, the resource usage behavior fingerprint model can be updated based on the resource indicator data of multiple time windows obtained during the execution of the target computing task.

[0286] For example, the target computing task and the historical computing task have the same task type, both belonging to the target task type. The target computing task is any computing task in the real-time detection phase of abnormal resource usage behavior.

[0287] For example, feature similarity can be the cosine similarity between a statistical feature vector and any behavioral fingerprint template, or it can be a similarity calculated in other ways, such as Gaussian similarity, etc. This application does not limit it in this regard.

[0288] As an example, the Gaussian similarity between the statistical feature vector corresponding to this time window and any behavioral fingerprint template can be calculated using the following formula (29): (29) in, The standardized statistical feature vector representing the time window. Behavioral fingerprint template The mean vector, Behavioral fingerprint template Variance vector.

[0289] If the Gaussian similarity between the statistical feature vector corresponding to the time window and any behavioral fingerprint template of the resource use behavior fingerprint model is less than the anomaly threshold corresponding to the target task type, the resource use behavior fingerprint model update process is triggered to re-cluster and generate new fingerprint templates.

[0290] For example, the anomaly threshold can be a threshold used to determine whether the resource usage behavior of a time window is abnormal during the real-time detection of abnormal resource usage behavior of a computing task of the target task type. It serves the same function as the first threshold in the above embodiment.

[0291] In this embodiment, both the target computation task and the historical computation task belong to the target task type. Based on the feature similarity between the target computation eigenvector and any behavioral fingerprint template corresponding to the target task type, and the feature similarity of any time window during task execution, it is compared with the anomaly threshold corresponding to the target task type. If it is less than the anomaly threshold, the resource usage behavior fingerprint model is updated, thereby realizing the real-time update of the resource usage behavior fingerprint model and improving the accuracy of subsequent resource usage behavior anomaly detection.

[0292] The method for constructing the resource usage behavior fingerprint model and the method for detecting resource usage behavior anomalies in the above embodiments can be executed by a resource usage behavior anomaly detection system. To facilitate understanding of the solutions in this application's embodiments, the following example uses a model training scenario, combined with... Figure 7 To explain, Figure 7 This is a schematic diagram illustrating the calling process between the modules of the resource usage behavior anomaly detection system provided in this application embodiment.

[0293] In this embodiment, the resource usage behavior anomaly detection system may include a feature acquisition and preprocessing module, a training behavior fingerprint modeling module, a real-time analysis and similarity calculation module, an unauthorized access determination and response module, and a visualization and alarm center module.

[0294] 1. The feature acquisition and preprocessing module is responsible for collecting multi-dimensional resource usage data for training tasks from the cluster monitoring system and performing standardization and feature extraction. Its main functions include: Multi-dimensional resource metrics collection: collecting GPU utilization, video memory usage, CPU load, memory usage, I / O throughput, network bandwidth, etc.

[0295] Training semantic feature fusion: combining parameters such as batch size, learning rate, and gradient synchronization interval of the training task.

[0296] Time series feature extraction: The collected data is aggregated by time window to extract statistical features (such as mean, variance, kurtosis, periodicity, etc.).

[0297] Abnormal data cleaning: Remove monitoring anomalies and sampling loss points.

[0298] 2. The training behavior fingerprint modeling module is used to construct a behavior fingerprint model that reflects the resource usage patterns of the training task. Its main functions include: Feature space construction: Mapping multi-dimensional resource features to a unified feature space.

[0299] Phased modeling: Establish resource usage behavior fingerprint models for different stages of training (pre-training, fine-tuning, etc.).

[0300] Fingerprint template generation: Extract typical resource usage patterns through clustering algorithms (such as DBSCAN, K-means).

[0301] Model storage and version management: Supports versioned storage and dynamic updates of fingerprint models.

[0302] 3. The real-time behavior analysis and similarity calculation module is used to analyze resource usage behavior in the current training task in real time and calculate similarity with the fingerprint model. Its main functions include: Real-time data stream access: Access resource indicator streams from the monitoring system in real time.

[0303] Feature matching and normalization: Standardize real-time data and align it with the feature dimensions of the fingerprint model.

[0304] Similarity calculation: Dynamic time warping (DTW) or cosine similarity is used to calculate behavioral similarity.

[0305] Deviation detection: When the similarity is below a threshold, an anomaly marker is generated for the current time window. This anomaly marker indicates abnormal resource usage behavior within the current time window.

[0306] 4. The unauthorized access detection and response module is used to detect and respond to unauthorized access based on similarity analysis results. Its main functions include: Multi-threshold determination mechanism: different thresholds are set according to different training stages (such as the first threshold mentioned above).

[0307] Occupation type identification: Distinguish between explicit occupancy (such as external tasks occupying the GPU) and implicit occupancy (such as internal thread exceptions).

[0308] Response strategy execution: including resource isolation, task suspension, administrator alerts, etc.

[0309] Adaptive threshold adjustment: The threshold is dynamically adjusted based on historical detection results.

[0310] 5. The visualization and alarm module provides a visual display of the system's operating status and detection results. Key functions include: Behavioral fingerprint visualization: Displays a comparison between the resource usage curve of the training task and the behavioral fingerprint template.

[0311] Intrusion event tracking: Records the time, type, and scope of impact of intrusion events.

[0312] Alarm notification mechanism: Notify administrators via email, message queue, etc.

[0313] Interpretability analysis: Reveals the key feature dimensions that lead to deviations.

[0314] like Figure 7 As shown, during the construction phase: the feature acquisition and preprocessing module can collect historical resource indicator data from the monitoring system (such as a monitoring system used to monitor a cluster). The feature acquisition and preprocessing module can provide the historical resource indicator data from the training phase to the training behavior fingerprint modeling module. The training behavior fingerprint modeling module builds a training behavior fingerprint model (corresponding to the above-mentioned resource usage behavior fingerprint model) based on the received data and outputs the fingerprint model to the real-time behavior analysis and similarity calculation module for its real-time detection.

[0315] During the real-time detection phase: The feature acquisition and preprocessing module continuously collects real-time resource indicator data from the monitoring system and provides a real-time feature stream to the real-time behavior analysis and similarity calculation module. This module calculates the similarity between the real-time resource indicator data within the current time window and the trained behavior fingerprint model, calculates the deviation based on the similarity, and sends the similarity calculation result along with an anomaly marker for the current time window to the unauthorized access determination and response module. The unauthorized access determination and response module determines the anomaly level and access type, executes the response strategy, and pushes the detection results and response logs to the visualization and alarm module. The visualization and alarm module can generate visual icons and alarm information, send alarm information to the administrator, receive feedback from the administrator, and then feed back the access determination result to the unauthorized access determination and response module. Additionally, the visualization and alarm module can trigger the trained behavior fingerprint module to optimize the fingerprint model and update thresholds.

[0316] Taking a large model training scenario as an example, the solution of this application embodiment can take the multi-dimensional resource usage characteristics of the training stage as the core, establish a resource usage pattern profile of the training task, collect resource indicators such as GPU, CPU, memory, I / O, and network in real time, and perform dynamic similarity analysis with the resource usage fingerprint model construction method corresponding to the training stage. When a significant deviation is detected between the real-time behavior and the fingerprint model, the system determines that there is unauthorized resource usage behavior (i.e., the above-mentioned unauthorized resource occupation behavior), thereby triggering an alarm or automatic isolation mechanism.

[0317] In large-scale model training scenarios, the solution in this application can combine the semantic features of large-scale model training (such as batch size, gradient synchronization frequency, model parameter scale, etc.) with resource behavior patterns to form a semantically enhanced behavioral fingerprint template. It can replace the traditional identity authentication mechanism with a behavioral similarity measurement mechanism to detect implicit intrusions. It supports multi-stage training behavior modeling such as pre-training and fine-tuning, can dynamically adapt to the resource usage patterns of different training stages, and has self-learning and adaptive capabilities to automatically update the behavioral fingerprint model according to the evolution of training tasks.

[0318] It is understandable that the above-mentioned training behavior fingerprint modeling module can be a resource usage behavior fingerprint model building module, which can be used in model training scenarios as well as in other computing scenarios for building resource usage behavior fingerprint models.

[0319] In this embodiment of the application, if only resource indicator data is used in the real-time stage, then only the resource behavior template in the behavior fingerprint template is used in the similarity calculation, and the semantic behavior description in the behavior fingerprint template is not involved in the similarity calculation.

[0320] In some embodiments, when constructing a behavioral fingerprint template, resource features (i.e., resource indicator data) and semantic features of task types can jointly participate in clustering modeling to improve the accuracy of pattern segmentation. After clustering, the resource features in the fingerprint template are used for similarity calculation in the real-time stage, while the semantic features are mainly used to describe the semantic attributes of the clusters or as auxiliary judgment conditions. When using only resource features for abnormal resource usage behavior detection, similarity is calculated using only the resource behavior template.

[0321] To implement the above embodiments, this application also proposes a resource usage behavior anomaly detection device. Figure 8 This is a schematic diagram of the structure of a resource usage behavior anomaly detection device provided in an embodiment of this application.

[0322] like Figure 8 As shown, the resource usage behavior anomaly detection device 800 includes: The first acquisition module 810 is used to acquire resource indicator data for the target time window; wherein, the resource indicator data includes resource indicators at each sampling time point within the target time window acquired during the execution of the target computing task. The first determining module 820 is used to determine the first similarity between the resource indicator data and the resource usage behavior fingerprint model corresponding to the task type to which the target computing task belongs; wherein, the resource usage behavior fingerprint model is used to characterize the resource usage pattern of the task type; The detection module 830 is used to detect abnormal resource usage behavior of the target computing task based on the first similarity and obtain the detection results.

[0323] Optionally, the first determining module 820 is used for: Statistical features are extracted from the same resource feature dimension in the resource indicator data to obtain the statistical feature vector of the target time window; Determine the second similarity between the statistical feature vector and the resource use behavior fingerprint model; The first similarity is determined based on the second similarity.

[0324] Optionally, the resource usage behavior fingerprint model includes multiple behavior fingerprint templates, where one behavior fingerprint template is used to characterize a resource usage pattern for a task type. The first determining module 820 is used for: Determine the behavioral similarity between the statistical feature vector and the mean vector in the behavioral fingerprint template; where the mean vector is used to describe resource usage patterns. The highest behavioral similarity among multiple behavioral fingerprint templates is determined as the second similarity.

[0325] Optionally, the detection module 830 is configured to determine the detection result based on at least one of the following: The relationship between the first similarity score and the first threshold corresponding to the task type; The average of the first similarity and the third similarity; where the third similarity is the similarity between the time windows adjacent to the target time window, and the similarity is used to characterize the degree of similarity between the resource indicator data and the resource usage behavior fingerprint model of the adjacent time windows; The change in key indicators when there is unauthorized resource occupation in the target calculation task; among them, key indicators are resource indicators associated with unauthorized resource occupation. The contribution of key indicators to the first similarity score when the target computation task exhibits unauthorized resource occupation behavior; When the target computation task exhibits unauthorized resource occupation behavior, the correlation of similarity between adjacent time windows is considered; where the corresponding similarity is the similarity between the resource indicator data of each time window in adjacent time windows and the resource usage behavior fingerprint model.

[0326] Optionally, the detection module 830 is used for: In response to the size relationship indicating that the first similarity is greater than or equal to the first threshold, a detection result is generated to indicate that there is no abnormal resource usage behavior in the target time window; In response to the size relationship indicating that the first similarity is less than the first threshold, the abnormality level of abnormal resource use behavior is determined based on the mean. Determine the type of resource occupation based on at least one of the following: change, contribution, and correlation. Based on at least one of the occupancy type and anomaly level, generate detection results to indicate abnormal resource usage behavior within a target time window.

[0327] Optionally, the determination of unauthorized resource usage by the target computation task is made by the second determination module in the device: Based on the feature contribution vector, the feature contribution variance is determined; whereby the feature contribution vector includes the contribution of resource indicators under each resource feature dimension to the first similarity. Anomaly scores are determined based on the first similarity score, the variance of feature contribution, and the rate of change of similarity. In response to an abnormal score exceeding the second threshold, the target computing task is determined to have unauthorized resource occupation behavior based on at least one of the resource usage behavior categories and feature contribution vectors from multiple consecutive time windows. Among them, the similarity change rate is used to characterize the trend of the similarity between resource indicator data and resource usage behavior fingerprint model over time in multiple consecutive time windows, and the anomaly score is used to characterize the degree of anomaly in abnormal resource usage behavior.

[0328] Optionally, the first threshold is determined using a third determining module in the device: Determine the first mean and standard deviation of the similarity between resource indicator data and resource usage behavior fingerprint model for multiple time windows during the execution of historical computing tasks; wherein the historical computing tasks and the target computing tasks have the same task type. The first threshold is determined based on the first mean and standard deviation.

[0329] Optionally, the device may further include: a fourth determining module, used to determine a second average value of the similarity between resource indicator data and resource usage behavior fingerprint model in multiple time windows during the operation of the target computing task; and an adjusting module, used to adjust the first threshold according to the second average value to obtain the adjusted first threshold.

[0330] Optionally, the resource usage behavior fingerprint model includes a behavior fingerprint template, and a behavior fingerprint template includes a semantic behavior description. Based on the first similarity, the detection module 830 is used to: obtain the semantic feature vector of the target computing task; determine the fourth similarity between the semantic feature vector and the semantic behavior description; determine the target similarity based on the first similarity and the fourth similarity; and detect abnormal resource usage behavior of the target computing task based on the target similarity to obtain the detection result.

[0331] Optionally, the detection results include the abnormality level of the abnormal resource usage behavior of the target computing task and the occupation type of the unauthorized resource occupation behavior. The device may also include: a fifth determining module, used to determine the corresponding response strategy according to the task type, abnormality level and occupation type; and an execution module, used to execute the response strategy and obtain the response result.

[0332] Optionally, the device may further include: a generation module, used to generate alarm information for abnormal resource usage behavior of the target computing task based on the detection results and response results; a sixth determination module, used to determine the alarm level based on the anomaly level; a sending module, used to send the alarm information to the target object using the alarm strategy corresponding to the alarm level; and a second acquisition module, used to acquire the feedback result of the target object on the alarm information.

[0333] Optionally, the device may further include: an update module, used to update the first threshold based on feedback results and detection results, and obtain the updated first threshold; wherein the first threshold is used to determine whether resource usage behavior is abnormal within a target time window.

[0334] It should be noted that the explanation of the above-mentioned resource usage behavior anomaly detection method embodiment also applies to the resource usage behavior anomaly detection device of this embodiment, so it will not be repeated here.

[0335] In this embodiment, resource indicator data during the execution of computing tasks is collected within a target time window and compared with a resource usage behavior fingerprint model pre-built for that task type. This quantifies the consistency between the current running behavior and the expected normal behavior, enabling abnormal detection of resource usage behavior in the target computing task. This allows for the identification of abnormal behavior under legitimate scheduling, significantly improving the sensitivity and accuracy of resource usage behavior detection. Furthermore, because the model is customized based on task type, it takes into account the differences in resource usage behavior under different business loads, achieving adaptive detection of resource usage behavior for computing tasks of different task types and improving the accuracy of detection.

[0336] To implement the above embodiments, this application also proposes a device for constructing a resource usage behavior fingerprint model. Figure 9 This is a schematic diagram of the structure of a resource usage behavior fingerprint model construction device provided in an embodiment of this application.

[0337] like Figure 9 As shown, the resource usage behavior fingerprint model construction apparatus 900 includes: The first acquisition module 910 is used to acquire resource indicator data for multiple time windows; wherein, the resource indicator data includes resource indicators at each sampling time point within the corresponding time window acquired during the operation of historical computing tasks; The first extraction module 920 is used to extract statistical features from the same resource feature dimension in the resource indicator data to obtain the statistical feature vector of the corresponding time window. The second extraction module 930 is used to obtain clusters obtained by clustering statistical feature vectors of multiple time windows, and to extract behavioral fingerprint templates of the clusters; wherein, a behavioral fingerprint template is used to characterize a resource usage pattern. The first determining module 940 is used to determine the resource usage behavior fingerprint model corresponding to the target task type to which the historical computing task belongs, based on the behavior fingerprint template of each cluster; wherein, the resource usage behavior fingerprint model is used to characterize the resource usage pattern of the target task type.

[0338] Optionally, the second extraction module 930 is used for: Obtain the semantic feature vector of the historical computing task; whereby the semantic feature vector is used to indicate the attribute information of the historical computing task. The semantic feature vector and the statistical feature vector of the time window are fused to obtain the joint feature vector of the time window; Clustering is performed on the joint feature vectors of multiple time windows to obtain clusters.

[0339] Optionally, the second extraction module 930 is used for: Statistical features are extracted from the statistical feature vectors of each joint feature vector in the cluster to obtain the resource behavior template of the cluster; Semantic category statistics are performed on the semantic feature vectors in each joint feature vector of the cluster to obtain the semantic behavior description of the cluster; The cluster's resource behavior template and semantic behavior description are used as a behavior fingerprint template.

[0340] Optionally, the second extraction module 930 is used for: The mean and variance of the same statistical feature dimension are calculated for the statistical feature vectors in the cluster to obtain the mean and variance of each statistical feature dimension. Based on the mean of each statistical feature dimension, a mean vector is obtained; the mean vector is used to describe resource usage patterns. Based on the variance of each statistical feature dimension, a variance vector is obtained; whereby the variance vector is used to characterize the fluctuation range of resource usage patterns. Based on the mean vector and variance vector, a resource behavior template is obtained.

[0341] Optionally, the second extraction module 930 is used for: The statistical feature vectors of multiple time windows are standardized to obtain standardized feature vectors for multiple time windows. The semantic feature vector is concatenated with the standardized feature vector of the time window to obtain the joint feature vector.

[0342] Optionally, the computational process to which the historical computation task belongs includes computational tasks of multiple task types, and the target task type is determined using the following modules in the device: The second acquisition module is used to acquire the statistical feature vectors corresponding to two adjacent time windows in the calculation process. The second determining module is used to determine the resource characteristic change rate based on the statistical feature vectors corresponding to two adjacent time windows; wherein, the resource characteristic change rate is used to characterize the degree of change of resource characteristics between adjacent time windows; The third determination module is used to determine the target task type based on the relationship between the resource characteristic change rate and the type switching threshold.

[0343] Optionally, the device may further include: The third acquisition module is used to acquire the statistical feature vector corresponding to any time window during the execution of the target computing task; wherein, the target computing task and the historical computing tasks have the same task type; The fourth determination module is used to determine the feature similarity between the statistical feature vector corresponding to any time window and any behavioral fingerprint template; The update module is used to update the resource usage behavior fingerprint model in response to an anomaly threshold corresponding to the target task type when the feature similarity is less than the anomaly threshold. This update is based on resource indicator data obtained from multiple time windows during the execution of the target computation task.

[0344] It should be noted that the explanation of the above-mentioned method for constructing the resource usage behavior fingerprint model also applies to the apparatus for constructing the resource usage behavior fingerprint model in this embodiment, and therefore will not be repeated here.

[0345] In this embodiment, statistical features are extracted from resource indicator data collected across multiple time windows for historical computing tasks, forming statistical feature vectors for each time window. Clustering is then used to identify clusters with similar resource usage patterns, and behavioral fingerprint templates are extracted from each cluster. Finally, a corresponding resource usage behavior fingerprint model is constructed for each target task type. This cluster-driven behavior fingerprint template generation effectively accommodates reasonable behavioral diversity within the same task type while isolating differences in resource usage patterns across different task types. This significantly improves the targeting, accuracy, robustness, and generalization ability of subsequent anomaly detection in resource usage behavior. Furthermore, the statistical feature-based modeling approach balances noise robustness and computational efficiency, providing scalable and generalizable technical support for establishing automated behavioral baselines for large-scale, multi-type computing tasks.

[0346] According to embodiments of this application, this application also provides an electronic device, a readable storage medium, and a computer program product.

[0347] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0348] like Figure 10 As shown, device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes based on a computer program stored in ROM (Read-Only Memory) 1002 or loaded from storage unit 1008 into RAM (Random Access Memory) 1003. RAM 1003 may also store various programs and data required for the operation of device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. I / O (Input / Output) interface 1005 is also connected to bus 1004.

[0349] Multiple components in device 1000 are connected to I / O interface 1005, including: input unit 1006, such as keyboard, mouse, etc.; output unit 1007, such as various types of monitors, speakers, etc.; storage unit 1008, such as disk, optical disk, etc.; and communication unit 1009, such as network card, modem, wireless transceiver, etc. Communication unit 1009 allows device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0350] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphics Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as the resource usage behavior anomaly detection method. For example, in some embodiments, the resource usage behavior anomaly detection method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the resource usage behavior anomaly detection method described above can be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform a resource usage behavior anomaly detection method by any other suitable means (e.g., by means of firmware).

[0351] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System-on-Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0352] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0353] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0354] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0355] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include LANs (Local Area Networks), WANs (Wide Area Networks), the Internet, and blockchain networks.

[0356] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is established by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0357] It should be noted that the electronic device used to implement the resource usage behavior fingerprint model construction method of the embodiments of this application is similar to the electronic device described above, so it will not be described again here.

[0358] According to embodiments of this application, this application also provides a computer program product that, when the instruction processor in the computer program product is executed, performs the resource usage behavior anomaly detection method or the resource usage behavior fingerprint model construction method proposed in the above embodiments of this application.

[0359] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0360] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for detecting abnormal resource usage behavior, comprising: Obtain resource indicator data for a target time window; wherein, the resource indicator data includes resource indicators at each sampling time point within the target time window obtained during the execution of the target computing task; Determine the first similarity between the resource indicator data and the resource usage behavior fingerprint model corresponding to the task type to which the target computing task belongs; wherein, the resource usage behavior fingerprint model is used to characterize the resource usage pattern of the task type; Based on the first similarity, abnormal resource usage behavior of the target computing task is detected, and the detection results are obtained.

2. The method as described in claim 1, wherein, Determining the first similarity between the resource indicator data and the resource usage behavior fingerprint model corresponding to the task type to which the target computing task belongs includes: Statistical feature extraction is performed on the same resource feature dimension in the resource indicator data to obtain the statistical feature vector of the target time window; Determine the second similarity between the statistical feature vector and the resource usage behavior fingerprint model; The first similarity is determined based on the second similarity.

3. The method as described in claim 2, wherein, The resource usage behavior fingerprint model includes multiple behavior fingerprint templates, one of which is used to characterize a resource usage pattern of the task type. Determining the second similarity between the statistical feature vector and the resource usage behavior fingerprint model includes: Determine the behavioral similarity between the statistical feature vector and the mean vector in the behavioral fingerprint template; wherein the mean vector is used to describe the resource usage pattern; The behavior similarity with the highest value among the behavior similarities corresponding to multiple behavior fingerprint templates is determined as the second similarity.

4. The method of claim 1, wherein, The step of detecting abnormal resource usage behavior of the target computing task based on the first similarity and obtaining the detection result includes: The detection result is determined based on at least one of the following: The magnitude relationship between the first similarity and the first threshold corresponding to the task type; The average of the first similarity and the third similarity; wherein the third similarity is the similarity of the time windows adjacent to the target time window, and the similarity is used to characterize the degree of similarity between the resource indicator data of the adjacent time windows and the resource usage behavior fingerprint model; The change in key indicators when the target computing task exhibits unauthorized resource occupation behavior; wherein, the key indicators are resource indicators associated with the unauthorized resource occupation behavior; The contribution of the key indicators to the first similarity when the target computing task exhibits unauthorized resource occupation behavior; When the target computing task exhibits unauthorized resource occupation behavior, the correlation of similarity between adjacent time windows is mentioned; wherein, the corresponding similarity is the similarity between the resource indicator data of each time window in the adjacent time windows and the resource usage behavior fingerprint model.

5. As described in claim 4, wherein, The determination of the detection result based on at least one of the following includes: In response to the size relationship indicating that the first similarity is greater than or equal to the first threshold, the detection result is generated to indicate that there is no abnormal resource usage behavior in the target time window; In response to the size relationship indicating that the first similarity is less than the first threshold, the abnormality level of the abnormal resource usage behavior is determined based on the mean. The type of resource occupation to which the unauthorized resource occupation behavior belongs is determined based on at least one of the change amount, the contribution degree, and the correlation. Based on at least one of the occupancy type and the anomaly level, a detection result is generated to indicate the presence of the abnormal resource usage behavior within the target time window.

6. The method of claim 4, wherein, The determination of unauthorized resource usage in the target computing task was made using the following method: Based on the feature contribution vector, the feature contribution variance is determined; wherein, the feature contribution vector includes the contribution of resource indicators under each resource feature dimension to the first similarity; An anomaly score is determined based on the first similarity, the variance of the feature contribution, and the rate of change of similarity. In response to the abnormal score being greater than the second threshold, the target computing task is determined to have unauthorized resource occupation behavior based on at least one of the resource usage behavior categories of multiple consecutive time windows and the feature contribution vector. The similarity change rate is used to characterize the trend of the similarity between resource indicator data and the resource usage behavior fingerprint model over time in multiple consecutive time windows, and the anomaly score is used to characterize the degree of anomaly in the abnormal resource usage behavior.

7. The method of claim 4, wherein, The first threshold is determined in the following manner: Determine the first average value and standard deviation of the similarity between resource indicator data of multiple time windows during the operation of historical computing tasks and the resource usage behavior fingerprint model; wherein, the historical computing tasks and the target computing task have the same task type; The first threshold is determined based on the first average value and the standard deviation.

8. The method of claim 4, further comprising: Determine the second average value of the similarity between resource indicator data of multiple time windows during the operation of the target computing task and the resource usage behavior fingerprint model; The first threshold is adjusted based on the second average value to obtain the adjusted first threshold.

9. The method of claim 1, wherein, The resource usage behavior fingerprint model includes a behavior fingerprint template, and each behavior fingerprint template includes a semantic behavior description. The step of detecting abnormal resource usage behavior in the target computing task based on the first similarity and obtaining the detection result includes: Obtain the semantic feature vector of the target computation task; Determine the fourth similarity between the semantic feature vector and the semantic behavior description; The target similarity is determined based on the first similarity and the fourth similarity. Based on the target similarity, abnormal resource usage behavior of the target computing task is detected, and the detection result is obtained.

10. The method according to any one of claims 1-9, wherein, The detection results include the anomaly level of the abnormal resource usage behavior of the target computing task and the type of resource usage behavior to which the unauthorized resource usage behavior belongs. The method further includes: Based on the task type, the anomaly level, and the occupancy type, determine the corresponding response strategy; Execute the aforementioned response strategy and obtain the response result.

11. The method of claim 10, further comprising: Based on the detection results and the response results, an alarm message for abnormal resource usage behavior of the target computing task is generated; Determine the alarm level based on the aforementioned anomaly level; The alarm information is sent to the target object using the alarm strategy corresponding to the alarm level. Obtain the feedback result of the target object to the alarm information.

12. The method of claim 11, further comprising: Based on the feedback results and the detection results, the first threshold is updated to obtain the updated first threshold; wherein, the first threshold is used to determine whether the resource usage behavior within the target time window is abnormal.

13. A method for constructing a resource usage behavior fingerprint model, comprising: Resource indicator data for multiple time windows are obtained; wherein, the resource indicator data includes resource indicators at each sampling time point within the corresponding time window obtained during the execution of historical computing tasks; Statistical features are extracted from the same resource feature dimension in the resource indicator data to obtain the statistical feature vector of the corresponding time window; Clusters are obtained by clustering statistical feature vectors of multiple time windows, and behavioral fingerprint templates of the clusters are extracted; wherein, one of the behavioral fingerprint templates is used to characterize a resource usage pattern. Based on the behavioral fingerprint templates of each cluster, a resource usage behavior fingerprint model corresponding to the target task type to which the historical computing task belongs is determined; wherein, the resource usage behavior fingerprint model is used to characterize the resource usage pattern of the target task type.

14. The method of claim 13, wherein, The step of obtaining clusters by clustering statistical feature vectors of multiple time windows includes: Obtain the semantic feature vector of the historical computing task; wherein the semantic feature vector is used to indicate the attribute information of the historical computing task; The semantic feature vector and the statistical feature vector of the time window are fused to obtain the joint feature vector of the time window; Clustering is performed on the joint feature vectors of multiple time windows to obtain the cluster.

15. The method of claim 14, wherein, The extraction of the behavioral fingerprint template of the cluster includes: Statistical feature extraction is performed on the statistical feature vectors in each joint feature vector of the cluster to obtain the resource behavior template of the cluster; Semantic category statistics are performed on the semantic feature vectors in each joint feature vector of the cluster to obtain the semantic behavior description of the cluster; The resource behavior template and semantic behavior description of the cluster are used as a behavior fingerprint template.

16. The method of claim 15, wherein, The step of extracting statistical features from the statistical feature vectors of each joint feature vector in the cluster to obtain the resource behavior template of the cluster includes: The mean and variance of the same statistical feature dimension are calculated for the statistical feature vectors in the cluster to obtain the mean and variance of each statistical feature dimension. A mean vector is obtained based on the mean of each of the statistical feature dimensions; wherein the mean vector is used to describe the resource usage pattern. Based on the variance of each of the statistical feature dimensions, a variance vector is obtained; wherein, the variance vector is used to characterize the fluctuation range of the resource usage pattern; The resource behavior template is obtained based on the mean vector and the variance vector.

17. The method of claim 14, wherein, The process of fusing the semantic feature vector with the statistical feature vector of the time window to obtain the joint feature vector of the time window includes: The statistical feature vectors of the multiple time windows are standardized to obtain the standardized feature vectors of the multiple time windows; The semantic feature vector is concatenated with the standardized feature vector of the time window to obtain the joint feature vector.

18. The method of claim 13, wherein, The computation process to which the historical computation task belongs includes computation tasks of multiple task types, and the target task type is determined in the following way: Obtain the statistical feature vectors corresponding to two adjacent time windows in the calculation process; Based on the statistical feature vectors corresponding to the two adjacent time windows, the resource feature change rate is determined; wherein, the resource feature change rate is used to characterize the degree of change of resource features between adjacent time windows; The target task type is determined based on the relationship between the resource characteristic change rate and the type switching threshold.

19. The method of any one of claims 13-18, further comprising: Obtain the statistical feature vector corresponding to any time window during the execution of the target computing task; wherein, the target computing task and the historical computing task have the same task type; Determine the feature similarity between the statistical feature vector corresponding to any time window and any behavioral fingerprint template; In response to the feature similarity being less than the anomaly threshold corresponding to the target task type, the resource usage behavior fingerprint model is updated based on resource indicator data from multiple time windows obtained during the execution of the target computing task.

20. A resource usage behavior anomaly detection device, comprising: The first acquisition module is used to acquire resource indicator data for a target time window; wherein, the resource indicator data includes resource indicators at each sampling time point within the target time window acquired during the execution of the target computing task. The first determining module is used to determine the first similarity between the resource indicator data and the resource usage behavior fingerprint model corresponding to the task type to which the target computing task belongs; wherein, the resource usage behavior fingerprint model is used to characterize the resource usage pattern of the task type; The detection module is used to detect abnormal resource usage behavior of the target computing task based on the first similarity and obtain the detection result.

21. An apparatus for constructing a resource usage behavior fingerprint model, comprising: The first acquisition module is used to acquire resource indicator data for multiple time windows; wherein, the resource indicator data includes resource indicators at each sampling time point within the corresponding time window acquired during the operation of historical computing tasks; The first extraction module is used to extract statistical features from the same resource feature dimension in the resource indicator data to obtain the statistical feature vector of the corresponding time window. The second extraction module is used to obtain clusters obtained by clustering statistical feature vectors of multiple time windows, and to extract behavioral fingerprint templates of the clusters; wherein, one of the behavioral fingerprint templates is used to characterize a resource usage pattern. The first determining module is used to determine the resource usage behavior fingerprint model corresponding to the target task type to which the historical computing task belongs, based on the behavior fingerprint template of each cluster; wherein, the resource usage behavior fingerprint model is used to characterize the resource usage pattern of the target task type.

22. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-19.

23. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-19.

24. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-19.

Images