Anomaly detection method and apparatus

Abstract

This application provides an anomaly detection method and apparatus, relating to the field of computer technology, which can improve the efficiency of container instance detection. The method includes: training a first model using historical data from N container instances in a virtual machine, wherein the N container instances are related to a first service executed by the virtual machine, and N is an integer greater than or equal to 1; and performing anomaly detection on a first container instance using the first model, wherein the first container instance is related to the first service.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an anomaly detection method and apparatus. Background Technology

[0002] Containers are a virtualization technology used in computer operating systems for service-oriented applications in cloud computing. They provide processes with relatively independent and isolated runtime environments, thereby enhancing software portability and security, and improving system resource utilization. Container technology is widely used.

[0003] Current technologies mostly focus on anomaly detection for specific metrics, and there are no solutions for anomaly detection for container instances. For example, by training a model based on data from an application programming interface (API) over a period of time, the trained model can be used to predict whether the API will experience anomalies.

[0004] During the upgrade process of a microservice container, after obtaining the upgrade package, the current container is killed, and a new container is created to run the microservice-related processes. This instability of containers makes it impossible to train or predict models for container instances. For example, if a model is trained based on historical data of container instance A, and then used for prediction, container A may have already been killed during the upgrade process, rendering the trained model unusable for detecting newly added container instances during the upgrade. For newly appearing container instances (e.g., those added during scaling or upgrades), it requires a long period of data accumulation to train a model before it can potentially detect container anomalies. In short, container instance detection is inefficient. Summary of the Invention

[0005] This application provides an anomaly detection method and apparatus that can improve the efficiency of container instance detection.

[0006] In a first aspect, an anomaly detection method is provided, comprising: training a first model using historical data of N container instances in a virtual machine, wherein the N container instances are related to a first service executed by the virtual machine, and N is an integer greater than or equal to 1; and performing anomaly detection on a first container instance using the first model, wherein the first container instance is related to the first service.

[0007] In this application, during model training, all container instances related to the same business (e.g., a microservice) are abstracted into a single container instance; that is, historical data generated by all container instances are used to train a single model. After the model is trained, independent anomaly detection is performed on each container instance, ensuring that anomalies of each container instance are reported individually. This improves the efficiency of container instance detection. For container instances added during scaling or upgrades, there is no need to spend time accumulating data, allowing for rapid anomaly detection. Furthermore, container instances added during upgrades will not enter an infinite loop waiting for data, thus avoiding the problem of undetectable anomalies.

[0008] In conjunction with the first aspect, in the first possible implementation of the first aspect, the first container instance is the updated container instance of the second container instance among the N container instances, or the first container instance is a newly created container instance of the virtual machine; or the first container instance is any one of the N container instances.

[0009] In this application, model training and model prediction are decoupled. Even if the historical data of container instances are not used for model training, as long as the container instances are related to the same business, the trained model can be used for anomaly detection. This includes container instances added during expansion or newly generated container instances during container upgrades. For these newly added container instances, there is no need to accumulate data for a long time to train the model, which improves the efficiency of container instance anomaly detection.

[0010] In a second possible implementation of the first aspect, in combination with any of the first aspects, the method further includes: marking the historical data of the N container instances as the historical data of the target container instance.

[0011] In this application, during model training, historical data of different container instances can be marked as historical data of the same container instance, which can better identify the data used to train the first model. It can also classify the features of different container instances related to the same business into the same model, so as to use the trained model to perform anomaly detection on container instances related to the same business.

[0012] In conjunction with any of the first aspects or the above possible implementations of the first aspect, in the third possible implementation of the first aspect, marking the historical data of the N container instances as the historical data of the same container instance includes: marking the historical data of the N container instances at the first moment of the first period as the historical data of the target container instance at the first moment of N second periods; the time length of the first period is greater than the time length of the second period.

[0013] In this application, the identification period of the indicator can be used to mark the data of different container instances at the same time as the data of the same container instance in different identification periods, without affecting the characteristics of the data itself.

[0014] In a fourth possible implementation of the first aspect, in combination with any of the first aspects above, the step of training the first model using historical data from N container instances in the virtual machine includes: determining abnormal state information of M features of the first model based on the historical data of the N container instances and the hyperparameters of the first model; the abnormal state information is used to detect whether the features corresponding to the abnormal state information are abnormal.

[0015] This application provides a specific implementation for anomaly detection of container instances, including: determining the anomaly state information of M features of a first model in order to determine whether the features represented by the real-time data of the container instance are abnormal. Thus, the anomaly of the container instance is determined based on whether the features are abnormal.

[0016] In a fifth possible implementation of the first aspect, in combination with any of the first aspects, the step of using the first model to perform anomaly detection on the first container instance includes: when any one of the M features is detected to be abnormal, then determining that the container instance related to the first service is abnormal.

[0017] This application provides a specific implementation for determining whether a container instance is abnormal using the features of the model.

[0018] In conjunction with any possible implementation of the first aspect or the above-mentioned first aspect, in the fourth and second possible implementation of the first aspect, the abnormal state information of the M features includes one or more of the following: the numerical threshold of the relevant indicators of the N container instances, the abnormality threshold of the relevant indicators, and the interface access frequency threshold of the N container instances; wherein, the numerical threshold of the relevant indicators may be the upper limit and / or lower limit of the numerical value of the relevant indicators, the abnormality threshold of the relevant indicators may be the upper limit and / or lower limit of the variation range of the numerical value of the relevant indicators, and the interface access frequency threshold of the N container instances may be the upper limit and / or lower limit of the interface access frequency of the N container instances.

[0019] This application provides a specific implementation of abnormal state information with M features.

[0020] In a second aspect, an apparatus is provided, comprising: a training unit for training a first model using historical data from N container instances in a virtual machine, wherein the N container instances are related to a first service executed by the virtual machine, and N is an integer greater than or equal to 1; and a detection unit for performing anomaly detection on a first container instance using the first model, wherein the first container instance is related to the first service.

[0021] In conjunction with the second aspect, in the first possible implementation of the second aspect, the first container instance is the updated container instance of the second container instance among the N container instances, or the first container instance is a newly created container instance of the virtual machine; or the first container instance is any one of the N container instances.

[0022] In conjunction with any possible implementation of the second aspect or the above second aspect, in the second possible implementation of the second aspect, the training unit is further configured to mark the historical data of the N container instances as the historical data of the target container instance.

[0023] In a third possible implementation of the second aspect, in conjunction with any of the second aspects above, the historical data of the N container instances at the first moment of the first period are marked as the historical data of the target container instance at the first moment of the N second periods; the duration of the first period is greater than the duration of the second period.

[0024] In a fourth possible implementation of the second aspect, in conjunction with any of the second aspects above, the training unit is specifically used to determine the abnormal state information of M features of the first model based on the historical data of the N container instances and the hyperparameters of the first model; the abnormal state information is used to detect whether the features corresponding to the abnormal state information are abnormal.

[0025] In a fifth possible implementation of the second aspect, in conjunction with any of the second aspects above, the detection unit is configured to determine that the container instance related to the first service is abnormal when any one of the M features is detected to be abnormal.

[0026] In combination with any possible implementation of the second aspect or the above second aspect, in the sixth possible implementation of the second aspect, the abnormal state information of the M features includes one or more of the following: the numerical threshold of the relevant indicators of the N container instances, the abnormality threshold of the relevant indicators, and the interface access frequency threshold of the N container instances.

[0027] Wherein, the numerical threshold of the relevant indicator can be the upper limit and / or lower limit of the value of the relevant indicator, the anomaly threshold of the relevant indicator can be the upper limit and / or lower limit of the change range of the value of the relevant indicator, and the interface access frequency threshold of the N container instances can be the upper limit and / or lower limit of the interface access frequency of the N container instances.

[0028] Thirdly, an apparatus is provided, comprising at least one processor and a memory, wherein the at least one processor is coupled to the memory; the memory is used to store a computer program;

[0029] The at least one processor is configured to execute a computer program stored in the memory, such that the apparatus performs the method as described in the first aspect and any implementation thereof.

[0030] Fourthly, a computer-readable storage medium is provided, comprising: instructions stored in the computer-readable storage medium; and, when the computer-readable storage medium is operated on an apparatus according to the second aspect and any implementation thereof, causing the apparatus to perform the communication method according to the first aspect and any implementation thereof.

[0031] Fifthly, a wireless device is provided, comprising a processor, for example, applied in the device, for implementing the method described in the first aspect and any implementation thereof, the device being, for example, a chip system. In one feasible implementation, the chip system further comprises a memory for storing program instructions and data necessary for implementing the functions of the method described in the first aspect.

[0032] The chip system mentioned above can be a system on chip (SOC) or a baseband chip, etc. The baseband chip can include processors, channel encoders, digital signal processors, modems and interface modules, etc. Attached Figure Description

[0033] Figure 1 A schematic diagram of the network system provided in the embodiments of this application;

[0034] Figure 2 This is a schematic diagram illustrating the container instance upgrade provided in an embodiment of this application;

[0035] Figure 3 A schematic diagram illustrating the model training principle provided in the embodiments of this application;

[0036] Figure 4 Structural block diagram of the device provided in the embodiments of this application;

[0037] Figure 5A flowchart illustrating the anomaly detection method provided in this application embodiment;

[0038] Figure 6 A flowchart of the model training process provided in this application embodiment;

[0039] Figure 7 This is a data conversion diagram provided for an embodiment of this application;

[0040] Figure 8 This is a schematic diagram illustrating the model training and prediction decoupling provided in the embodiments of this application;

[0041] Figure 9 This is a schematic diagram of container instance anomaly detection provided in an embodiment of this application;

[0042] Figure 10 Another schematic diagram of the communication method provided in the embodiments of this application;

[0043] Figure 11 Another structural block diagram of the device provided in the embodiments of this application. Detailed Implementation

[0044] Figure 1 A schematic diagram of a network system to which the technical solution provided in this application applies is given. (Reference) Figure 1 The system includes a physical machine 10, one or more virtual machines (only virtual machines 20, 30 and 40 are shown in the figure) running on the operating system of the physical machine 10, and a monitoring server 50.

[0045] Physical machine 10 is responsible for managing and allocating hardware resources, presenting a virtual hardware platform for virtual machines, such as providing virtual CPUs, memory, virtual disks, virtual network cards, etc. Containers can be created within virtual machines, providing relatively independent and isolated runtime environments for processes. For example, container 1 supports the execution of process 1, and container 2 supports the execution of process 2. Monitoring server 50 is used to monitor the container instances running in the virtual machines, such as monitoring the interface call latency, interface call success rate, failure rate, and download speed of the container instances.

[0046] First, the terminology used in the embodiments of this application will be explained:

[0047] (1) Virtual machine: refers to one or more virtual computers simulated on a physical computer. These virtual machines can work like real physical computers.

[0048] (2) Process: is the entity that executes instructions. Processes can be used to run programs and execute various instructions.

[0049] (3) Instance: Includes multiple processes, which can be a collection of all processes executing a project. For example, if a user needs to access file 1 as a project, responding to the user's access command requires the execution of process 1, process 2, and process 3. The collection of process 1, process 2, and process 3 can be called an instance.

[0050] (4) Container: Used to provide a relatively independent and isolated running environment for a process. For example, a container contains an independent file system, namespace, resource view, etc. Container instance: After a process runs in the environment provided by a container, the container can be called a container instance.

[0051] During the container upgrade process, after obtaining the upgrade image file (i.e. the upgrade package), the current container is killed, and a new container is created in the corresponding virtual machine. The relevant processes are then run in the newly created container.

[0052] Example, reference Figure 2 The microservice is implemented using two containers, A and B. Historical data from container instance A can be collected and used to train a model (Model 1). Assuming container A has not been upgraded, this model can be used to predict whether container instance A will experience anomalies. Alternatively, historical data from container instance B can be collected and used to train a model (Model 2). Assuming container B has not been upgraded, this model can be used to predict whether container instance B will experience anomalies.

[0053] However, if the container is upgraded before the model is trained, the trained model cannot be used to detect whether container instances are abnormal. See the example below. Figure 2 During the upgrade process of container instance A, container instance A is destroyed and upgraded to container instance C. Model 1 cannot be used to predict whether container instance A is abnormal. It is necessary to wait for container instance C to accumulate enough historical data before training the model for anomaly detection of container instance C.

[0054] During the upgrade process of container instance B, container instance B is destroyed and upgraded to container instance D. Model 2 cannot be used to predict whether container instance B is abnormal. It is necessary to wait for container instance D to accumulate enough historical data before training the model for anomaly detection of container instance D.

[0055] In real-world development scenarios, container instances have short lifecycles, which can lead to an endless wait for historical training data, making it impossible to detect anomalies in container instances.

[0056] In the method provided in this application embodiment, during training, all container instances related to the same business (e.g., a microservice) are abstracted into a single container instance; that is, historical data generated by all container instances are used to train a model. After the model is trained, independent anomaly detection is performed on the container instances to ensure that anomalies of each container instance are reported separately, thus solving the problem of not being able to perform anomaly detection on a container instance basis.

[0057] Furthermore, for newly added container instances in this business expansion scenario, even if the newly added container itself has no historical data, there is no need to accumulate historical data; the pre-trained model can be used to quickly perform anomaly detection.

[0058] Example, reference Figure 3 Container instances are identified by the IP addresses they access. The container instances associated with microservice 1 are 192.168.0.1, 192.168.0.2, 192.168.0.3, and 192.168.0.4. Historical data from these four container instances is collected for model training, and these four container instances are abstracted into a single container instance. The trained model can individually detect whether container instance 192.168.0.1, 192.168.0.2, 192.168.0.3, or 192.168.0.4 is abnormal.

[0059] Figure 4 The diagram shown is a hardware structure schematic of the device 40 provided in an embodiment of this application. The device 40 can be deployed in... Figure 1 The device in the system shown can also be Figure 1 The devices in the system shown, for example, device 40, are data acquisition device 10, data receiving device 20, controller 30, or data analysis device 40. (See reference) Figure 4 The device 40 includes a processor 401, a memory 402, and at least one network interface. Figure 4 (This is merely an example illustration using network interface 403 as an example only.) The processor 401, memory 402, and network interface 403 are interconnected.

[0060] Processor 401 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of programs according to the present application.

[0061] Network interface 403 is the interface of device 40 for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0062] Memory 402 may be a read-only memory (ROM) or other type of static data center capable of storing static information and instructions, random access memory (RAM) or other type of dynamic data center capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic data center, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory may exist independently and be connected to the processor via communication lines. Memory may also be integrated with the processor.

[0063] The memory 402 stores computer execution instructions for implementing the scheme of this application, and the processor 401 controls the execution. The processor 401 executes the computer execution instructions stored in the memory 402 to implement the intent processing method provided in the following embodiments of this application.

[0064] Optionally, the computer execution instructions in the embodiments of this application may also be referred to as application code, and the embodiments of this application do not specifically limit this.

[0065] In a specific implementation, as one example, processor 401 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 in the CPU.

[0066] In a specific implementation, as one embodiment, device 40 may include multiple processors, for example... Figure 4 Processors 401 and 404 are specified in the text. Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor here may refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0067] The aforementioned device 40 can be a general-purpose device or a special-purpose device. In specific implementations, device 40 can be a desktop computer, a network device, an embedded device, or other device with... Figure 4 Devices with similar structures. The types of devices 40 are not limited to those described in this application.

[0068] This application provides an anomaly detection method, such as... Figure 5 As shown, applied to Figure 1 The monitoring server 50 in the system shown includes the following steps in the method:

[0069] 501. Train a first model using historical data from N container instances in a virtual machine, wherein the N container instances are related to the first business executed by the virtual machine.

[0070] The first service can be a service provided by a virtual machine, such as a microservice provided by the virtual machine. For example, a service related to a certain application can be a microservice, or a service related to certain functions of an application can be a microservice. This application embodiment does not impose any restrictions on this. The container instance related to the first service can be considered as a container instance used to implement (or execute) the first service.

[0071] The N container instances can be all container instances related to the first service within a certain period prior to the current time, or they can be a subset of container instances related to the first service within the same period prior to the current time. N is an integer greater than or equal to 1. The current time is the time at which step 501 is executed. The length of the time period can be set by the monitoring server 50 or stored locally on the monitoring server 50.

[0072] Historical data of a container instance can be data generated by the container instance over a period of time prior to the current moment. This data can be business-related data generated by the container instance's historical behavior, such as access latency, processing latency, input / output (I / O) latency, etc.

[0073] In one possible implementation, when training a model using historical data from multiple container instances related to the same business, these container instances are abstracted into a single current container instance, and then the model is trained using this data. For example, the historical data of the N container instances can be labeled as the historical data of the target container instance.

[0074] In a specific implementation, marking the historical data of the N container instances as the historical data of the target container instance can be achieved by marking the historical data of the N container instances with the identification information of the target container instance. The target container instance can be any one of the N container instances, or it can be an abstract, non-existent container instance. The identification information of the container instance can be the IP address accessed by the container instance. When the target container instance is an abstract, non-existent container instance, the identification information of the target container instance can be an abstract identifier, for example, a non-existent IP address.

[0075] Assuming the historical data of the N container instances is the historical data of each container instance at the first moment of the first period, this data can be marked as the historical data of the target container instance at the first moment of the N second periods. The duration of the first period is longer than the duration of the second period.

[0076] For example, the first period is "month" and the second period is "day". Assuming the container instances related to the first business include container instance 1, container instance 2, and container instance 3, the historical data of the N container instances can be the historical data of container instance 1, container instance 2, and container instance 3 within the past month (the month prior to the execution of step 501). For example, including the historical data of container instance 1 on the 3rd, the data of container instance 2 on the 4th, and the historical data of container instance 3 on the 5th, these data can be marked as three historical data entries for the target container instance (e.g., container instance x) on September 3rd, September 4th, and September 5th.

[0077] In one possible implementation, the specific implementation of training the first model using historical data from N container instances in the virtual machine includes: determining the abnormal state information of M features of the first model based on the historical data of the N container instances and the hyperparameters of the first model; the abnormal state information is used to detect whether the features corresponding to the abnormal state information are abnormal, and never judging whether the container instance is abnormal based on whether the features of the model are abnormal.

[0078] In one possible implementation, the abnormal state information of the M features includes one or more of the following: the numerical threshold of the relevant indicators of the N container instances, the abnormality threshold of the relevant indicators of the N container instances, and the interface access frequency threshold of the N container instances.

[0079] The numerical thresholds of the relevant metrics for the N container instances are used to characterize the normal state of the relevant metric values ​​for the N container instances; the numerical thresholds can be upper and / or lower limits of the values. For example, the relevant metric for the N container instances is the number of API interface accesses, and the numerical thresholds can be the upper or lower limit of the number of API interface accesses. The normal values ​​of the relevant metrics for the N container instances should not exceed the upper limit of the metric value and should not be lower than the lower limit of the metric value. When the number of API interface accesses exceeds the upper limit or falls below the lower limit, it indicates that the relevant metric is abnormal.

[0080] The anomaly threshold for the relevant metrics of the N container instances is used to characterize the normal range of change of the relevant metrics of the N container instances. The anomaly threshold can be the upper and / or lower limit of the normal range of change of the relevant metrics. For example, if the relevant metrics of the N container instances are API interface access counts (or API interface access frequency), the anomaly threshold can be the upper and / or lower limit of the normal range of change of the API interface access count. The normal range of change of the relevant metrics should not exceed the upper limit of the range of change indicated by the anomaly threshold, or, not be lower than the lower limit of the range of change indicated by the anomaly threshold. That is, under normal circumstances, the range of change of the values ​​of the relevant metrics will not be too large or too small. When the range of change of the values ​​of the relevant metrics is not too large but exceeds the anomaly threshold, it indicates that the relevant metrics are abnormal. Or, when the range of change of the values ​​of the relevant metrics is not too small but is lower than the anomaly threshold, it indicates that the relevant metrics are abnormal.

[0081] The interface access frequency thresholds for the N container instances are used to characterize the normal interface access frequency of the N container instances. The interface access frequency thresholds can be the upper and / or lower limits of the container instance interface access frequency. When the interface access frequency of a container instance becomes abnormal, an alarm can be issued to indicate that the container instance is abnormal. For example, when the interface access frequency of a container instance exceeds the upper limit or falls below the lower limit, an alarm can be issued to indicate that the container instance is abnormal.

[0082] In one possible implementation, if a container instance has exhibited zero-dropping ("0") behavior over a period of time (i.e., low interface access frequency, such as below the lower limit of interface access frequency), no alert may be issued when the container instance subsequently exhibits zero-dropping behavior again. Specifically, the presence of zero-dropping behavior in the past can be determined based on the container instance's past interface access frequency.

[0083] For example, the interface access frequency can be the proportion of a window with zero requests within a sliding window. Taking time X as an example, the sliding window can be a window of length m minutes before time X, with one minute as the monitoring window length, to determine whether zero-request behavior occurs within one minute. Assuming that zero-request behavior occurs in n minutes within the sliding window (i.e., a window of length m minutes before time X), then the interface access frequency can be n / m.

[0084] When the interface access frequency is lower than a threshold (e.g., lower than the lower limit of the interface access frequency), it can be assumed that the container instance has had zero-dropping behavior in the past. When the container instance is detected to have zero-dropping behavior again in the future (e.g., after time X), one possible action is not to issue an alarm.

[0085] 502. Use the first model to perform anomaly detection on the first container instance, where the first container instance is related to the first service.

[0086] In specific implementation, the first model trained in step 501 can be used for anomaly detection of any container instance related to the first service. For example, it can be used to detect anomalies in newly added container instances during scaling scenarios. For instance, the first container instance is a newly created container instance from the virtual machine, and this container instance is related to the first service. Alternatively, if the historical data of the container instance was used in model training, and the container instance disappears during the upgrade process, anomaly detection can be performed on the newly added container instance after the upgrade. For instance, the first container instance is the updated container instance of the second container instance among the N container instances. Of course, if the historical data of the container instance was not used in the training of the first model, but the container instance is related to the first service, the first model can be used to detect anomalies in newly added container instances after the upgrade of the container instance. Alternatively, if the historical data of the container instance was used in the training of the first model, the first model can be used to detect anomalies in the container instance subsequently. For instance, the first container instance is any one of the N container instances.

[0087] In one possible implementation, the anomaly detection of the first container instance using the first model includes: when any one of the M features is detected to be abnormal, the container instance related to the first service is determined to be abnormal. Specifically, data of the first container instance over a period of time is input into the first model; when any one of the M features is determined to be abnormal based on the input data, the first container instance is determined to be abnormal.

[0088] In one possible way, through Figure 6 The steps shown are used to train the historical data of the N container instances to obtain the first model. (Reference) Figure 6 Specifically, it includes the following four steps:

[0089] S1. Load historical data and model hyperparameters from N container instances.

[0090] The historical data of N container instances can be the data of N container instances over a period of time in the past, and the period of time to be obtained can be a period of time before the execution of step S1.

[0091] Model hyperparameters mainly include sensitivity and metric type. Sensitivity controls the model's anomaly alert sensitivity, i.e., the model's sensitivity in detecting anomalies. Higher sensitivity makes it easier to detect anomalies. Metric type refers to the type of metric used to train the container instance anomaly detection model, and can include request volume, latency, success rate, failure rate, download speed, etc.

[0092] S2. Clean historical data.

[0093] Cleaning historical data primarily involves filtering out anomalies. These anomalies can be data that doesn't match the typical characteristics of the metrics. For example, if the latency of calling interface A is approximately 100ms, and a historical data record shows a latency of 5s for calling interface A, then this data can be considered an anomaly and filtered out in step S2, preventing it from participating in model training.

[0094] S3. Determine the identification cycle of relevant indicators of the model.

[0095] Among them, model-related metrics are the metrics indicated by the metric type in the model hyperparameters. They can be metrics used to detect anomalies in the model. By detecting model-related metrics, it can be determined whether the model is abnormal.

[0096] In practice, the variance of the indicator data can be calculated over different periods. The smaller the variance, the better the indicator data matches the characteristics of the corresponding period. For example, the daily, weekly, and monthly variances of the indicator data can be calculated. Assuming the monthly variance is the smallest, it indicates that the changes in the indicator data within a month better match the characteristics of the indicator data. In this case, the identification period for the indicator data is "month".

[0097] S4. Determine the characteristics of the model.

[0098] The features of the model are used to detect whether the container instance is abnormal. For example, the data of the container instance is input into the model. If the model determines that the data of the container instance exceeds the threshold of the model's features, it will issue an alarm to indicate that the container instance is abnormal.

[0099] Taking the Boxplot algorithm as an example, the features of a model built using the Boxplot algorithm can be threshold values ​​for metrics, anomaly threshold values ​​for metrics, and interface access frequency threshold values ​​for container instances. These metrics can be model-related metrics.

[0100] Specifically, the data to be trained (e.g., historical data of the N container instances mentioned above over a period of time) can be aggregated, and the upper or lower limit of the indicator value can be determined based on a specified percentage. For example, 25% of the data is greater than q1, 25% of the data is less than q2, and the remaining data is between q1 and q2. Here, q1 and q2 are the upper and lower limits of the indicator value, respectively, and are both threshold values ​​for the indicator value.

[0101] The interface access frequency threshold for a container instance can be the upper limit of the proportion of zero-request behavior that occurred within the container instance over a past period. If the interface access frequency exceeds the threshold—for example, if the number of requests with a value of 0 within a certain sliding window over a past period exceeds the interface access frequency threshold—then the container instance is considered to have engaged in zero-request behavior over the past period. Subsequent detection of zero-request behavior in this container instance does not require issuing an alert.

[0102] Anomaly thresholds for metrics are used to limit the degree of normal variation in metrics. When the length of a metric's change exceeds the threshold, it indicates that the metric's change is abnormal. For example, if the cumulative anomaly score over an n-minute window is higher than the upper limit or lower than the lower limit, the container instance is considered to be abnormal.

[0103] When performing anomaly detection, an alarm can be triggered if any of the following thresholds is abnormal: the threshold for the value of the metric, the threshold for the degree of anomaly of the metric, or the threshold for the frequency of interface access of the container instance. This indicates that an anomaly has occurred in the container instance.

[0104] The following describes the anomaly detection method provided in this application with specific examples, in which container instances can be marked using IP addresses (i.e., insranceIp). Reference Figure 7 On May 5, 2020 at 20:00:00, a microservice reported data for three container instances: container instance 172.16.0.0, container instance 172.16.0.1, and container instance 172.16.0.2.

[0105] When training the model, the historical data of the three container instances mentioned above are treated as the data of a single container instance for model training. Specifically, the historical data of the three container instances is loaded, the data is cleaned, and the cleaned data is used to train the model.

[0106] Specifically, historical data from different container instances can be virtualized as historical data for the same container instance based on the identification period of container instance-related metrics. For example, assuming that the identification period for model-related metrics for an independent container instance is one day, with each day being an independent period, and the characteristic curve of the same metric being roughly the same on each day, the data from different container instances at the same time can be equated to the data of the same container instance on different days. This allows the historical data from different container instances to be used as the historical data of the same container instance for model training, thereby fusing the model features from different instances into a single model. (Reference) Figure 7 The three data entries for 172.16.0.0, container instance 172.16.0.1, and container instance 172.16.0.2 at 2020-05-05-20:00:00 are virtualized as three data entries for container instance 127.0.0.1 at 2020-05-05-20:00:00, 2020-05-06-20:00:00, and 2020-05-07-20:00:00. Among them, 127.0.0.1 is an abstracted, non-existent IP address used to identify an abstracted container instance.

[0107] refer to Figure 8 In the process of model prediction (i.e., using the model to detect anomalies in container instances), the previously trained model (e.g., the first model) is loaded, and the real-time data of the container instances is input into the model for anomaly detection. The container instances used for anomaly detection are decoupled from the container instances used for training the parameter model. That is, the container instances used for training the model and the container instances used for prediction can be different. For example, the first model is trained based on the historical data of container instances A, B, and C related to microservice 1, and then anomaly detection is performed on container instances A, B, and C separately, generating alarms A, B, and C respectively.

[0108] It should be noted that when performing anomaly detection, instances within the time frame requiring prediction are detected individually. (Reference) Figure 9 During model training, there were three container instances associated with the same microservice: 172.16.0.0, 172.16.0.1, and 172.16.0.2. However, container instance 172.16.0.2 was destroyed at 20:03:00, and a new container instance, 172.16.0.3, was created. Anomaly detection was performed separately for each of these three microservices.

[0109] Although 172.16.0.3 is a newly added container instance, according to the method provided in the embodiments of this application, this container instance can participate in anomaly detection very shortly after its creation, without the need to re-accumulate data and retrain the model for this container instance 172.16.0.3.

[0110] When dividing each function into modules according to its corresponding function. Figure 10 A possible structural schematic diagram of the device involved in the above embodiments is shown. Figure 10 The device shown can be the monitoring server described in the embodiments of this application, a component within the monitoring server implementing the above method, or a chip applied in the monitoring server. The chip can be a system-on-a-chip (SOC) or a baseband chip with communication functions, etc. Figure 10 As shown, the apparatus includes a training unit 1001 and a detection unit 1002. The training unit 1001 is used to support the monitoring server in performing step 501 and / or other processes used in the techniques described herein.

[0111] The detection unit 1002 is used to support the monitoring server in performing step 502 and / or other processes used in the techniques described herein.

[0112] It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0113] like Figure 11 As shown, the device may also include a storage unit 1003 for storing the device's program code and / or data.

[0114] This application provides a computer-readable storage medium storing instructions; the instructions are used to perform actions such as... Figure 5 The method shown.

[0115] This application provides a computer program product including instructions that, when run on a device, cause the device to perform actions such as... Figure 5 The method shown.

[0116] This application provides a wireless device, comprising: the wireless device storing instructions; when the wireless device is in... Figure 4 , Figure 10 , Figure 11 When the device is operated as shown, the device performs the following actions: Figure 5 The method is shown. The device can be a chip.

[0117] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0118] The processor in this application embodiment may include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, etc., which are various computing devices that run software. Each computing device may include one or more cores for executing software instructions to perform calculations or processing. The processor may be a separate semiconductor chip or integrated with other circuits into a single semiconductor chip. For example, it may be integrated with other circuits (such as encoding / decoding circuits, hardware acceleration circuits, or various bus and interface circuits) to form a SoC (System-on-a-Chip), or it may be integrated as a built-in processor within an ASIC. The ASIC with the integrated processor may be packaged separately or together with other circuits. In addition to the cores for executing software instructions to perform calculations or processing, the processor may further include necessary hardware accelerators, such as field-programmable gate arrays (FPGAs), PLDs (programmable logic devices), or logic circuits that implement dedicated logic operations.

[0119] The memory in the embodiments of this application may include at least one of the following types: read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions; random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions; or electrically erasable programmable-only memory (EEPROM). In some scenarios, the memory may also be a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto.

[0120] In this application, "at least one" refers to one or more. "More than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, or B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple. In addition, to facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art will understand that the words "first" and "second" do not limit the quantity or the order of execution, and that the words "first" and "second" do not necessarily imply that they are different.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed database access apparatus and method can be implemented in other ways. For example, the database access apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection of the database access apparatus or unit may be electrical, mechanical, or other forms.

[0122] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0123] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0124] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0125] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An anomaly detection method characterized by, include: The historical data of N container instances in the virtual machine are marked as the historical data of the target container instance. The N container instances are related to the first business executed by the virtual machine, and N is an integer greater than or equal to 1. The first model is trained using historical data from the target container instance; The first model is used to perform anomaly detection on the first container instance, which is related to the first service.

2. The method of claim 1, wherein, The first container instance is the updated container instance of the second container instance among the N container instances, or... The first container instance is a newly created container instance of the virtual machine; or, The first container instance is any one of the N container instances.

3. The method of claim 1, wherein, The step of marking the historical data of the N container instances as the historical data of the target container instance includes: The historical data of the N container instances at the first moment of the first period are marked as the historical data of the target container instance at the first moment of the N second periods; the time length of the first period is greater than the time length of the second period.

4. The method according to any one of claims 1 to 3, characterized in that, The step of training the first model using historical data from the target container instance includes: Based on the historical data of the target container instance and the hyperparameters of the first model, the abnormal state information of M features of the first model is determined; the abnormal state information is used to detect whether the feature corresponding to the abnormal state information is abnormal.

5. The method of claim 4, wherein, The step of using the first model to perform anomaly detection on the first container instance includes: If any one of the M features is found to be abnormal, then the container instance related to the first service is determined to be abnormal.

6. The method of claim 4, wherein, The abnormal state information of the M features includes one or more of the following: The numerical threshold values ​​of the relevant metrics for the N container instances, the anomaly threshold values ​​of the relevant metrics, and the interface access frequency threshold values ​​for the N container instances. Wherein, the numerical threshold of the relevant indicator is the upper and / or lower limit of the value of the relevant indicator, the anomaly threshold of the relevant indicator is the upper and / or lower limit of the change range of the value of the relevant indicator, and the interface access frequency threshold of the N container instances is the upper and / or lower limit of the interface access frequency of the N container instances.

7. An apparatus, characterized in that, include: The training unit is used to mark the historical data of N container instances in the virtual machine as the historical data of the target container instance, wherein the N container instances are related to the first business executed by the virtual machine, and N is an integer greater than or equal to 1; The training unit is also used to train a first model using historical data from the target container instance; The detection unit is used to perform anomaly detection on the first container instance using the first model, wherein the first container instance is related to the first service.

8. The apparatus according to claim 7, characterized in that, The first container instance is the updated container instance of the second container instance among the N container instances, or... The first container instance is a newly created container instance of the virtual machine; or, The first container instance is any one of the N container instances.

9. The apparatus according to claim 7, characterized in that, The training unit is specifically used to mark the historical data of the N container instances at the first moment of the first period as the historical data of the target container instance at the first moment of the N second periods; the time length of the first period is greater than the time length of the second period.

10. The apparatus according to any one of claims 7-9, characterized in that, The training unit is specifically used to determine the abnormal state information of M features of the first model based on the historical data of the target container instance and the hyperparameters of the first model; the abnormal state information is used to detect whether the features corresponding to the abnormal state information are abnormal.

11. The apparatus according to claim 10, characterized in that, The detection unit is used to determine that the container instance related to the first service is abnormal when any one of the M features is detected to be abnormal.

12. The apparatus according to claim 10, characterized in that, The abnormal state information of the M features includes one or more of the following: The numerical threshold values ​​of the relevant metrics for the N container instances, the anomaly threshold values ​​of the relevant metrics, and the interface access frequency threshold values ​​for the N container instances. Wherein, the numerical threshold of the relevant indicator is the upper and / or lower limit of the value of the relevant indicator, the anomaly threshold of the relevant indicator is the upper and / or lower limit of the change range of the value of the relevant indicator, and the interface access frequency threshold of the N container instances is the upper and / or lower limit of the interface access frequency of the N container instances.

13. An apparatus, characterized in that, Includes a processor, which is coupled to a memory; Memory, used to store computer programs; A processor for executing a computer program stored in the memory to cause the apparatus to perform the method as described in any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that, Includes a program or instructions that, when executed by a processor, cause the method described in any one of claims 1 to 6 to be performed.

15. A computer program product, characterized in that, The computer program product includes instructions that, when executed, cause the method as described in any one of claims 1 to 6 to be performed.

16. A chip, characterized in that, The chip includes a processor and an interface circuit, the interface circuit being coupled to the processor, the processor being configured to run a computer program or instructions such that the method described in any one of claims 1 to 6 is executed.

Images