A transaction identification method, device and equipment

Abstract

The embodiment of the specification discloses a kind of transaction identification method, device and equipment, for cluster, it is helpful to more reliable more in advance identify transaction, especially smaller transaction, to avoid small transaction degradation to cause greater failure, so as to improve the work reliability of cluster, also help to quickly locate the change involved in the service that can cause the transaction, to prevent this change from bringing transaction risk to more application server.Solution includes: collecting service call data in cluster, the cluster includes multiple servers;According to service name dimension, the service call data is aggregated, and the aggregated data corresponding to the multiple servers of different services is obtained;By comparing the aggregated data of the multiple servers, it is identified whether there is a server with transaction in the multiple servers.

Description

Technical Field

[0001] This manual relates to the field of operation and maintenance technology, and in particular to an anomaly identification method, device, and equipment. Background Technology

[0002] With the rapid development of internet technology and the widespread use of smart terminals, many businesses can rely on applications on smart terminals. These applications have a large user base, so the corresponding application servers need to have powerful processing capabilities.

[0003] Currently, services are typically provided in the form of clusters, which contain multiple (usually large-scale) application servers. Monitoring the operational status of these clusters is necessary. Existing monitoring systems are cluster-wide and lack sufficient sensitivity; minor anomalies are often buried in the cluster data, averaged out, or masked. Therefore, it's difficult to detect small anomalies early on; they are only detected when a major anomaly occurs in the cluster, by which time the fault has often already occurred.

[0004] Therefore, for clusters, there is a need for solutions that can help identify anomalies more reliably and earlier to avoid failures. Summary of the Invention

[0005] This specification provides one or more embodiments of anomaly identification method, apparatus, device, and storage medium to solve the following technical problem: for clusters, there is a need for a solution that helps to identify anomalies more reliably and earlier to avoid failures.

[0006] To solve the above-mentioned technical problems, one or more embodiments of this specification are implemented as follows:

[0007] This specification provides one or more embodiments of an anomaly identification method, including:

[0008] Collect service call data from a cluster containing multiple servers;

[0009] The service call data is aggregated according to the service name dimension to obtain aggregated data corresponding to the multiple servers for different services;

[0010] By comparing the aggregated data of the multiple servers, it is possible to identify whether there are any servers with abnormal activity among the multiple servers.

[0011] This specification provides one or more embodiments of an anomaly detection device, comprising:

[0012] The service call data acquisition module collects service call data from the cluster, which contains multiple servers.

[0013] The service call data aggregation module aggregates the service call data according to the service name dimension to obtain aggregated data for different services corresponding to the multiple servers.

[0014] The aggregation comparison and anomaly identification module identifies whether there are any servers with anomalies by comparing the aggregated data of the multiple servers.

[0015] This specification provides one or more embodiments of an anomaly detection device, comprising:

[0016] At least one processor; and,

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to:

[0019] Collect service call data from a cluster containing multiple servers;

[0020] The service call data is aggregated according to the service name dimension to obtain aggregated data corresponding to the multiple servers for different services;

[0021] By comparing the aggregated data of the multiple servers, it is possible to identify whether there are any servers with abnormal activity among the multiple servers.

[0022] This specification provides one or more embodiments of a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0023] Collect service call data from a cluster containing multiple servers;

[0024] The service call data is aggregated according to the service name dimension to obtain aggregated data corresponding to the multiple servers for different services;

[0025] By comparing the aggregated data of the multiple servers, it is possible to identify whether there are any servers with abnormal activity among the multiple servers.

[0026] The above-described at least one technical solution adopted in one or more embodiments of this specification can achieve the following beneficial effects: It can collect service call data of each machine in the application server cluster, classify and compare the service call data of all individual application servers based on different services, accurately identify application servers with abnormalities, and thus help to identify abnormalities that are still in a small state before they deteriorate into failures, thereby preventing the occurrence of failures and improving the working reliability of the entire cluster; Moreover, since it is accurate to the service name level, it helps to quickly locate the changes involved in the service that may cause the abnormality, thereby preventing such changes from bringing abnormality risks to more application servers. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating an anomaly identification method provided in one or more embodiments of this specification;

[0029] Figure 2 This is a schematic diagram of the architecture of one embodiment of the anomaly identification method provided in one or more embodiments of this specification;

[0030] Figure 3 A flowchart illustrating one embodiment of the anomaly identification method provided in one or more embodiments of this specification;

[0031] Figure 4 A schematic diagram illustrating the principle of collecting and reporting service call data provided in one or more embodiments of this specification;

[0032] Figure 5 A flowchart illustrating a classification and comparison scheme for aggregated data provided in one or more embodiments of this specification;

[0033] Figure 6 A flowchart illustrating a server anomaly interaction verification scheme provided in one or more embodiments of this specification;

[0034] Figure 7 A schematic diagram of the structure of an anomaly identification device provided in one or more embodiments of this specification;

[0035] Figure 8 This is a schematic diagram of the structure of an anomaly identification device provided for one or more embodiments of this specification. Detailed Implementation

[0036] This specification provides an anomaly identification method, apparatus, device, and storage medium through its embodiments.

[0037] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0038] Faults often arise from very small anomalies that gradually deteriorate. Based on this principle, in the cluster scenario mentioned in the background, the applicant found that because small anomalies may initially occur only on individual or a small number of servers in the cluster, and are not enough to render those individual servers inoperable, the entire cluster appears normal from a global perspective, making it difficult to detect the small anomaly. By the time the anomaly deteriorates into a fault, it is too late, and multiple servers may have become inoperable due to the fault. Therefore, there is an urgent need for a solution that can identify (mainly, detect as early as possible) server anomalies in the cluster in advance to prevent faults from occurring.

[0039] The solutions the applicant had tried before were not able to meet this requirement well.

[0040] The first type of solution attempted was a monitoring component-based approach based on log collection. This involved relying on business systems to print relevant logs, with the monitoring component collecting these logs through proxy modules or devices, aggregating logs from all servers in the cluster, and deriving various trends in the overall cluster metrics, such as service call volume trends, or generating system metrics for a specific server. The drawbacks of this approach are: it depends on application systems printing standardized logs; different systems print different logs, requiring corresponding changes to the rules for processing the collected data; and it only allows viewing the trend of a single metric for the cluster or a single machine, failing to quickly identify a list of abnormal single machines.

[0041] The second type of solution explored was a health check component based on a heartbeat mechanism. Specifically, this involves servers in the cluster having a built-in heartbeat sending component that periodically sends heartbeats to a health check server, indicating that the server is still operating normally. The drawback of this type of solution is that it cannot detect minor business anomalies and can only be used to identify scenarios of complete server crashes, which are often far more severe than many other failure scenarios.

[0042] In view of this, the applicant has proposed a superior solution, which can identify even minor anomalies in advance and, by combining the scope of the change, accurately pinpoint the cause of the anomaly. This not only prevents further deterioration that the anomaly might cause but also helps to eliminate the anomaly at its source. The solution in this application will be explained in detail below.

[0043] Figure 1 This diagram illustrates a flow chart of an anomaly identification method provided in one or more embodiments of this specification. The process can be executed by servers in a cluster or by other devices managing these servers. The cluster itself can be used for different platforms to provide different services, such as large-scale payment platforms, instant messaging platforms, e-commerce platforms, etc. Common services include application services. In this case, the servers in the cluster can be called application servers, such as Java application servers. Intermediate results involved in this process (e.g., aggregation results, comparison results, etc.) can also be manually intervened to improve the accuracy of the anomaly identification results or subsequent anomaly cause location results.

[0044] Figure 1 The process includes the following steps:

[0045] S102: Collect service call data from the cluster, which contains multiple servers.

[0046] A cluster contains multiple nodes, typically worker nodes and management nodes responsible for managing these worker nodes. Each node can contain one or more servers, typically a single machine. This process can be executed on some or all servers in the cluster. For ease of description, the servers mentioned below refer to each server involved in the process.

[0047] In one or more embodiments of this specification, each server collects its own service call data locally. This data can then be aggregated through methods such as reporting and synchronization to obtain the overall service call data for the cluster. The granularity of the service can be predefined as needed. For example, it can be broadly defined as an executable function within the server; or, in scenarios where services are provided by processes, the service can be defined as such a process; and so on.

[0048] Service call data can reflect basic information about the service call and at least some of the call performance. For example, it can include the server identifier, the service name of the service involved, and whether the call was successful. Of course, if necessary, it can also include more detailed data, such as parameters, return results (e.g., return code, calculation result, etc.), call time, timestamp, whether the corresponding business was successful, and so on.

[0049] S104: Aggregate the service call data according to the service name dimension to obtain aggregated data corresponding to the multiple servers for different services.

[0050] In one or more embodiments of this specification, the service name represents the corresponding service, enabling low-cost differentiation between different services. To reliably identify anomalies with fine granularity, service call data from multiple servers is aggregated based on different services. This facilitates comparisons of service performance across multiple servers for different services. Consequently, it helps prevent anomalies exhibited by a single server in a particular service from being masked by its normal performance across most other service providers, thus aiding in the identification of minor anomalies on individual servers.

[0051] Based on the service name dimension, data for different services can be separated, and then data belonging to different servers for the same service can be aggregated. In this way, multiple aggregated data can be formed, each of which corresponds to a different service.

[0052] In practical applications, on the same server, some services may have similar names and rely on similar hardware and software resources. In such cases, the aggregated data corresponding to several similar or identical services can be further aggregated and then used for subsequent comparison and identification.

[0053] S106: By comparing the aggregated data of the multiple servers, identify whether there are any servers with abnormal activity among the multiple servers.

[0054] In one or more embodiments of this specification, comparisons can be performed within each aggregated data set. The comparisons actually focus on the performance differences of different servers on the same service. Servers with similar performance are grouped together, while servers with abnormal performance, due to their relatively large performance differences, may stand out as one or more separate instances. Through such comparison and classification, servers with abnormal performance can be identified more accurately. Furthermore, this method uses a dynamic relative reference standard (the normal performance of multiple normally functioning servers may also fluctuate), thus helping to avoid manually setting a large number of potentially inaccurate judgment thresholds. It is highly adaptable to different services and is very suitable for cluster scenarios with a large number of similar machines.

[0055] Furthermore, if the abnormal behavior of a server is identified among multiple servers, the cause of the abnormality can be located. In many cases, server abnormalities are caused by software issues, especially minor ones, while hardware issues are relatively less common. Moreover, once a hardware problem occurs, it may directly lead to a failure, rather than a continuous process where a minor abnormality gradually deteriorates into a major abnormality and then into a failure.

[0056] Software-related issues are often caused by recent changes (such as system changes, plugin updates, etc.), so these changes can be analyzed. Specifically, by comparing aggregated data from multiple servers to identify any servers with anomalies, and if so, determining the recent changes in the cluster (the "recent" timeframe can be set as needed, such as the last time, the last day, the last three days, or the last week), and the set of servers in the cluster that implemented these changes, it can be determined whether the identified server with anomalies is in this set. If so, the change can be classified as a risky change or further verification can be performed. For risky changes, the development or operations team should be notified promptly for appropriate action. For example, if the risky change has already been implemented on a server, it should be rolled back. Simultaneously, efforts should be made to prevent the implementation of the risky change on more servers. The risk should be mitigated through debugging and other methods before implementing safer changes.

[0057] This risk-shifting approach also helps to promptly identify more servers with potential for anomalies that have slipped through the cracks but have not yet been detected, and it also helps to reduce... Figure 1 The number of times a process is executed within the cluster can improve the efficiency of identifying anomalies across the entire cluster.

[0058] pass Figure 1 This method can collect service call data from each machine in the application server cluster. Based on different services, it classifies and compares the service call data of all individual application servers, accurately identifying application servers with anomalies. This helps to identify anomalies that are still in a small state before they deteriorate into failures, thereby preventing failures and improving the overall reliability of the cluster. Moreover, because it is accurate to the service name level, it helps to quickly locate the changes involved in the services that may cause the anomalies, thus preventing such changes from bringing anomalies to more application servers.

[0059] based on Figure 1 In addition to the method described herein, this specification also provides some specific implementation schemes and extension schemes of this method, which will be further explained below.

[0060] In one or more embodiments of this specification, to facilitate timely and reliable collection of service call data, a collection component can be pre-deployed on each server. The collection component intercepts services on the corresponding server and obtains service call data for the server by collecting aspects of the service (or other methods such as filters or interceptors can be used instead of aspects). The service call data reflects one or more metrics. For the service call data obtained by the server within a specified time range, the data is accumulated or averaged on different metrics (extreme values ​​can also be taken) and then reported to summarize the service call data of multiple servers and obtain the overall service call data of the cluster.

[0061] The metrics reflected in the service call data include, for example, at least one of the following: total number of calls, number of successful calls, total call duration, and return code information. Accordingly, when identifying a move, specifically, it involves identifying whether, among multiple servers, there exists a server exhibiting anomalies in at least one metric across at least one service.

[0062] These metrics can reflect the overall status of the server's current operation to a certain extent. Since service call data may be generated irregularly over a period of time, for cumulative metrics, the data for each metric is accumulated on each server. For example, for the metric of total number of calls to the same service, the total number of calls is incremented by one for each call. For metrics such as the total call time of the same service, the average of multiple calls or the maximum value can be taken to comprehensively assess whether the total call time of the server is abnormally long.

[0063] Furthermore, aggregation can be performed based on these metrics. Specifically, within the overall service call data of the cluster, multiple service call data sets are determined based on server identifiers (e.g., server IP address, MAC address, etc.), with each service call data set corresponding to a server. For each specified service name, one or more metrics corresponding to that service name are extracted from each service call data set and aggregated to obtain aggregated metric data for different services corresponding to multiple servers. This allows for the comprehensive identification of anomalies for a single service based on multiple metrics.

[0064] In the case of comprehensive indicators, the multidimensional characteristics exhibited by multiple servers on the same service can be determined based on the indicators contained in the aggregated data (when multiple indicators are integrated). Outlier identification and classification are performed based on the multidimensional characteristics. Based on the identification and classification results of outliers, it can be determined whether there are any servers with anomalies among the multiple servers. For each service, this identification process can be performed once. If a server corresponds to an outlier in at least one round of identification (for example, the data indicating the outlier was collected from this server, or further extracted from the data collected from this server), it can be determined that the server has anomalies. These anomalies are highly likely to be caused by changes involved in the service corresponding to at least one round, thereby enabling the rapid identification of risk changes.

[0065] It should be noted that outlier identification is used here as an example to achieve aggregated data comparison. This is because it is more efficient, avoiding the waste of a lot of computing power and time in comparing each pair one by one. Moreover, it can more reliably establish a dynamic and normal reference standard (i.e., the service performance corresponding to most cluster points).

[0066] Based on the above description, one or more embodiments of this specification also provide an architectural diagram of one implementation scheme of the above-described anomaly identification method, such as... Figure 2 As shown. In Figure 2 This example involves multiple cooperating entities, including application servers, data pipelines, computing engines, notification components, management consoles, and operations and maintenance personnel. Except for the operations and maintenance personnel, all other entities can be part of the cluster. There are multiple application servers; one is shown as an example. The aforementioned data collection actions can be executed on their respective application servers; the aforementioned aggregation actions can be executed in the data pipelines connecting the application servers and the computing engine; the aforementioned comparison actions can be executed in the computing engine; if an abnormal server is detected, the notification component, in response to the comparison and identification results from the computing engine, notifies the operations and maintenance personnel to handle the issue; the necessary configurations for these entities can be configured by the operations and maintenance personnel on the management console and then distributed to the corresponding entities.

[0067] exist Figure 2 In the diagram, the arrows from operations and maintenance personnel to the control console and then to other entities represent the flow of configuration data, which can be called the configuration flow. The remaining arrows represent the flow of service call data and the data obtained from its subsequent processing, which can be called the service call data flow.

[0068] based on Figure 2 The architecture described herein, along with one or more embodiments, also provides flowcharts illustrating this implementation, such as... Figure 3 As shown.

[0069] Figure 3The process includes the following steps:

[0070] S302: Operations personnel configure the required configuration information in the cluster on the management console, including at least one of the following: collection configuration, aggregation configuration, algorithm parameters and change information, and notification configuration.

[0071] S304: The control console will send configuration information to the corresponding entities in the cluster, including at least one of the following: sending the collection configuration to the application server, sending the aggregation configuration to the data pipeline, sending the algorithm parameters and change information to the computing engine, and sending the notification configuration to the notification component.

[0072] S306: The application server collects local service call data through a pre-deployed SDK and reports it periodically. The reported data will be transmitted through a data pipeline.

[0073] For example, one or more embodiments of this specification provide a schematic diagram of the principle of service call data collection and reporting, see [link / reference]. Figure 4 .

[0074] exist Figure 4 The diagram illustrates how the SDK deployed on the application server works. This SDK primarily consists of a data collection component and a scheduled reporting component. The application server only needs to integrate this SDK; it will automatically load upon application server startup. The SDK's data collection component acts as an interceptor, intercepting incoming service call traffic (which can continue normally after exiting the SDK). Through aspect-based data collection, it obtains service call data, such as service name, parameters, return results, call success status, and execution time. It then performs comprehensive processing on the service call data, including accumulating or averaging metrics based on service name or custom dimensions, and caches this data on the application server. The SDK's scheduled reporting component then transmits the cached metric data to the data pipeline.

[0075] More intuitively, the following is the structure of metric data cached on the application server within a certain time range:

[0076]

[0077] The aggregation dimensions and configuration information such as business success rules are configurations issued from the management console.

[0078] S308: The data pipeline preprocesses the metric data and aggregates it according to the service name dimension, generating metric data for different servers based on the service name dimension.

[0079] At this point, the data volume is large and the text similarity is high, so the data can be compressed before transmission.

[0080] Alternatively, the preprocessing steps can also be performed within the computing engine, in which case the data pipeline is simply responsible for transmitting the data.

[0081] S310: The computing engine receives pre-processed data from the data pipeline, uses an outlier identification algorithm to calculate and identify whether there are any abnormal application servers. If so, it locates the problem by combining the change information and notifies the relevant parties through the notification component.

[0082] The anomalies that can be identified include at least one of the following: anomalies in total service call volume, anomalies in service call success rate, anomalies in service business success rate, anomalies in service return code distribution, and anomalies in service call time.

[0083] For example, one or more embodiments of this specification provide a flowchart illustrating a classification and comparison scheme for aggregated data. See [link to relevant documentation]. Figure 5 .

[0084] Figure 5 The process mainly includes three parts:

[0085] Part 1: Feature Processing

[0086] The input features may include at least one of the following: total number of calls, number of successful business calls, number of successful calls, total call time, return codes and their number. These features are processed for use by the algorithm, such as averaging the time spent and converting the number of successful calls into a success rate.

[0087] Part Two: Algorithm Recognition.

[0088] Outlier detection algorithms are used to identify application servers that exhibit abnormal behavior or even more serious anomalies. For example, density-based LOF algorithms work on the principle that if the density around a data point is relatively low, it may be an outlier.

[0089] Part Three: Linking and calibrating changes based on the change information.

[0090] By acquiring system changes to servers around the same time and a list of changed application servers, this information is compared with the identified abnormal application servers. If the abnormal server is within the scope of the changes, it is considered that the change caused the problem, and this change can be pushed to operations and maintenance personnel for rapid emergency response. Additionally, this change information can be used for semi-supervised learning to improve the accuracy of the algorithm.

[0091] Furthermore, to more accurately and reliably identify anomalies and their causes, and to be more sensitive to minor anomalies, one or more embodiments of this specification also provide a flowchart of a server anomaly interaction verification scheme, see [link to relevant documentation]. Figure 6In this scheme, if the service call data subset corresponding to the time slice in which the above-mentioned occurrence occurs belongs to the identified outlier, the change will not be directly identified as a risky change, but will be further verified at a finer level.

[0092] Figure 6 The process includes the following steps:

[0093] S602: Determine the time of occurrence of the change, and the service call data of the server with the change in the aggregated data, wherein the time of occurrence is within the time range corresponding to the service call data.

[0094] S604: Divide the time range into multiple time slices, and divide the service call data according to the time slices to obtain multiple service call data subsets.

[0095] When performing time sharding, the impact of busy / idle periods can be considered to achieve reasonable sharding. Furthermore, only a subset of time shards can be selected (e.g., sampling at intervals to ensure sufficient distance between selected time shards, rather than too close together) to execute subsequent steps. With reasonable sharding, if there are no anomalies in the cluster within that time range, the similarity of these multiple service call data subsets will be high.

[0096] S606: Outlier identification and classification are performed based on the subsets of service call data.

[0097] S608: If the service call data subset corresponding to the time slice in which the occurrence time is located belongs to the identified outlier, then the change is determined to be a risk change.

[0098] If no outliers are found, it indicates that the above anomaly determination results may be incorrect, and the anomaly identification can be tried again.

[0099] In this way, by using a fine-grained subset of service call data, it is possible to further verify whether there are indeed any changes to a single machine within its own performance, and to determine more reliably and efficiently whether the changes actually pose a risk.

[0100] Following this fine-grained approach, it is possible to further verify, based on a subset of service call data, whether any servers that were not previously identified as having anomalies have slipped through the net.

[0101] Specifically, for example, after identifying an outlier by determining if the service call data subset corresponding to the time slice where the event occurred is a valid outlier: One or more service call data subsets corresponding to the abnormal server (preferably those reflecting anomalies) are used as a reference subset. For multiple servers in the cluster that are not yet considered abnormal, the service call data subsets corresponding to the time slice where the event occurred are obtained for each of these servers, serving as a verification subset. Outliers in the reference subset and each verification subset are identified and categorized to determine the degree of outlier in the reference subset. Based on the degree of outlier in the reference subset, it is verified whether there are still abnormal servers among the multiple servers not yet considered abnormal. If the reference subset is sufficiently outlier, it indicates that the verified server is likely indeed not abnormal; conversely, if the service call data subsets of the verified server are similar to those in the reference subset, it indicates that the verified server may be abnormal. This helps to uncover more abnormal servers or correct more misjudged servers.

[0102] After completing the anomaly identification, the notification component consumes data from the computing engine. When the computing engine identifies anomalies in certain application servers, the notification component can be used to notify the operation and maintenance personnel of these application servers. The notification method and time limit are configurable and can be SMS, telephone, etc.

[0103] pass Figure 3 The proposed solution effectively addresses the problems encountered with previous solutions attempted by the applicant. These include:

[0104] This solves the problem that monitoring component solutions based on log collection rely on application system log printing. Specifically, data can be obtained by intercepting service requests through SDK aspects, without relying on log printing and log collection.

[0105] This solves the problem that relying on application system log printing only allows you to see the overall trend or the metrics of a specific machine. Specifically, it can use algorithms to classify the data of all application servers in the entire cluster and identify application servers with abnormal activity.

[0106] This solution addresses the issue that health check components based on heartbeat mechanisms cannot detect business anomalies other than complete service crashes. Specifically, the SDK can be used to obtain service call information through aspect-oriented programming, identify business anomalies, including anomalies in business success rate and return code distribution, and detect business anomalies other than complete service crashes.

[0107] Furthermore, the solution proposed in this application can be combined with change information to reliably detect related suspected changes and identify calibration anomalies. It is also highly adaptable, has a complete process framework, can monitor functions at multiple levels, and does not intrude on business operations.

[0108] Based on the same idea, one or more embodiments of this specification also provide apparatus and devices corresponding to the above methods, such as... Figure 7 , Figure 8 As shown. The apparatus and equipment are capable of performing the above methods and related alternatives accordingly.

[0109] Figure 7 This is a schematic diagram of the structure of an anomaly detection device provided in one or more embodiments of this specification. The device includes:

[0110] The service call data acquisition module 702 collects service call data from a cluster containing multiple servers.

[0111] The service call data aggregation module 704 aggregates the service call data according to the service name dimension to obtain aggregated data corresponding to the multiple servers for different services.

[0112] The aggregation comparison and anomaly identification module 706 identifies whether there are any servers with anomalies among the multiple servers by comparing the aggregated data of the multiple servers.

[0113] Optionally, the service call data acquisition module 702 intercepts the corresponding service on the server by using acquisition components pre-deployed on each of the servers;

[0114] By collecting data from the service aspect, service call data corresponding to the server is obtained, and the service call data reflects one or more metrics.

[0115] For the service call data obtained by the server within a specified time range, the data is accumulated or averaged on different metrics and then reported to summarize the service call data corresponding to the multiple servers and obtain the overall service call data of the cluster.

[0116] Optionally, the service call data aggregation module 704 determines multiple service call data sets based on server identifiers in the service call data, with each service call data set corresponding to one of the servers.

[0117] For each specified service name, one or more metrics corresponding to that service name are extracted from each service call data set and aggregated to obtain aggregated metric data for different services corresponding to the multiple servers.

[0118] Optionally, the metrics include at least one of the following: total number of calls, number of successful calls, number of successful calls, total call duration, and return code information;

[0119] The aggregation comparison identification anomaly module 706 identifies whether, among the plurality of servers, there is a server that has an anomaly in at least one of the aforementioned metrics on at least one service.

[0120] Optionally, the aggregation comparison and anomaly identification module 706 determines the multidimensional features exhibited by the multiple servers on the same service based on the indicators contained in the aggregated data.

[0121] Outlier identification and classification are performed based on the multidimensional features. Based on the identification and classification results of outliers, it is determined whether there are any servers with abnormal activity among the multiple servers.

[0122] Optionally, the data collection is performed in each of the corresponding servers;

[0123] The aggregation is performed in the data pipeline connecting the server and the computing engine, or in the computing engine.

[0124] The comparison is performed in the computing engine.

[0125] Optionally, the aggregation comparison and anomaly identification module 706, after identifying whether there are any servers with anomalies among the multiple servers by comparing the aggregated data for the multiple servers, determines the recent changes in the cluster and the set of servers in the cluster that have performed the changes.

[0126] Determine whether the identified server with abnormal activity is in the server set;

[0127] If so, the change will be classified as a risk change.

[0128] Optionally, the aggregation comparison and anomaly identification module 706 determines the time of occurrence of the change and the service call data of the server with the change in the aggregated data, wherein the time of occurrence is within the time range corresponding to the service call data;

[0129] The time range is divided into multiple time segments, and the service call data is divided according to the time segments to obtain multiple service call data subsets.

[0130] Outlier identification and classification are performed based on the multiple service call data subsets;

[0131] If the service call data subset corresponding to the time slice in which the occurrence occurred belongs to the identified outlier, then the change is determined to be a risky change.

[0132] Optionally, the aggregation comparison and anomaly identification module 706, after the service call data subset corresponding to the time segment where the occurrence time is located is identified as an outlier, uses the service call data subset as a reference subset, and for multiple servers in the cluster that are not yet identified as having anomalies, obtains the service call data subset corresponding to the time segment where the occurrence time is located for each of the servers that are not yet identified as having anomalies, as a verification subset;

[0133] Outlier points in the reference subset and each of the verification subsets are identified and classified to determine the degree of outlier status of the reference subset.

[0134] Based on the outlier degree of the reference subset, it is verified whether there are still servers with abnormal activity among the multiple servers that are not yet considered to have abnormal activity.

[0135] Optionally, the service refers to an executable function in the server.

[0136] Figure 8 This specification provides a schematic diagram of the structure of an anomaly detection device according to one or more embodiments. The device includes:

[0137] At least one processor; and,

[0138] A memory communicatively connected to the at least one processor; wherein,

[0139] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to:

[0140] Collect service call data from a cluster containing multiple servers;

[0141] The service call data is aggregated according to the service name dimension to obtain aggregated data corresponding to the multiple servers for different services;

[0142] By comparing the aggregated data of the multiple servers, it is possible to identify whether there are any servers with abnormal activity among the multiple servers.

[0143] Based on the same idea, one or more embodiments of this specification also provide a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0144] Collect service call data from a cluster containing multiple servers;

[0145] The service call data is aggregated according to the service name dimension to obtain aggregated data corresponding to the multiple servers for different services;

[0146] By comparing the aggregated data of the multiple servers, it is possible to identify whether there are any servers with abnormal activity among the multiple servers.

[0147] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0148] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0149] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0150] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0151] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0152] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0153] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0154] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0155] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0156] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0157] The above description is merely one or more embodiments of this specification and is not intended to limit this specification. Various modifications and variations can be made to the one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of this specification.

Claims

1. An anomaly identification method, comprising: The process involves collecting service call data from a cluster containing multiple servers. Specifically, it includes: intercepting services on the respective servers using collection components pre-deployed on each server; obtaining service call data corresponding to the server by collecting aspects of the service; and the service call data including at least a server identifier, service name, and at least one of the following metrics: total number of calls, number of successful calls, number of successful calls, total call duration, and return code information. The service call data is aggregated according to the service name dimension to obtain aggregated data corresponding to the multiple servers for different services, forming multiple aggregated data sets, each corresponding to a different service. Specifically, this includes: determining multiple service call data sets based on the server identifier in the service call data, with each service call data set corresponding to one server; for each specified service name, extracting the metrics corresponding to that service name from each service call data set and aggregating them to obtain aggregated metric data corresponding to the multiple servers for different services. Identifying whether there are any abnormal servers among the multiple servers by comparing the aggregated data of the multiple servers includes: comparing the performance differences of different servers on the same service within each aggregated data set; the comparison includes: determining the multidimensional features exhibited by the multiple servers on the same service based on the indicators contained in the aggregated data; and identifying and classifying outliers based on the multidimensional features to determine whether there are any abnormal servers.

2. The method as described in claim 1, wherein collecting service call data in the cluster specifically includes: For the service call data obtained by the server within a specified time range, the data is accumulated or averaged on different metrics and then reported to summarize the service call data corresponding to the multiple servers and obtain the overall service call data of the cluster.

3. The method as described in claim 1 or 2, wherein identifying whether there is an abnormal server among the plurality of servers specifically includes: The identification process determines whether a server exists among the plurality of servers that exhibits anomalies in at least one of the aforementioned metrics across at least one service.

4. The method as described in claim 1, wherein the data collection is performed in each of the corresponding servers; The aggregation is performed in the data pipeline connecting the server and the computing engine, or in the computing engine. The comparison is performed in the computing engine.

5. The method of claim 1, wherein after identifying whether there is an abnormal server among the plurality of servers by comparing the aggregated data of the plurality of servers, the method further includes: Identify recent changes in the cluster, and the set of servers in the cluster that performed those changes; Determine whether the identified server with abnormal activity is in the server set; If so, the change will be classified as a risk change.

6. The method of claim 5, wherein determining the change as a risk change specifically includes: Determine the time of occurrence of the change, and the service call data of the server with the change in the aggregated data, wherein the time of occurrence is within the time range corresponding to the service call data; The time range is divided into multiple time segments, and the service call data is divided according to the time segments to obtain multiple service call data subsets. Outlier identification and classification are performed based on the multiple service call data subsets; If the service call data subset corresponding to the time slice in which the occurrence occurred belongs to the identified outlier, then the change is determined to be a risky change.

7. The method of claim 6, wherein if the subset of service call data corresponding to the time slice in which the occurrence time occurs belongs to the identified outlier, the method further includes: Using the service call data subset as a reference subset, for multiple servers in the cluster that are not yet abnormal, obtain the service call data subset corresponding to the time slice where the occurrence time is located for each of the servers that are not yet abnormal, and use it as a verification subset. Outlier points in the reference subset and each of the verification subsets are identified and classified to determine the degree of outlier status of the reference subset. Based on the outlier degree of the reference subset, it is verified whether there are still servers with abnormal activity among the multiple servers that are not yet considered to have abnormal activity.

8. The method as described in claim 1, wherein the service refers to an executable function in the server.

9. An anomaly detection device, comprising: The service call data collection module collects service call data from a cluster containing multiple servers. Specifically, it includes: intercepting services on the corresponding servers using collection components pre-deployed on each server; obtaining service call data corresponding to the server by targeting the service collection aspect; the service call data includes at least server identifier, service name, and at least one of the following indicators: total number of calls, number of successful business calls, number of successful calls, total call time, and return code information. The service call data aggregation module aggregates the service call data according to the service name dimension, obtaining aggregated data corresponding to the multiple servers for different services, forming multiple sets of aggregated data, each set corresponding to a different service. Specifically, this includes: determining multiple service call data sets based on the server identifier in the service call data, with each set corresponding to one server; for each specified service name, extracting the metrics corresponding to that service name from each set of service call data, and aggregating them to obtain aggregated metric data corresponding to the multiple servers for different services. The aggregation comparison and anomaly identification module identifies whether there are any servers exhibiting anomalies by comparing the aggregated data of the multiple servers. This includes: comparing the performance differences of different servers on the same service within each set of aggregated data; the comparison includes: determining the multidimensional features exhibited by the multiple servers on the same service based on the indicators contained in the aggregated data; and identifying and classifying outliers based on the multidimensional features to determine whether there are any servers exhibiting anomalies.

10. The apparatus of claim 9, wherein the service call data acquisition module accumulates or averages the service call data obtained by the server within a specified time range on different indicators and then reports the data to summarize the service call data corresponding to the multiple servers and obtain the overall service call data of the cluster.

11. The apparatus of claim 9 or 10, wherein the aggregation comparison identification anomaly module identifies whether, among the plurality of servers, there is a server that has an anomaly in at least one of the aforementioned metrics on at least one service.

12. The apparatus of claim 9, wherein the data acquisition is performed in each of the corresponding servers; The aggregation is performed in the data pipeline connecting the server and the computing engine, or in the computing engine. The comparison is performed in the computing engine.

13. The apparatus of claim 9, wherein the aggregation comparison and anomaly identification module, after identifying whether there are any servers with anomalies among the plurality of servers by comparing the aggregated data for the plurality of servers, determines recent changes in the cluster and the set of servers in the cluster that have performed the changes. Determine whether the identified server with abnormal activity is in the server set; If so, the change will be classified as a risk change.

14. The apparatus of claim 13, wherein the aggregation comparison and anomaly identification module determines the time of occurrence of the change, and the service call data belonging to the server with the anomaly in the aggregated data, wherein, The occurrence time is within the time range corresponding to the service call data; The time range is divided into multiple time segments, and the service call data is divided according to the time segments to obtain multiple service call data subsets. Outlier identification and classification are performed based on the multiple service call data subsets; If the service call data subset corresponding to the time slice in which the occurrence occurred belongs to the identified outlier, then the change is determined to be a risky change.

15. The apparatus of claim 14, wherein the aggregation comparison and identification anomaly module, after determining that the service call data subset corresponding to the time slice in which the occurrence time is located belongs to the identified outlier, uses the service call data subset as a reference subset, and for multiple servers in the cluster that do not yet belong to the anomaly category, obtains the service call data subset corresponding to the time slice in which the occurrence time is located for each of the servers that do not yet belong to the anomaly category, as a verification subset; Outlier points in the reference subset and each of the verification subsets are identified and classified to determine the degree of outlier status of the reference subset. Based on the outlier degree of the reference subset, it is verified whether there are still servers with abnormal activity among the multiple servers that are not yet considered to have abnormal activity.

16. The apparatus of claim 9, wherein the service refers to an executable function in the server.

17. An anomaly detection device, comprising: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform: The process involves collecting service call data from a cluster containing multiple servers. Specifically, it includes: intercepting services on the respective servers using collection components pre-deployed on each server; obtaining service call data corresponding to the server by collecting aspects of the service; and the service call data including at least a server identifier, service name, and at least one of the following metrics: total number of calls, number of successful calls, number of successful calls, total call duration, and return code information. The service call data is aggregated according to the service name dimension to obtain aggregated data corresponding to the multiple servers for different services, forming multiple aggregated data sets, each corresponding to a different service. Specifically, this includes: determining multiple service call data sets based on the server identifier in the service call data, with each service call data set corresponding to one server; for each specified service name, extracting the metrics corresponding to that service name from each service call data set and aggregating them to obtain aggregated metric data corresponding to the multiple servers for different services. Identifying whether there are any abnormal servers among the multiple servers by comparing the aggregated data of the multiple servers includes: comparing the performance differences of different servers on the same service within each aggregated data set; the comparison includes: determining the multidimensional features exhibited by the multiple servers on the same service based on the indicators contained in the aggregated data; and identifying and classifying outliers based on the multidimensional features to determine whether there are any abnormal servers.

Images