Abnormality detection method and device for server data collection behavior
By acquiring the server's collected statistical information and calculating the mean and variance, the system automatically detects anomalies in the server's data collection behavior, solving the inaccuracies and omissions caused by complex rule configurations in existing technologies, and achieving efficient anomaly detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ALIBABA (CHINA) CO LTD
- Filing Date
- 2022-02-28
- Publication Date
- 2026-04-17
AI Technical Summary
Existing technologies rely on complex detection rule configurations for anomaly detection in the collection of massive amounts of server data, leading to inaccuracies or omissions.
By acquiring statistical information from multiple servers within a target time period, calculating the mean and variance of the number of data collections, the system automatically detects whether server data collection behavior is abnormal, thus avoiding complex detection rule configurations.
It enables accurate detection of anomalies in server data collection behavior without the need for complex rule configuration, improving detection accuracy and efficiency and reducing omissions.
Smart Images

Figure CN114546704B_ABST
Abstract
Description
Technical Field
[0001] The embodiments in this specification relate to the field of computer technology, and more specifically, to a method and apparatus for detecting anomalies in server data collection behavior. Background Technology
[0002] Internet Data Centers (IDCs) typically have a massive number of servers, and some servers exhibit different data collection behaviors. Currently, the detection of anomalies in data collection behavior across these massive numbers of servers generally relies on manual configuration of complex detection rules to match each server's collection behavior one-to-one. However, these complex rule configurations are often inaccurate or prone to omissions.
[0003] Therefore, there is an urgent need for a reasonable and reliable solution that can effectively detect anomalies in server data collection behavior without requiring complex detection rule configuration. Summary of the Invention
[0004] This specification provides an embodiment of a method and apparatus for detecting anomalies in server data collection behavior, which can effectively detect anomalies in server data collection behavior without requiring complex detection rule configuration.
[0005] In a first aspect, embodiments of this specification provide an anomaly detection method for server data collection behavior, comprising: acquiring collection statistics information of multiple servers within a target time period, wherein the multiple servers involve several collection items, and the collection statistics information includes the number of times each of the multiple servers collects data under at least one of the several collection items; for a first collection item among the several collection items, calculating a first statistical result for the number of times the multiple servers collect data under the first collection item in the collection statistics information; and detecting whether the data collection behavior of the multiple servers for the first collection item is abnormal based on the first statistical result.
[0006] In some embodiments, the first statistical result includes the calculated variance; and the detection of whether the data collection behavior of the plurality of servers for the first collection item is abnormal includes: if the variance in the first statistical result exceeds a first threshold, then determining that the data collection behavior of the plurality of servers for the first collection item is abnormal.
[0007] In some embodiments, the first statistical result further includes a calculated mean; and the step of detecting whether the data collection behavior of the plurality of servers for the first collection item is abnormal further includes: obtaining a second statistical result associated with the plurality of servers, including a historical mean, the historical mean being calculated based on the historical collection frequency of the plurality of servers under the first collection item; calculating a first difference between the mean in the first statistical result and the historical mean; if the first difference does not exceed a second threshold and the variance in the first statistical result does not exceed the first threshold, then it is determined that the data collection behavior of the plurality of servers for the first collection item is normal.
[0008] In some embodiments, the first statistical result includes the calculated mean; and the detection of whether the data collection behavior of the plurality of servers for the first collection item is abnormal includes: for a server among the plurality of servers, calculating a second difference between its target collection count and the mean in the first statistical result; if the second difference exceeds a third threshold, then determining that the server's data collection behavior for the first collection item is abnormal, wherein the target collection count is the number of times the server collects data under the first collection item in the collection statistics information.
[0009] In some embodiments, obtaining the collection statistics information of multiple servers within a target time period includes: obtaining the collection data of the multiple servers within the target time period, including the collection content of each of the multiple servers under at least one of the plurality of collection items; for each of the multiple servers and the collection item involved by that server in the plurality of collection items, determining the number of collection content items of that server under that collection item in the collection data, and determining the number of items as the number of times that server collects under that collection item; and generating the collection statistics information based on the number of times collected for each of the multiple servers.
[0010] In some embodiments, the plurality of servers each have attribute values under the target server attribute; and the method further includes: dividing the plurality of servers into several server groups according to the target server attribute; the several servers are specifically the target server group among the several server groups, and the servers in the target server group are related to the first collection item.
[0011] In some embodiments, after detecting whether the data collection behavior of the plurality of servers for the first collection item is abnormal, the method further includes: in response to the detection result that the data collection behavior of at least one of the plurality of servers is abnormal, obtaining a collection count set, which is formed by the collection count of the plurality of servers under the first collection item in the collection statistics information; for the collection count in the collection count set, determining the number of servers corresponding to the collection count, wherein the number of servers is the number of servers among the plurality of servers associated with the collection count; generating a first detection result, which includes the collection count in the collection count set and the number of servers corresponding to the collection count.
[0012] In some embodiments, the collected statistical information includes multiple sub-data items, wherein the sub-data items include a server identifier, the server identifier indicating a specific collection item involved in the plurality of collection items, and the number of times the server collects data under that specific collection item; and after detecting whether the data collection behavior of the plurality of servers for the first collection item is abnormal, the method further includes: in response to the detection result indicating that the data collection behavior of at least one of the plurality of servers is abnormal, obtaining a plurality of sub-data items from the multiple sub-data items based on the server identifier of the plurality of servers and the first collection item; sorting the plurality of sub-data items according to the number of collections; and generating a second detection result, which includes the sorted plurality of sub-data items.
[0013] Secondly, embodiments of this specification provide an anomaly detection device for server data collection behavior, comprising: an acquisition unit configured to acquire collection statistics of multiple servers within a target time period, wherein the multiple servers involve several collection items, and the collection statistics include the number of times each of the multiple servers collects data under at least one of the several collection items; a calculation unit configured to calculate a first statistical result for the number of times the multiple servers collect data under the first collection item in the collection statistics; and a detection unit configured to detect whether the data collection behavior of the multiple servers for the first collection item is abnormal based on the first statistical result.
[0014] Thirdly, embodiments of this specification provide a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed in a computer, it causes the computer to perform the method described in any implementation of the first aspect.
[0015] Fourthly, embodiments of this specification provide a computing device including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method described in any of the implementations in the first aspect.
[0016] Fifthly, embodiments of this specification provide a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the method described in any implementation of the first aspect.
[0017] The solution provided by the above embodiments of this specification can obtain collection statistics information from multiple servers within a target time period. These multiple servers involve several collection items, and the collection statistics information includes the number of times each of the multiple servers collects data under at least one of these collection items. Then, for a first collection item (such as any collection item), a first statistical result can be calculated based on the number of times the multiple servers collect data under the first collection item in the collection statistics information. Based on the first statistical result, it can be detected whether the data collection behavior of the multiple servers for the first collection item is abnormal. It should be noted that the first statistical result may, for example, include the calculated mean and / or variance. The mean can reflect the overall average situation, and the variance can reflect individual outliers. By detecting whether the server's data collection behavior is abnormal based on the mean and / or variance, effective anomaly detection can be performed on the server's data collection behavior, obtaining more accurate detection results. Moreover, compared with existing technologies, complex detection rule configuration can be avoided, thereby avoiding situations where the detection rule configuration is inaccurate or incomplete. Attached Figure Description
[0018] To more clearly illustrate the technical solutions of the various embodiments disclosed in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only a few embodiments disclosed in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 This is an exemplary system architecture diagram to which some embodiments of this specification can be applied;
[0020] Figure 2 This is a flowchart of an embodiment of an anomaly detection method for server data collection behavior;
[0021] Figure 3 This is a schematic diagram illustrating the detection process for whether the data collection behavior of several servers for the first data collection item is abnormal;
[0022] Figure 4 This is a schematic diagram illustrating the detection process for whether the server's data collection behavior for the first data item is abnormal;
[0023] Figure 5 This is a schematic diagram of a device for detecting anomalies in server data collection behavior. Detailed Implementation
[0024] The present specification will now be described in further detail with reference to the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. The described embodiments are only a part of the embodiments described herein, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without inventive effort are within the scope of protection of this application.
[0025] It should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described herein can be combined with each other. Furthermore, the terms "first," "second," etc., used in the embodiments of this specification are for informational purposes only and do not constitute any limitation.
[0026] As mentioned earlier, in order to detect anomalies in data collection behavior on massive servers, it is generally necessary to manually configure complex detection rules to match the data collection behavior of massive servers one by one. Complex detection rule configurations are often inaccurate or have omissions.
[0027] Based on this, some embodiments of this specification provide methods for anomaly detection of server data collection behavior, which can effectively detect anomalies in server data collection behavior without requiring complex detection rule configuration. Specifically, Figure 1 An exemplary system architecture diagram applicable to these embodiments is shown.
[0028] like Figure 1 As shown, the system architecture may include a detection system that can periodically detect anomalies in the data collection behavior of multiple servers, for example, by checking every set interval. This set interval could be, for example, 1, 5, or 10 minutes, and is not specifically limited here. It should be noted that the detection system can connect to an internet data center, and these multiple servers can be servers within that internet data center.
[0029] In practice, when the detection cycle arrives, the detection system can obtain statistical information collected from the aforementioned multiple servers within the target time period. The target time period can be, for example, approximately 0.5, 1, or 1.5 hours, etc., without specific limitations.
[0030] The aforementioned servers may involve several collection items, which can be a set of collection items involved by each of the aforementioned servers. The aforementioned collection statistics may include the number of times each of the aforementioned servers collected data under at least one of the aforementioned collection items.
[0031] In this context, a single data collection item can be the name of a type of data collection task. This type of task might include, for example, CPU (Central Processing Unit) utilization data collection, memory utilization data collection, or heat data collection, etc. Typically, one data collection item can represent the same type of data collection and have the same collection cycle. It should be understood that the embodiments in this specification do not specifically limit the data collection items involved in the server.
[0032] In one embodiment, the system architecture may further include a target database, which may store the collected data from the aforementioned multiple servers. The detection system can obtain the collected data from the target database within a target time period from the aforementioned multiple servers, and determine the aforementioned collection statistics based on the collected data. The collected data may, for example, include the collection content of each of the aforementioned multiple servers under at least one of the aforementioned collection items. Of course, the collected data may also include other information items, such as the server identifier corresponding to the collected content, timestamp, collection item, etc.
[0033] Taking the aforementioned servers as servers within an internet data center, and the target time period as approximately one hour, for example... Figure 1 As shown, the detection system can acquire the collected data of the aforementioned multiple servers in the Internet data center within the past hour, and determine the collected statistical information of the aforementioned multiple servers within the past hour based on the collected data.
[0034] Subsequently, the detection system can calculate the statistical results of the number of times each server collects data under the same data collection item in the above-mentioned collection statistics information for several servers involved in the same data collection item, and detect whether the data collection behavior of the several servers for the same data collection item is abnormal based on the statistical results.
[0035] It should be noted that the above statistical results may include, for example, the calculated mean and / or variance. The mean reflects the overall average, while the variance reflects individual outliers. By detecting whether the server's data collection behavior is abnormal based on the mean and / or variance, effective anomaly detection can be performed on server data collection behavior, obtaining more accurate detection results. Moreover, compared to existing technologies, it avoids the need for complex detection rule configuration, thereby preventing situations where detection rule configuration is inaccurate or omitted.
[0036] In one embodiment, the detection results obtained during the detection process can be provided to the Internet data center. Further, in one example, the Internet data center may have a target group chat created on a target communication platform, and the detection results can be sent to the target group chat via the target communication platform. The group members of the target group chat may include the Internet data center's operations and maintenance personnel. By sending the detection results to the target group chat, the operations and maintenance personnel can be promptly informed of the relevant information regarding the server's data collection activities.
[0037] The specific implementation steps of the above method are described below with reference to specific embodiments.
[0038] See Figure 2 The document illustrates a flow 200 of an embodiment of an anomaly detection method for server data collection behavior. The execution entity of this method can be... Figure 1 The detection system shown. The method includes the following steps:
[0039] Step 202: Obtain collection statistics information of multiple servers within the target time period. The multiple servers involve several collection items. The collection statistics information includes the number of times each of the multiple servers collects data under at least one of the several collection items.
[0040] Step 204: For the first collection item among the several collection items, calculate the first statistical result for the number of times several servers collect data under the first collection item in the collection statistics information;
[0041] Step 206: Based on the first statistical results, detect whether the data collection behavior of the aforementioned servers for the first collection item is abnormal.
[0042] The following is a further explanation of each of the above steps.
[0043] In step 202, statistical information collected from multiple servers within a target time period can be acquired periodically. For example, it can be acquired every set interval. These multiple servers can be, for example, servers within an internet data center. The set interval can be, for example, 1, 5, or 10 minutes, and the target time period can be, for example, approximately 0.5, 1, or 1.5 hours, etc., without specific limitations.
[0044] The aforementioned multiple servers may involve several collection items, which can be a set of collection items involved by each of the aforementioned multiple servers. The aforementioned collection statistics may include the number of times each of the aforementioned multiple servers has collected data under at least one of the aforementioned collection items.
[0045] Further, in step 202, the collected data from the aforementioned multiple servers within the target time period can be obtained, for example, from a target database. This collected data may include the collected content from each of the multiple servers under at least one of the aforementioned collection items. Then, for each of the multiple servers and the collection items involved in that server among the aforementioned collection items, the number of records of the server's collected content under that collection item in the collected data can be determined, and this number is defined as the number of times the server collected content under that collection item. Then, based on the number of times each of the multiple servers has been collected, the aforementioned collection statistics can be generated.
[0046] Next, in step 204, for the first collection item (such as any collection item) among the above collection items, a first statistical result can be calculated for the number of times the servers in the above collection statistics are collected under the first collection item.
[0047] In one example, the aforementioned servers can specifically refer to each of the aforementioned multiple servers involved in the first collection item.
[0048] In another example, the aforementioned servers may all possess attribute values under the target server attribute. A single target server attribute can be, for example, a manufacturer, model, or service type, etc., without specific limitations here. Before step 204, the aforementioned servers can be divided into several server groups based on the target server attribute, wherein all servers in any server group have the same attribute value under the target server attribute. Based on this, in step 204, the aforementioned servers can specifically refer to the target server group within these server groups. The servers in the target server group are involved in the first data collection item.
[0049] In practice, when the aforementioned servers are divided into several server groups, for any server group within those groups, and for the data collection items involved in the servers within that group (i.e., the data collection items involved in that server group), the statistical results of the number of times the servers in that group collect data under that specific data collection item can be calculated from the aforementioned data collection statistics. Therefore, based on these statistical results, it is possible to detect whether the data collection behavior of that server group for that specific data collection item is abnormal.
[0050] Next, in step 206, based on the first statistical results, it can be detected whether the data collection behavior of the aforementioned servers for the first collection item is abnormal.
[0051] As an example, the first statistical result may include the calculated mean and variance, and in step 206, the following can be performed: Figure 3 The detection process is shown. This detection process includes:
[0052] Step 302: Obtain the second statistical results associated with the aforementioned servers, including the historical average, which is calculated based on the number of historical collections of the aforementioned servers under the first collection item.
[0053] Step 304: Calculate the first difference between the mean of the first statistical result and the historical mean;
[0054] Step 306: If the first difference does not exceed the second threshold and the variance in the first statistical result does not exceed the first threshold, then it is determined that the data collection behavior of several servers for the first collection item is normal.
[0055] The second statistical result is a historical statistical result related to the aforementioned servers and the first collection item. This historical statistical result may be calculated in the previous detection period, for example. The second statistical result includes a historical mean, which is calculated based on the number of historical collections under the first collection item for the aforementioned servers. Of course, the second statistical result may also include a historical variance, which can also be calculated based on the number of historical collections.
[0056] It should be noted that the mean reflects the overall average, while the variance reflects individual outliers. The second threshold measures whether the mean fluctuates, and the first threshold measures whether the variance is close to zero. Specifically, when the mean exceeds the second threshold, it indicates that the mean fluctuates. When the variance exceeds the first threshold, it indicates that the variance is not close to zero, thus reflecting the presence of individual outliers. The specific values of the first and second thresholds can be set according to actual needs and are not specifically limited here.
[0057] Based on this, in step 306, if the first difference does not exceed the second threshold and the variance in the first statistical result does not exceed the first threshold, it can be indicated that the mean in the first statistical result has no fluctuation and the variance in the first statistical result is close to 0. The overall data collection behavior of the aforementioned servers is normal and stable, thus confirming that the data collection behavior of the aforementioned servers for the first data collection item is normal. It should be noted that the detection result "the data collection behavior of the aforementioned servers for the first data collection item is normal" obtained here reflects that the overall data collection behavior of the aforementioned servers is normal.
[0058] By execution Figure 3 The detection process shown can be compared horizontally, that is, compared with previous historical statistical results.
[0059] As another example, the first statistical result may include the calculated variance, which can be compared with the first threshold as described above in step 206. If the variance exceeds the first threshold, it indicates that the variance is not close to 0, and that there are individual outliers among the aforementioned servers, thus determining that the data collection behavior of the aforementioned servers regarding the first collection item is abnormal. It should be noted that the detection result of "abnormal data collection behavior of the aforementioned servers regarding the first collection item" is used to reflect the overall abnormal collection behavior of the aforementioned servers.
[0060] The two detection methods listed above are for detecting whether the overall data collection behavior of the aforementioned servers is abnormal. In practice, it is also possible to detect whether a single server exhibits abnormal data collection behavior, thus identifying the malfunctioning server.
[0061] Specifically, the first statistical result may include the calculated mean. In step 206, for any server among the aforementioned servers, such as any server, the following can be executed on that server: Figure 4 The detection process is shown. This detection process includes the following steps:
[0062] Step 402: Calculate the second difference between the target number of collections by the server and the mean in the first statistical result, wherein the target number of collections is the number of collections by the server under the first collection item in the above collection statistics information;
[0063] Step 404: If the second difference exceeds the third threshold, it is determined that the server's data collection behavior for the first collection item is abnormal.
[0064] The third threshold is used to measure whether the number of data collections is close to the mean, that is, whether the number of data collections is the same as or close to the mean. When the number of data collections is close to the mean, it indicates that the data collection behavior is normal; otherwise, it indicates that the data collection behavior is abnormal. Based on this, in step 404, when the second difference exceeds the third threshold, it can be indicated that the server's target number of data collections is not close to the mean in the first statistical result, thus determining that the server's data collection behavior for the first data collection item is abnormal. In addition, the server can be identified as an abnormal server.
[0065] By execution Figure 4 The detection process shown allows for longitudinal comparisons, such as comparing the collected data and the mean value in the same dimension, which can help locate abnormal servers.
[0066] It should be noted that the various detection methods listed in step 206 above can be combined with each other, and no specific limitations are made here. For example, while performing anomaly detection on the overall data collection behavior of the aforementioned servers, anomaly detection can also be performed on the data collection behavior of a single server.
[0067] Figure 2 The corresponding embodiment provides a solution that can obtain collection statistics from multiple servers within a target time period. These multiple servers involve several collection items, and the collection statistics include the number of times each server collects data under at least one of these collection items. Then, for a first collection item, a first statistical result can be calculated based on the number of times each server collects data under the first collection item in the collection statistics. Based on the first statistical result, it can be used to detect whether the data collection behavior of the servers for the first collection item is abnormal. It should be noted that the first statistical result may include, for example, the calculated mean and / or variance. The mean reflects the overall average, while the variance reflects individual outliers. By detecting whether the server's data collection behavior is abnormal based on the mean and / or variance, effective anomaly detection can be performed on the server's data collection behavior, obtaining more accurate detection results. Moreover, compared to existing technologies, complex detection rule configuration can be avoided, thus preventing inaccurate or omitted detection rule configurations.
[0068] In one embodiment, after performing the step of detecting whether the data collection behavior of the aforementioned servers for the first collection item is abnormal, if the detection result is that the data collection behavior of at least one of the aforementioned servers is abnormal, the number of collections of the aforementioned servers under the first collection item can be grouped and / or sorted for statistical purposes. The resulting statistical results can facilitate relevant maintenance personnel to observe the distribution of the number of collections of the servers.
[0069] Specifically, during group statistics, in response to a detection result indicating abnormal data collection behavior of at least one of the aforementioned servers, a collection count set can be obtained. This set is formed by the collection counts of the aforementioned servers under the first collection item in the collection statistics information. Then, for each collection count in this collection count set, the number of servers corresponding to that collection count can be determined. This number of servers is the number of servers among the aforementioned servers associated with that collection count. Finally, a first detection result can be generated, which includes the collection count in the collection count set and the number of servers corresponding to that collection count.
[0070] In practice, the aforementioned collected statistical information may include multiple sub-data items, which may include a server identifier, the server identifier indicating which of the aforementioned collection items the server is involved in, and the number of times the server collects data under that specific collection item.
[0071] During the sorting and statistical analysis, in response to a detection result indicating abnormal data collection behavior from at least one of the aforementioned servers, several sub-data items can be obtained from the multiple sub-data items based on the server identifiers of the aforementioned servers and the first collection item. Each of these sub-data items includes the first collection item and the server identifier of one of the aforementioned servers. Then, these sub-data items can be sorted according to the number of collections, for example, in ascending or descending order. Finally, a second detection result can be generated, which includes the sorted sub-data items.
[0072] In one embodiment, both the first detection result and the second detection result may further include the detection result obtained by performing the step of detecting whether the data collection behavior of the aforementioned servers for the first collection item is abnormal.
[0073] In one embodiment, when the aforementioned multiple servers are servers within an Internet data center, the first detection result and / or the second detection result can also be provided to the Internet data center, which facilitates the operation and maintenance personnel in the Internet data center to understand the relevant situation of server data collection behavior in a timely manner.
[0074] In one example, the internet data center may have a target group chat created on a target communication platform, through which the first detection result and / or the second detection result can be sent. The members of the target group chat may include the internet data center's operations and maintenance personnel.
[0075] The detection scheme described above can quickly and automatically detect the stability of each data collection item across a massive number of servers, and locate servers with abnormal or unstable data collection, thereby assisting operations and maintenance personnel in following up and resolving related issues.
[0076] Further reference Figure 5 This specification provides an embodiment of an anomaly detection device for server data collection behavior, which can be applied to, for example... Figure 1 The detection system shown.
[0077] like Figure 5As shown, the anomaly detection device 500 for server data collection behavior in this embodiment includes: an acquisition unit 501, a calculation unit 502, and a detection unit 503. The acquisition unit 501 is configured to acquire collection statistics information of multiple servers within a target time period. These multiple servers involve several collection items, and the collection statistics information includes the number of times each of the multiple servers collects data under at least one of the several collection items. The calculation unit 502 is configured to calculate a first statistical result for the number of times the multiple servers collect data under the first collection item in the collection statistics information. The detection unit 503 is configured to detect whether the data collection behavior of the multiple servers for the first collection item is abnormal based on the first statistical result.
[0078] In some embodiments, the first statistical result may include the calculated variance; and the detection unit 503 may be further configured to: if the variance in the first statistical result exceeds a first threshold, determine that the data collection behavior of the aforementioned servers for the first collection item is abnormal.
[0079] In some embodiments, the first statistical result may further include a calculated mean; and the detection unit 503 may be further configured to: acquire a second statistical result associated with the plurality of servers, including a historical mean, which is calculated based on the historical number of collections by the plurality of servers under the first collection item; calculate a first difference between the mean in the first statistical result and the historical mean; if the first difference does not exceed a second threshold and the variance in the first statistical result does not exceed the first threshold, then determine that the data collection behavior of the plurality of servers for the first collection item is normal.
[0080] In some embodiments, the first statistical result may include the calculated mean; and the detection unit 503 may be further configured to: for a server among the aforementioned servers, calculate a second difference between its target number of collections and the mean in the first statistical result; if the second difference exceeds a third threshold, determine that the server's data collection behavior for the first collection item is abnormal, wherein the target number of collections is the number of times the server collects data under the first collection item in the aforementioned collection statistics.
[0081] In some embodiments, the acquisition unit 501 may be further configured to: acquire the collection data of the plurality of servers within a target time period, including the collection content of each of the plurality of servers under at least one of the plurality of collection items; for each of the plurality of servers and the collection item involved in the plurality of collection items, determine the number of collection content items of the server under the collection item in the collection data, and determine the number of items as the number of times the server collects under the collection item; and generate the collection statistics information based on the number of times of collection determined for each of the plurality of servers.
[0082] In some embodiments, the plurality of servers may all have attribute values under the target server attribute; and the apparatus 500 may further include: a grouping unit (not shown in the figure), configured to divide the plurality of servers into several server groups according to the target server attribute; the several servers are specifically the target server group among the several server groups, and the servers in the target server group are related to the first collection item.
[0083] In some embodiments, the apparatus 500 may further include a determining unit (not shown in the figure) and a first generating unit (not shown in the figure). The acquiring unit 501 may also be configured to: after the detection unit 503 detects whether the data collection behavior of the plurality of servers for the first collection item is abnormal, in response to the detection result that the data collection behavior of at least one of the plurality of servers is abnormal, acquire a collection count set, which is formed by the collection count of the plurality of servers under the first collection item in the collection statistics information; the determining unit may be configured to: for the collection count in the collection count set, determine the number of servers corresponding to the collection count, wherein the number of servers is the number of servers associated with the collection count among the plurality of servers; the first generating unit may be configured to: generate a first detection result, which includes the collection count in the collection count set and the number of servers corresponding to the collection count.
[0084] In some embodiments, the aforementioned collected statistical information may include multiple sub-data items, wherein the sub-data items include a server identifier, the server identifier indicating which specific collection item the server is involved in among the aforementioned plurality of collection items, and the number of times the server collects data under that specific collection item. The aforementioned apparatus 500 may further include a sorting unit (not shown in the figure) and a second generation unit (not shown in the figure). The acquisition unit 501 may also be configured to, after the detection unit 503 detects whether the data collection behavior of the aforementioned plurality of servers for the first collection item is abnormal, in response to the detection result indicating that the data collection behavior of at least one of the aforementioned plurality of servers is abnormal, acquire several sub-data items from the aforementioned multiple sub-data items based on the server identifiers of the aforementioned plurality of servers and the first collection item; the sorting unit may be configured to sort the several sub-data items according to the number of collections; the second generation unit may be configured to generate a second detection result, which includes the sorted several sub-data items.
[0085] exist Figure 5 In the corresponding embodiments, the specific processing of each unit and the resulting technical effects can be found in the relevant descriptions of the method embodiments above, and will not be repeated here.
[0086] This specification also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the anomaly detection methods for server data collection behavior described in the above method embodiments.
[0087] This specification also provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the anomaly detection methods for server data collection behavior described in the above method embodiments.
[0088] This specification also provides a computer program in its embodiments, wherein when the computer program is executed in a computer, it causes the computer to perform the anomaly detection methods for server data collection behavior described in the above method embodiments.
[0089] Those skilled in the art will recognize that the functions described in the various embodiments disclosed in this specification in one or more of the examples above can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.
[0090] In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0091] The specific embodiments described above further illustrate the purpose, technical solutions, and beneficial effects of the multiple embodiments disclosed in this specification. It should be understood that the above descriptions are merely specific implementations of the multiple embodiments disclosed in this specification and are not intended to limit the protection scope of the multiple embodiments disclosed in this specification. Any modifications, equivalent substitutions, improvements, etc., made based on the technical solutions of the multiple embodiments disclosed in this specification should be included within the protection scope of the multiple embodiments disclosed in this specification.
Claims
1. A method for detecting anomalies in server data collection behavior, comprising: Collect statistical information of multiple servers within a target time period, wherein the multiple servers involve several collection items, and the collection statistics include the number of times each of the multiple servers collects data under at least one of the several collection items, wherein each of the multiple servers has attribute values under the target server attribute. Based on the target server attributes, the multiple servers are divided into several server groups, wherein each server in any server group has the same attribute value under the target server attributes. For the first collection item among the plurality of collection items, calculate the first statistical result for the number of collections under the first collection item for several servers belonging to the same server group in the collection statistics information; Based on the first statistical results, detect whether the data collection behavior of the plurality of servers for the first collection item is abnormal.
2. The method of claim 1, wherein, The first statistical result includes the calculated variance; as well as The detection of whether the data collection behavior of the plurality of servers for the first collection item is abnormal includes: If the variance in the first statistical result exceeds the first threshold, then it is determined that the data collection behavior of the plurality of servers for the first collection item is abnormal.
3. The method of claim 2, wherein, The first statistical result also includes the calculated mean; as well as The step of detecting whether the data collection behavior of the plurality of servers for the first collection item is abnormal also includes: Obtain a second statistical result associated with the plurality of servers, including a historical average, which is calculated based on the number of historical collections of the plurality of servers under the first collection item; Calculate the first difference between the mean of the first statistical result and the historical mean; If the first difference does not exceed the second threshold, and the variance in the first statistical result does not exceed the first threshold, then it is determined that the data collection behavior of the plurality of servers for the first collection item is normal.
4. The method of claim 1, wherein, The first statistical result includes the calculated mean; as well as The detection of whether the data collection behavior of the plurality of servers for the first collection item is abnormal includes: For each of the servers, a second difference is calculated between the target number of data collections and the mean of the first statistical results. If the second difference exceeds a third threshold, it is determined that the server's data collection behavior for the first collection item is abnormal. The target number of data collections is the number of times the server collects data under the first collection item in the collection statistics.
5. The method of claim 1, wherein, The acquisition of statistical information collected from multiple servers within a target time period includes: Acquire the data collected by the plurality of servers within the target time period, including the data collected by each of the plurality of servers under at least one of the plurality of collection items; For each of the plurality of servers, and for each collection item that the server is involved in among the plurality of collection items, determine the number of collection items that the server collects under that collection item in the collection data, and determine the number of collection items as the number of times the server collects under that collection item; The collection statistics are generated based on the number of collections determined for each of the multiple servers.
6. The method according to one of claims 1 to 5, wherein After detecting whether the data collection behavior of the plurality of servers for the first collection item is abnormal, the method further includes: In response to a detection result indicating that at least one of the servers exhibits abnormal data collection behavior, a collection count set is obtained, which is formed by the collection count of the servers under the first collection item in the collection statistics information. For each collection count in the collection count set, the number of servers corresponding to that collection count is determined, where the number of servers is the number of servers associated with that collection count among the plurality of servers; A first detection result is generated, which includes the number of collections in the collection set and the number of servers corresponding to the collection count.
7. The method according to any one of claims 1-5, wherein, The collected statistical information includes multiple sub-data items, including a server identifier, which indicates the server involved in a certain collection item among the several collection items, and the number of times the server collected data under that certain collection item. as well as After detecting whether the data collection behavior of the plurality of servers for the first collection item is abnormal, the method further includes: In response to a detection result indicating that at least one of the servers exhibits abnormal data collection behavior, several sub-data items are obtained from the plurality of sub-data items based on the server identifiers of the servers and the first collection item. The sub-data are sorted according to the number of times they are collected; A second detection result is generated, which includes the sorted sub-data.
8. An anomaly detection device for server data collection behavior, comprising: The acquisition unit is configured to acquire collection statistics information of multiple servers within a target time period, wherein the multiple servers involve several collection items, and the collection statistics information includes the number of times each of the multiple servers has collected data under at least one of the several collection items, wherein each of the multiple servers has attribute values under the target server attribute. The calculation unit is configured to divide the plurality of servers into several server groups according to the target server attributes, wherein each server in any server group has the same attribute value under the target server attributes; for the first collection item among the plurality of collection items, a first statistical result is calculated on the number of collections of several servers belonging to the same server group under the first collection item in the collection statistics information; The detection unit is configured to detect whether the data collection behavior of the plurality of servers for the first collection item is abnormal, based on the first statistical result.
9. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed in the computer, it causes the computer to perform the method of any one of claims 1-7.
10. A computing device comprising a memory and a processor, wherein, The memory stores executable code, and when the processor executes the executable code, it implements the method of any one of claims 1-7.
11. A computer program, characterized in that, in, When the computer program is executed in the computer, it causes the computer to perform the method of any one of claims 1-7.
Citation Information
Patent Citations
Abnormal data detection method and device and server
CN110377447A