A method, device, storage medium and electronic device for locating abnormality

Abstract

The present specification discloses a method, device, storage medium and electronic device for locating anomalies, which can obtain data to be detected, and combine the dimensional values ​​contained in different data dimensions to obtain a dimensional value combination. Then, for each dimensional value combination, the influence degree and discrimination degree corresponding to the dimensional value combination can be determined. The influence degree is used to characterize the degree to which the data hit by the dimensional value combination in the data to be detected causes anomalies in the data to be detected. The discrimination degree is used to characterize the degree of difference between the actual data distribution of the data hit by the dimensional value combination in the data to be detected and the standard data distribution corresponding to the determined dimensional value combination. According to the influence degree and discrimination degree corresponding to each dimensional value combination, the target dimensional value combination is determined, and the anomaly is located according to the target dimensional value combination, thereby improving the comprehensiveness and accuracy of locating the cause of the anomaly.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, storage medium and electronic device for locating anomalies. Background Art

[0002] In actual applications, there may be a variety of statistical indicators, such as transaction volume, page click volume, etc., and statistical indicators can often be counted through multiple data dimensions. For example, for the statistical indicator of transaction volume, statistics can be performed through data dimensions such as region, merchant, and computer room. When a statistical indicator is abnormal, it is often necessary to find out the cause of the abnormality of the statistical indicator, and based on the cause, promptly repair the problem to prevent privacy data leakage, such as determining which computer room has a fault, or determining which region's data causes the abnormality of the statistical indicator.

[0003] In the prior art, in the process of determining the possible causes of abnormalities in the total amount of statistical indicators, a dimension value under one dimension is often eventually located.

[0004] For example, when counting the number of transactions, the total number of transactions can be counted, or the number of transactions can be counted for each dimension value. The dimension values ​​included in the region dimension can be region A, region B, etc., and the dimension values ​​included in the computer room dimension can be computer room 1, computer room 2, etc. The business platform can count the number of transactions generated in region A, and can also count the number of transactions supported by computer room 1. When the total number of transactions is abnormal (such as a sudden increase or decrease), then under the existing technology, a certain computer room under the computer room dimension can be located, which may be the cause of the abnormality in the total amount of statistical indicators.

[0005] However, in reality, the cause of the abnormality in the total amount of statistical indicators may be a combination of multi-dimensional factors. For example, the cause may need to be located in a certain area or a certain computer room. Therefore, investigation through existing technologies may be inaccurate or unable to comprehensively investigate all factors that may cause the abnormality.

[0006] Therefore, how to improve the accuracy and comprehensiveness of abnormal cause investigation is an urgent problem to be solved. Summary of the invention

[0007] This specification provides a method, device, storage medium and electronic device for locating anomalies to improve the accuracy of anomaly detection.

[0008] This manual adopts the following technical solutions:

[0009] This specification provides a method for locating anomalies, including:

[0010] Acquire the data to be detected, and combine the dimensional values ​​contained in different data dimensions to obtain dimensional value combinations, wherein each dimensional value combination contains at least two dimensional values, and each dimensional value comes from a different data dimension;

[0011] For each dimensional value combination, determine the influence degree and discrimination degree corresponding to the dimensional value combination according to the data to be detected, the influence degree is used to characterize the influence degree of the abnormality of the data hit by the dimensional value combination in the data to be detected on the abnormality of the entire amount of the data to be detected, and the discrimination degree is used to characterize the degree of difference between the actual data distribution of the data hit by the dimensional value combination in the data to be detected and the standard data distribution corresponding to the dimensional value combination;

[0012] According to the influence degree and discrimination degree corresponding to each dimensional value combination, the target dimensional value combination is determined, and the anomaly is located based on the target dimensional value combination.

[0013] Optionally, determining the influence degree corresponding to the dimension value combination according to the data to be detected includes:

[0014] Predicting data that should be generated within a time period for generating the data to be detected as first predicted data, and predicting data that should be generated by the combination of dimension values ​​within the time period as second predicted data;

[0015] Determine, according to the first prediction data, the amount of abnormal data in the data to be detected as the first amount of abnormal data, and, according to the second prediction data, determine the amount of abnormal data in the data hit by the dimension value combination in the data to be detected as the second amount of abnormal data;

[0016] The impact degree corresponding to the dimensional value combination is determined according to the proportion of the second abnormal data volume in the first abnormal data volume.

[0017] Optionally, determining the degree of discrimination corresponding to the dimension value combination according to the data to be detected includes:

[0018] Predicting data that should be generated within a time period for generating the data to be detected as first predicted data, and predicting data that should be generated by the combination of dimension values ​​within the time period as second predicted data;

[0019] Determine the actual data distribution according to the proportion of the data hit by the dimension value combination in the data to be detected in the data to be detected, and determine the standard data distribution according to the proportion of the second predicted data in the first predicted data;

[0020] The degree of discrimination is determined according to the actual data distribution and the standard data distribution.

[0021] Optionally, predicting data that should be generated within a time period for generating the data to be detected as first predicted data, and predicting data that should be generated by the dimension value combination within the time period as second predicted data, includes:

[0022] Acquire data in a time period adjacent to the time period as reference data;

[0023] The first prediction data and the second prediction data are obtained according to the reference data.

[0024] Optionally, according to the influence degree and discrimination degree corresponding to each dimensional value combination, the target dimensional value combination is determined, including:

[0025] For each determined dimension combination, according to the influence degree corresponding to each dimension value combination contained in the dimension combination, a candidate dimension value combination under the dimension combination is screened out, wherein the dimension combination includes at least two data dimensions;

[0026] Determine the target dimension value combination based on the discrimination corresponding to each candidate dimension value combination.

[0027] Optionally, according to the influence degree corresponding to each dimension value combination included in the dimension combination, the candidate dimension value combinations under the dimension combination are screened out, including:

[0028] The dimension value combinations included in the dimension combination are sorted from large to small according to the degree of distinction, and the dimension value combinations whose influence degree is not less than the determined preset influence degree are sequentially added to the set corresponding to the dimension combination in the order of sorting, until the total influence degree corresponding to the dimension value combinations added to the set is not less than the determined preset total amount, thereby obtaining the target set corresponding to the dimension combination;

[0029] The dimension value combination in the target set is used as a candidate dimension value combination under the dimension combination.

[0030] Optionally, determining the preset impact degree and the preset total amount includes:

[0031] Predicting data that should be generated within a time period of generating the data to be detected as first predicted data, and determining an amount of abnormal data in the data to be detected based on the first predicted data as a first amount of abnormal data;

[0032] The preset impact degree and the preset total amount are determined according to the first abnormal data amount.

[0033] This specification provides a device for abnormal location, including:

[0034] An acquisition module is used to acquire the data to be detected, and to combine the dimensional values ​​contained in different data dimensions to obtain dimensional value combinations, wherein each dimensional value combination contains at least two dimensional values, and each dimensional value comes from a different data dimension;

[0035] A determination module, for determining, for each dimensional value combination, the degree of influence and the degree of discrimination corresponding to the dimensional value combination according to the data to be detected, wherein the degree of influence is used to characterize the degree of influence of the abnormality of the data hit by the dimensional value combination in the data to be detected on the abnormality of the entire amount of the data to be detected, and the degree of discrimination is used to characterize the degree of difference between the actual data distribution of the data hit by the dimensional value combination in the data to be detected and the standard data distribution corresponding to the dimensional value combination;

[0036] The positioning module is used to determine the target dimension value combination according to the influence degree and discrimination degree corresponding to each dimension value combination, and to locate the anomaly according to the target dimension value combination.

[0037] This specification provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned abnormality locating method is implemented.

[0038] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned abnormality locating method when executing the program.

[0039] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0040] In the method for locating anomalies provided in the present specification, data to be detected can be obtained, and the dimensional values ​​contained in different data dimensions can be combined with each other to obtain a dimensional value combination, wherein each dimensional value combination contains at least two dimensional values, and each dimensional value comes from a different data dimension. Then, for each dimensional value combination, according to the data to be detected, the degree of influence and the degree of discrimination corresponding to the dimensional value combination can be determined. The degree of influence is used to characterize the degree of influence of the anomaly of the data hit by the dimensional value combination in the data to be detected on the anomaly of the entire amount of data to be detected. The degree of discrimination is used to characterize the actual data distribution of the data hit by the dimensional value combination in the data to be detected, and the degree of difference between the standard data distribution corresponding to the determined dimensional value combination. According to the degree of influence and the degree of discrimination corresponding to each dimensional value combination, the target dimensional value combination is determined, and the anomaly is located according to the target dimensional value combination.

[0041] From the above content, it can be seen that the method of anomaly location provided in this specification can combine the degree of influence and discrimination corresponding to each dimensional value combination to determine which dimensional value combination may be the cause of the anomaly in the data to be detected. Compared with the existing technology, which usually only locates a single factor, this solution can more comprehensively and accurately locate the cause of the anomaly to several dimensional value combinations, that is, to locate the factor combination under multiple dimensions, thereby making it more convenient for relevant technical personnel to locate the anomaly. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0043] Figure 1 A flowchart of a method for locating an abnormality in this specification;

[0044] Figure 2 A schematic diagram of a method for screening combinations of values ​​of various dimensions using the transaction quantity as an example provided in this specification;

[0045] Figure 3 A schematic diagram of a device for locating anomalies provided in this specification;

[0046] Figure 4 A method corresponding to the Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0048] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0049] Figure 1 The following is a flow chart of a method for locating an abnormality in this specification, which specifically includes the following steps:

[0050] S100: Acquire data to be detected, and combine dimensional values ​​contained in different data dimensions to obtain dimensional value combinations, wherein each dimensional value combination contains at least two dimensional values, and each dimensional value comes from a different data dimension.

[0051] In many business scenarios, a large amount of data is generated in the business platform. Statistical indicators can be used to monitor whether the business is running normally. The statistical indicators mentioned here can refer to indicators for business volume statistics for any business scenario. For example, the statistical indicator can be the number of transactions in the business platform. For another example, the statistical indicator can be the number of page clicks on the business platform by users.

[0052] In some cases, the business platform needs to determine what causes the fluctuation in the total amount of statistical indicators. For example, if the total amount of a statistical indicator is abnormal, such as a sudden and sharp increase or decrease, the business platform needs to locate the cause that may have caused the abnormality in the total amount of the statistical indicator as soon as possible. For another example, when the total amount of a statistical indicator changes to a certain extent (not necessarily abnormal), the relevant personnel of the business platform need to locate the cause of the change in the statistical indicator, and this method can also be applied.

[0053] Based on this, the business platform can obtain the data to be tested, and combine the dimension values ​​contained in different data dimensions to obtain a dimension value combination, wherein each dimension value combination contains at least two dimension values, each dimension value comes from a different data dimension, and the data dimension is the dimension used to count the data to be tested, that is, the dimension values ​​of multiple different data dimensions can be combined together to obtain a dimension value combination.

[0054] The concept of dimension combination can also be introduced here. A dimension combination can include multiple different data dimensions. For each dimension value combination under a dimension combination, the dimension value combination is composed of a dimension value in each data dimension included in the dimension combination, wherein at least some of the dimension values ​​included in different dimension value combinations are different.

[0055] The data to be detected mentioned here may refer to business data related to a statistical indicator within a period of time. For example, when the statistical indicator is the number of transactions, the data to be detected may refer to transaction data within a period of time when the total transaction volume is abnormal.

[0056] The data dimensions mentioned above may be some data dimensions artificially divided for the data to be detected, or may be at least some dimensions related to the statistical indicator in the business scenario. Here, the number of transactions is still used as a statistical indicator to illustrate each data dimension, as shown in Table 1 below.

[0057] time area Operators Data Center Payment Methods Number of transactions 10：00 Region A Operator 1 Data Center 1 Payment Method 1 2 10：00 Region A Operator 2 Data Center 2 Payment Method 2 3 … … … … … … 10：01 Region B Operator 3 Data Center 3 Payment Method 2 5

[0058] Table 1

[0059] It can be seen from Table 1 that the data dimensions for the statistical indicator of transaction quantity may include four dimensions: region, operator, data center and payment method. The region dimension includes dimension values ​​such as region A and region B. The operator dimension includes dimension values ​​such as operator 1, operator 2 and operator 3. The data center dimension includes dimension values ​​such as data center 1, data center 2 and data center 3. The payment method dimension includes dimension values ​​such as payment method 1 and payment method 2. The data in the above Table 1 may refer to the data to be tested when monitoring the statistical indicator of transaction quantity. It can be seen that the data to be tested is the data related to a statistical indicator within a certain period of time. Taking the example of Table 1 as an example, the transaction quantity is directly counted according to province, operator, data center and payment method. At 10:00, some users paid in region A through payment method 1, using operator 1, and 2 transactions supported by data center 1 of the business platform occurred. Some users paid by transfer in region A and connected in Beijing, and 2 transactions supported by data center 2 of the business platform occurred.

[0060] Still taking the above Table 1 as an example, assuming that province, operator, data center and payment method are all data dimensions used to count the number of transactions, then by combining any several data dimensions in the form of the data to be detected in the above Table 1, a dimension combination can be obtained, and then a transaction quantity can be counted under each dimension value combination under the dimension combination. For example, the two data dimensions of province and operator are combined to obtain a dimension combination (region, operator). There can be several dimension value combinations under this dimension combination: (region A, operator 1), (region A, operator 2), (region B, operator 1), (region B, operator 2), etc. Of course, the dimension value under each dimension can also be determined, and the dimension value under a single dimension can also be determined. The dimension value under a single dimension can also be regarded as a dimension value combination, and the dimension value leading to the indicator can be determined by calculating the degree of influence and the degree of differentiation mentioned below, that is, (region) itself can be regarded as a dimension combination, (operator) itself can also be regarded as a dimension combination, and the dimension value combination included in the dimension combination of region can include any dimension value under the region dimension, such as (region 1), (region 2), (region 3), etc.

[0061] S102: For each dimensional value combination, determine the degree of influence and the degree of discrimination corresponding to the dimensional value combination according to the data to be detected, wherein the degree of influence is used to characterize the degree of influence of the anomaly of the data hit by the dimensional value combination in the data to be detected on the anomaly of the entire amount of the data to be detected, and the degree of discrimination is used to characterize the degree of difference between the actual data distribution of the data hit by the dimensional value combination in the data to be detected and the standard data distribution corresponding to the dimensional value combination.

[0062] S103: Determine a target dimensional value combination according to the influence degree and discrimination degree corresponding to each dimensional value combination, and perform anomaly location according to the target dimensional value combination.

[0063] After determining that there are multiple dimension value combinations, it is necessary to locate which dimension value combination is the dimension value combination that may cause the abnormality of the data to be detected (or cause fluctuations in the data to be detected) as the target dimension value combination. After determining the target dimension value combination, the abnormality can be located based on the obtained target dimension value combination (that is, the cause of the abnormality is located, that is, the given target dimension value combination may be the cause of the abnormality of the data to be detected). For example, if (region 1, operator 1), (data center 1, payment method 1), and (operator 2, data center 1) are determined as the target dimension value combinations, then these target dimension value combinations may be the causes of the abnormality of the data to be detected. The cause of the abnormality can be further determined based on these dimension value combinations, or operations such as abnormal repair can be performed.

[0064] Among them, for each dimensional value combination, the degree of influence and the degree of discrimination corresponding to the dimensional value combination can be determined according to the data to be detected. The degree of influence is used to characterize the degree of influence of the anomaly of the data hit by the dimensional value combination in the data to be detected on the entire amount of data to be detected. The degree of discrimination is used to characterize the degree of difference between the actual data distribution of the data hit by the dimensional value combination in the data to be detected and the standard data distribution corresponding to the dimensional value combination. Then, according to the degree of influence and the degree of discrimination corresponding to each dimensional value combination, the target dimensional value combination can be determined, and the anomaly can be located based on the target dimensional value combination.

[0065] The above-mentioned impact degree can be calculated as the proportion of the anomalies that may be generated under the dimension value combination to the total anomalies of the data to be detected. For example, the number of transactions generated under the dimension value combination (region A, operator 1) is actually 50, while the number of transactions that would have been generated under ideal circumstances (such as no anomalies) should be 100, 100-50=50, so for the dimension value combination (region A, operator 1), its own anomaly is 50. Through statistics of the data to be detected, it is determined that the total number of transactions is 500, and under ideal circumstances, 700 transactions will be generated, so the total anomaly is 200, and the corresponding impact degree of (Beijing, Mobile) is 25% (i.e., 50 / 200).

[0066] The discrimination can indirectly indicate the possibility of abnormal data in the dimension value combination. From the above definition, it can be seen that the discrimination is the difference between the actual data distribution corresponding to the data hit in the data to be detected in the dimension value combination and the standard data distribution of the dimension value combination. The actual data distribution can indirectly indicate the probability of the data in the dimension value combination actually appearing in the data to be detected, and the standard data distribution can refer to the probability of the data generated by the dimension value combination under ideal conditions appearing in the overall data. For example, if the statistical indicator corresponding to the data to be detected is the number of transactions, the actual data distribution is the probability distribution of the transaction volume generated under the dimension value combination appearing in the total transaction volume. The standard data distribution is the probability distribution of the transaction volume generated under the dimension value combination appearing in the total transaction volume under standard conditions. The greater the difference between the actual data distribution and the standard data distribution, the greater the difference between the actual and ideal conditions of the transaction volume generated under the dimension value combination, and the smaller the difference between the actual data distribution and the predicted data distribution, the smaller the difference between the actual and ideal conditions of the transaction volume generated under the dimension value combination. Therefore, the discrimination can also be used to determine whether the dimension value combination is the target dimension value combination.

[0067] For a dimension value combination, when calculating the degree of influence and the degree of discrimination, we can first predict the data that should be generated within the time period for generating the data to be detected as the first predicted data, and predict the data that the dimension value combination should generate within the time period as the second predicted data.

[0068] The first prediction data can be used to know the standard total business volume (F), and the second prediction data can be used to know the standard business volume (F) corresponding to the dimension value combination. ij ), where ij represents the jth dimension value combination under the i-th dimension combination, that is, represents the ijth dimension value combination, the actual total business volume (A) can be known through the data to be detected, and the actual business volume (A) corresponding to the dimension value combination can be known through the data corresponding to the dimension value combination in the data to be detected. ij ).

[0069] The standard total business volume (F) mentioned here may refer to the total business volume corresponding to the data to be tested under ideal conditions obtained by prediction (such as the number of transactions mentioned above), while the standard business volume (F ij ) may refer to the business volume of the data to be detected that can be counted under the ideal combination of dimension values ​​obtained by prediction.

[0070] When predicting the above-mentioned first prediction data and the second prediction data, business data in a time period adjacent to the time period corresponding to the data to be detected can be obtained as reference data, or business data within a preset time period can be obtained as reference data, and the predicted total amount and predicted business volume can be predicted based on the reference data.

[0071] Then, based on the first prediction data, the amount of abnormal data in the data to be detected can be determined as the first abnormal data amount, and based on the second prediction data, the amount of abnormal data in the data corresponding to the dimensional value combination in the data to be detected can be determined as the second abnormal data amount, and based on the proportion of the second abnormal data amount in the first abnormal data amount, the corresponding degree of influence of the dimensional value combination can be determined.

[0072] That is, the standard traffic volume (F ij ) and the actual business volume (A) obtained from the data corresponding to the target dimension value in the data to be detected ij ), determine the second abnormal data volume corresponding to the dimensional value combination, and determine the first abnormal data volume corresponding to the data to be detected according to the predicted total business volume (F) and the standard total business volume (A) corresponding to the data to be detected, and then determine the impact degree corresponding to the dimensional value combination according to the proportion of the second abnormal data volume corresponding to the dimensional value combination in the first abnormal data volume corresponding to the data to be detected.

[0073] The actual data distribution can be determined based on the proportion of data corresponding to the dimensional value combination in the data to be detected, and the standard data distribution can be determined based on the proportion of the second predicted data in the first predicted data, and the degree of differentiation can be determined based on the actual data distribution and the standard data distribution.

[0074] That is, the actual business volume (A ij ) in the actual total business volume (A) corresponding to the data to be tested, determine the actual data distribution, and according to the standard business volume (F ij ) in the standard total business volume (F), determine the standard data distribution, and then determine the discrimination corresponding to the dimension value combination based on the actual data distribution and the standard data distribution. The specific method for determining the discrimination can be achieved by calculating the js divergence.

[0075] The specific formula for calculating the degree of impact can be shown as follows:

[0076] EP ij =((A ij -F ij ) / (AF))

[0077] The specific formula for calculating the discrimination can be shown as follows:

[0078] p ij =F ij / F

[0079] q ij =A ij / A

[0080]

[0081] Among them, EP ij is the degree of influence, S ij is the discrimination, p ij is the standard data distribution, q ij is the actual data distribution.

[0082] After determining the influence degree and discrimination degree corresponding to any dimensional value combination, the target dimensional value combination can be determined based on the influence degree and discrimination degree corresponding to each dimensional value combination.

[0083] It should be noted that the reason why both the degree of influence and the degree of discrimination need to be considered when determining the target dimension value combination is that if the degree of influence corresponding to the dimension value combination is high, this dimension value combination may not necessarily be the cause of the abnormality. For example, the number of transactions under the dimension value combination (Beijing) and the number of transactions under the dimension value combinations (Beijing, China Unicom) (Beijing, China Mobile) (Beijing, QR code transfer) (Beijing, China Unicom, QR code transfer) influence each other, and the degree of influence of dimension value combinations containing the same dimension value will actually influence each other. When some of the degrees of influence are high, it may not be possible to distinguish which dimension value combinations are the possible causes of the abnormality. If a dimension value combination has a high degree of discrimination but a low degree of influence, it may not be the cause of the abnormality. For example, if the degree of discrimination corresponding to a dimension value combination is 80%, but the degree of influence is particularly low, only 0.2%, then even if the dimension value combination does have an abnormality, the dimension value combination will not be the cause of the abnormality.

[0084] Among them, there may be multiple methods for determining the target dimension value combination. For example, a dimension value combination whose influence and discrimination are both higher than a certain value can be used as the target dimension value combination. However, this method requires determining the influence and discrimination of all dimension value combinations at one time, which may be less efficient.

[0085] Therefore, a method is provided here that can improve efficiency to a certain extent, that is, for each determined dimension combination, the candidate dimension value combinations under the dimension combination can be screened out according to the influence degrees corresponding to the dimension value combinations contained in the dimension combination, and then the target dimension value combination can be determined according to the discrimination degrees corresponding to the screened candidate dimension value combinations, wherein the dimension combination includes at least one data dimension.

[0086] Specifically, during screening, the dimension value combinations contained in the dimension combination are sorted from large to small according to the degree of distinction, and in the order of sorting, the dimension value combinations whose influence degree is not less than the determined preset influence degree are added to the set corresponding to the dimension combination, until the total influence degree corresponding to the dimension value combinations added to the set is not less than the determined preset total amount, and the target set corresponding to the dimension combination is obtained, and then, the dimension value combinations in the target set can be used as candidate dimension value combinations under the dimension combination.

[0087] In other words, this method is that when screening, the candidate dimension value combinations in the dimension combination are screened out from each dimension combination in turn. Since all dimension value combinations contained in a dimension combination are mutually exclusive, the business volume under these dimension value combinations is the total statistical indicator corresponding to the data to be detected. Therefore, the influence degree corresponding to each dimension value combination under a dimension combination can also be superimposed. After arranging each dimension value combination from large to small according to the degree of discrimination, it will be judged from the largest degree of discrimination whether the influence degree corresponding to the dimension value combination is greater than the first threshold. This can exclude the dimension value combination with a smaller degree of influence, and when the total degree of influence of the dimension value combination in the set reaches a certain value, the dimension value combination that has not been put into the set will be excluded. This is equivalent to screening out some dimension value combinations with smaller influence and smaller degree of discrimination.

[0088] The above-mentioned mutual exclusion of all dimension value combinations in a dimension combination means that, for a dimension combination, the data under each dimension value combination contained in the dimension combination does not have an intersection. For example, for the dimension combination (operator, region), the dimension value combination may include (operator 1, region 1), (operator 2, region 1), (operator 1, region 2), (operator 2, region 1). The transaction data between these dimension value combinations does not have an intersection, and if these dimension value combinations are all dimension value combinations under the dimension combination, then the sum of the transaction volumes corresponding to these dimension value combinations is the total transaction volume corresponding to the data to be detected.

[0089] The above method of filtering dimension value combinations is similar to the method provided in this manual. Figure 2 in the way.

[0090] Figure 2 A schematic diagram of a method for filtering combinations of dimensional values ​​using transaction quantity as an example provided for this specification.

[0091] from Figure 2As can be seen from the figure, when the total transaction volume is abnormal, we can determine in turn whether the dimension value combination under each dimension combination is likely to be the target dimension value combination, that is, we can Figure 2 First, determine the value combination of each dimension under the region dimension: region 1, region 2, then determine the value combination of each dimension under the computer room dimension: computer room 1, computer room 2, and finally determine the value combination of each dimension under the payment method dimension: payment method 1, payment method 2. Figure 2 It can be seen that the transaction volumes under computer room 1 and payment method 1 may be abnormal. Therefore, we can further determine whether the dimension value combination under the dimension combination of computer room + payment method may contain the target dimension value combination.

[0092] Therefore, in actual applications, when screening candidate dimension value combinations, you can first screen the dimension value combinations under the dimension combination that contains one dimension, and then accumulate the number of dimensions in sequence, that is, the next time you can screen the dimension value combinations under the dimension combination that contains two dimensions, and then three. If, when screening the dimension value combinations under the dimension combination that contains one dimension, it is impossible to determine in this dimension combination that can serve as a candidate dimension value combination, then in the next screening, the data dimension corresponding to this dimension combination can be excluded, that is, the data dimension should not be included in the subsequent dimension combinations.

[0093] When the sets corresponding to each dimension combination are determined, the dimension value combinations contained in the sets corresponding to each dimension combination can be candidate dimension value combinations. After obtaining the candidate dimension value combinations, the dimension value combinations ranked in the top-n in terms of discrimination can be used as the target dimension value combinations. n can be a set value. For example, the top three dimension value combinations with the largest discrimination can be selected as the target dimension value combinations.

[0094] It should be noted that the above-mentioned first threshold and second threshold can be adaptively adjusted according to the degree of abnormality of the data to be detected, that is, the first threshold and the second threshold can be determined according to the amount of abnormality corresponding to the data to be detected, wherein the higher the total abnormality value, the higher the first threshold and the second threshold, and the lower the total abnormality value, the lower the first threshold and the second threshold.

[0095] It can be seen from the above method that this method can combine the degree of influence and discrimination corresponding to each dimensional value combination to determine which dimensional value combination may be the cause of the abnormality in the data to be detected. Compared with the existing technology that usually only locates a single factor, this solution can more comprehensively and accurately locate the cause to several dimensional value combinations, that is, to the combination of factors under multiple dimensions, thereby making it more convenient for relevant technical personnel to locate anomalies.

[0096] It should be noted that the method of calculating the discrimination corresponding to the dimension value combination is preferably the method of calculating the js divergence. Of course, it can also be calculated by other calculation methods. The reason why it is preferred to calculate by js divergence is proved by the following proof process. The discrimination corresponding to the dimension value combination calculated by js divergence satisfies that when a dimension value combination may be the target dimension value combination, the discrimination of the dimension value combination will be greater than other dimension value combinations containing each dimension value in the dimension value combination, or greater than the dimension value combination containing some dimension values ​​in the dimension value combination.

[0097] Before the proof, we first introduce the concepts of child nodes and parent nodes. For a dimension value combination, the parent node of the dimension value combination is the dimension value combination that contains some dimension values ​​in the dimension value combination, and the child node is the dimension value combination that contains each dimension value in the dimension value combination. For example, for the dimension value combination (region A, operator 1), the parent nodes of the dimension value combination are (region A), (operator 1), and the child nodes are (region A, operator 1, payment method 1).

[0098] Proof 1: The discrimination S corresponding to the cause node (the dimension value combination that causes the abnormality of the data to be detected, that is, the target dimension value combination) is greater than that of all its corresponding parent nodes.

[0099] When a certain dimension value combination is the target dimension value combination, the abnormal change propagates from the child node, and its δ = A ij -F ij , assuming δ<0, and vice versa.

[0100] The change of the parent node is equal to the child node, i.e., δ parent =δ child , and obviously, P parent >P cause ,q parent >q cause

[0101] It is easy to infer that the fluctuation of the parent node is smaller than that of the child node.

[0102] P cause -q cause >P parent -q parent

[0103] P cause / q cause >P parent / q parent (cond)

[0104]

[0105] According to formula (1) and inequality (cond), the discrimination S is given by the previous term With the latter The sum is composed of the preceding term greater than 0 and proportional to (pq) and p / q. When the cause occurs in the dimension value combination cause of a certain combination dimension, the preceding term of the cause node is greater than 0 and greater than the preceding term of the parent node. The subsequent term is less than 0, and the subsequent term of the root cause node is greater than the subsequent term of the parent node. Therefore, the discrimination corresponding to the dimension value combination cause is greater than the discrimination corresponding to its parent node. The proof is complete.

[0106] Proof 2: The discrimination S corresponding to the root node is greater than that of all its corresponding child nodes.

[0107] Obviously, When the root cause node occurs, the p and q of its child nodes change in equal proportion, that is,

[0108]

[0109] therefore, The discrimination corresponding to the root cause node is also greater than that of its child nodes.

[0110] The proof is complete.

[0111] It should also be noted that, from the above content, it can be seen that this method is mainly used to detect the target dimension value combination in each dimension value combination that causes the data abnormality. However, this method can also be used to detect whether a certain dimension value is the cause of the data abnormality, that is, the dimension value combination in this method can be replaced with a single dimension value. The method of determining the single dimension value that causes the data abnormality is basically the same as the above content, and will not be elaborated here.

[0112] The above is a method for locating an abnormality provided in one or more embodiments of this specification. Based on the same idea, this specification also provides a device for locating an abnormality, such as Figure 3 shown.

[0113] Figure 3 A schematic diagram of an abnormality location device provided in this specification specifically includes:

[0114] The acquisition module 301 is used to acquire the data to be detected, and to combine the dimensional values ​​contained in different data dimensions to obtain dimensional value combinations, wherein each dimensional value combination contains at least two dimensional values, and each dimensional value comes from a different data dimension;

[0115] A determination module 302 is used to determine, for each dimensional value combination, the influence degree and discrimination degree corresponding to the dimensional value combination according to the data to be detected, wherein the influence degree is used to characterize the influence degree of the abnormality of the data hit by the dimensional value combination in the data to be detected on the abnormality of the entire amount of the data to be detected, and the discrimination degree is used to characterize the degree of difference between the actual data distribution of the data hit by the dimensional value combination in the data to be detected and the standard data distribution corresponding to the dimensional value combination;

[0116] The positioning module 303 is used to determine the target dimension value combination according to the influence degree and discrimination degree corresponding to each dimension value combination, and perform anomaly positioning according to the target dimension value combination.

[0117] Optionally, the determination module 302 is specifically used to predict the data that should be generated within the time period for generating the data to be detected as the first prediction data, and to predict the data that should be generated by the dimension value combination within the time period as the second prediction data; determine the amount of abnormal data in the data to be detected based on the first prediction data as the first abnormal data amount, and determine the amount of abnormal data in the data hit by the dimension value combination in the data to be detected based on the second prediction data as the second abnormal data amount; determine the degree of influence corresponding to the dimension value combination based on the proportion of the second abnormal data amount in the first abnormal data amount.

[0118] Optionally, the determination module 302 is specifically used to predict the data that should be generated within the time period for generating the data to be detected as the first predicted data, and to predict the data that should be generated by the dimension value combination within the time period as the second predicted data; determine the actual data distribution according to the proportion of the data hit by the dimension value combination in the data to be detected in the data to be detected, and determine the standard data distribution according to the proportion of the second predicted data in the first predicted data; determine the discrimination based on the actual data distribution and the standard data distribution.

[0119] Optionally, the determination module 302 is specifically configured to obtain data in a time period adjacent to the time period as reference data; and obtain the first prediction data and the second prediction data according to the reference data.

[0120] Optionally, the detection module 303 is specifically used to, for each determined dimension combination, screen out candidate dimension value combinations under the dimension combination according to the influence degrees corresponding to the dimension value combinations contained in the dimension combination, wherein the dimension combination includes at least two data dimensions; and determine the target dimension value combination according to the discrimination corresponding to each candidate dimension value combination.

[0121] Optionally, the detection module 303 is specifically used to sort the dimension value combinations contained in the dimension combination from large to small according to the degree of distinction, and add the dimension value combinations whose influence degree is not less than the determined preset influence degree to the set corresponding to the dimension combination in the order of sorting, until the total influence degree corresponding to the dimension value combinations added to the set is not less than the determined preset total amount, so as to obtain the target set corresponding to the dimension combination; and use the dimension value combinations in the target set as candidate dimension value combinations under the dimension combination.

[0122] Optionally, the detection module 303 is specifically used to predict the data that should be generated within the time period of the data to be detected as the first predicted data, and determine the amount of abnormal data in the data to be detected as the first abnormal data amount based on the first predicted data; and determine the preset impact degree and the preset total amount based on the first abnormal data amount.

[0123] This specification also provides a computer-readable storage medium, which stores a computer program. The computer program can be used to execute the above-mentioned abnormality locating method.

[0124] This manual also provides Figure 4 The schematic structure diagram of the electronic device shown in FIG. Figure 4 As described above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and of course may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above-mentioned abnormality locating method. Of course, in addition to the software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0125] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0126] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0127] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0128] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0129] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0130] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0131] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0132] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0133] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0134] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0135] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0136] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0137] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0138] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing nodes connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage nodes.

[0139] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0140] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A method for locating anomalies, comprising: Acquire the data to be detected, and combine the dimension values ​​contained in different data dimensions to obtain the dimension value combinations, wherein each dimension value combination contains at least two dimension values, and each dimension value comes from a different data dimension, the data to be detected includes transaction data related to the statistical indicator within a period of time, and the data dimension includes relevant data under different dimensions involved in executing the transaction business; For each dimensional value combination, determine the degree of influence and discrimination corresponding to the dimensional value combination according to the data to be detected, the degree of influence is used to characterize the degree of influence of the abnormality of the data hit by the dimensional value combination in the data to be detected on the abnormality of the entire amount of the data to be detected, and the discrimination is used to characterize the degree of difference between the actual data distribution of the data hit by the dimensional value combination in the data to be detected and the standard data distribution corresponding to the dimensional value combination, and the standard data distribution represents the probability of the data generated by the dimensional value combination under ideal circumstances appearing in the overall data; According to the influence degree and discrimination degree corresponding to each dimensional value combination, the target dimensional value combination is determined, and the anomaly is located based on the target dimensional value combination.

2. The method according to claim 1, determining the influence degree corresponding to the dimension value combination according to the data to be detected, comprising: Predicting data that should be generated within a time period for generating the data to be detected as first predicted data, and predicting data that should be generated by the combination of dimension values ​​within the time period as second predicted data; Determine, according to the first prediction data, the amount of abnormal data in the data to be detected as the first amount of abnormal data, and, according to the second prediction data, determine the amount of abnormal data in the data hit by the dimension value combination in the data to be detected as the second amount of abnormal data; The impact degree corresponding to the dimensional value combination is determined according to the proportion of the second abnormal data volume in the first abnormal data volume.

3. The method according to claim 1, determining the discrimination degree corresponding to the dimension value combination according to the data to be detected, comprising: Predicting data that should be generated within a time period for generating the data to be detected as first predicted data, and predicting data that should be generated by the combination of dimension values ​​within the time period as second predicted data; Determine the actual data distribution according to the proportion of the data hit by the dimension value combination in the data to be detected in the data to be detected, and determine the standard data distribution according to the proportion of the second predicted data in the first predicted data; The discrimination degree is determined according to the actual data distribution and the standard data distribution.

4. The method according to claim 2 or 3, predicting data to be generated within a time period of generating the data to be detected as first predicted data, and predicting data to be generated by the dimension value combination within the time period as second predicted data, comprising: Acquire data in a time period adjacent to the time period as reference data; The first prediction data and the second prediction data are obtained according to the reference data.

5. The method according to claim 1, determining the target dimension value combination according to the influence degree and discrimination degree corresponding to each dimension value combination, comprises: For each determined dimension combination, according to the influence degree corresponding to each dimension value combination contained in the dimension combination, the candidate dimension value combination under the dimension combination is screened out, where: The dimension combination includes at least two data dimensions; Determine the target dimension value combination based on the discrimination corresponding to each candidate dimension value combination.

6. The method according to claim 5, screening out candidate dimension value combinations under the dimension combination according to the influence degree corresponding to each dimension value combination contained in the dimension combination, comprises: The dimension value combinations included in the dimension combination are sorted from large to small according to the degree of distinction, and the dimension value combinations whose influence degree is not less than the determined preset influence degree are sequentially added to the set corresponding to the dimension combination in the order of sorting, until the total influence degree corresponding to the dimension value combinations added to the set is not less than the determined preset total amount, thereby obtaining the target set corresponding to the dimension combination; The dimension value combination in the target set is used as a candidate dimension value combination under the dimension combination.

7. The method according to claim 6, wherein determining the preset impact degree and the preset total amount comprises: Predicting data that should be generated within a time period of generating the data to be detected as first predicted data, and determining an amount of abnormal data in the data to be detected based on the first predicted data as a first amount of abnormal data; The preset impact degree and the preset total amount are determined according to the first abnormal data amount.

8. A device for locating anomalies, comprising: An acquisition module, used to acquire data to be detected, and to combine the dimensional values ​​contained in different data dimensions to obtain dimensional value combinations, wherein each dimensional value combination contains at least two dimensional values, and each dimensional value comes from a different data dimension, the data to be detected includes transaction data related to statistical indicators within a period of time, and the data dimensions include relevant data under different dimensions involved in executing transaction business; A determination module, for determining, for each dimensional value combination, the degree of influence and the degree of discrimination corresponding to the dimensional value combination according to the data to be detected, wherein the degree of influence is used to characterize the degree of influence of the abnormality of the data hit by the dimensional value combination in the data to be detected on the abnormality of the entire amount of the data to be detected, and the degree of discrimination is used to characterize the degree of difference between the actual data distribution of the data hit by the dimensional value combination in the data to be detected and the standard data distribution corresponding to the dimensional value combination, and the standard data distribution represents the probability of the data generated by the dimensional value combination under ideal circumstances appearing in the overall data; The positioning module is used to determine the target dimension value combination according to the influence degree and discrimination degree corresponding to each dimension value combination, and to locate the anomaly according to the target dimension value combination.

9. A computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program.

Images